Skip to content
Velloreth
Technology 4 min read

Load Balancing Algorithms: Round Robin to Least Connections

A load balancer spreads traffic across backends. Learn the algorithms, the health check, the session affinity, and the layer.

By

The purpose

Spread traffic across several backends.

Remove a failed backend from the pool.

Provide a single address for the clients.

The algorithms

AlgorithmHowBest for
Round robinthe next in turnsimilar requests
Weighted round robinby a weightunequal servers
Least connectionsthe fewest activevaried requests
Least response timethe fastesta latency goal
IP hashby the source IPaffinity
Randoma random picka large pool
Consistent hashby a keya cache

Round robin is simple and works when the requests are similar.

Least connections adapts when the requests take different times.

Round robin

Request 1 -> backend A
Request 2 -> backend B
Request 3 -> backend C
Request 4 -> backend A

Simple and predictable.

A weighted variant sends more to a larger server.

A: weight 3
B: weight 1
-> A, A, A, B

Least connections

The request goes to the backend with the fewest active connections.

It adapts to a slow request, because a backend that is busy receives less.

It requires the balancer to track the connection count, which is a small cost.

The health check

health_check:
  path: /health
  interval: 5s
  timeout: 2s
  unhealthy_threshold: 3
  healthy_threshold: 2

A backend that fails the check is removed.

It is added back when it passes.

Without a health check, traffic continues to a failed backend.

A shallow check tests that the process is up, and a deep check tests a dependency.

A deep check that is too aggressive can remove every backend when a shared dependency fails.

Session affinity

Pinning a client to one backend.

The methods:

  • A cookie set by the balancer
  • The source IP hash
  • A header

It is a workaround for a stateful application.

It limits the scaling, because a backend that holds a popular session is loaded.

The better fix is a shared session store.

Layer 4 versus layer 7

Layer 4Layer 7
Routes byIP and portthe HTTP request
Speedfasterslightly slower
TLSpassthroughtermination
Routinglimitedby path, host, header
Rewritenoyes

A layer 4 balancer is fast and protocol-agnostic.

A layer 7 balancer reads the request, so it can route by the path and terminate TLS.

The TLS

Termination at the balancer means the balancer holds the certificate and the backend sees plain HTTP.

Passthrough means the balancer forwards the encrypted bytes and the backend holds the certificate.

Re-encryption means the balancer terminates and opens a new connection to the backend.

The choice depends on the trust in the internal network and the need for the client certificate.

The draining

A backend that is being removed must drain.

A connection drain waits for the active requests to finish.

A deploy should drain before the process stops, or a request is dropped.

A long-lived connection such as a WebSocket needs a graceful close.

The failure modes

A backend that is slow but not failed is the hardest case, because a health check passes.

A timeout is the guard, and it must be shorter than the client’s.

A retry on another backend is safe for an idempotent request.

A retry storm can amplify a failure, so the retries must be bounded.

Common mistakes

No health check. Traffic to a failed backend.

A health check that is too shallow. A broken backend stays in.

A health check that is too deep. Every backend is removed.

Session affinity by default. A scaling limit.

No connection drain. A dropped request on a deploy.

A retry without a bound. A retry storm.

A timeout longer than the client’s. A wasted request.

One balancer. A single point of failure.

No metrics. The distribution is unknown.

A layer 7 balancer for a non-HTTP protocol. It does not work.

A quotable summary

A load balancer spreads traffic across several backends, removes a failed one with a health check, and provides a single address for the clients. Round robin is simple and least connections adapts to the load, session affinity pins a user to one backend and limits the scaling, and a layer 4 balancer is faster while a layer 7 balancer can route by the path and terminate TLS. Drain the connections before a deploy, and bound the retries so a failure is not amplified.

For the related topics, see reverse-proxy-explained and caching-strategies-guide. For the broader context, nginx-basics and how-cdns-actually-work.

Frequently asked questions

What is a load balancer?
A component that receives traffic and distributes it across several backends, so no single server is overloaded and a failed server can be removed. It also provides a single address for the clients, which hides the pool behind it.
What is the difference between round robin and least connections?
Round robin sends each request to the next backend in turn, which is simple and works when the requests are similar. Least connections sends the request to the backend with the fewest active connections, which adapts when the requests take different times.
What is a health check?
A periodic request to a backend to confirm it is working, such as a request to a health endpoint. A backend that fails the check is removed from the pool, and it is added back when it passes. Without a health check, traffic continues to a failed backend.
What is session affinity?
Pinning a client to one backend, usually with a cookie or the source IP, so the session state is available. It is a workaround for a stateful application, and it limits the scaling because a backend that holds a popular session is loaded. The better fix is to move the state to a shared store.
What is the difference between layer 4 and layer 7?
A layer 4 balancer routes by the IP address and the port, so it is fast and protocol-agnostic. A layer 7 balancer reads the HTTP request, so it can route by the path, the header or the host, and it can terminate TLS and rewrite. Layer 7 is more flexible and slightly slower.
What is a sticky session problem?
A user is pinned to a backend, and that backend fails, so the session is lost. It also means a deploy must drain the connections rather than drop them. The fix is to store the session in a shared store such as Redis, so any backend can serve it.