Load balancers: route, detect and drain

L4 balances transport flows without needing HTTP semantics. L7 understands the application protocol and can route /images separately from /checkout. Routing encrypted HTTP fields requires terminating or otherwise accessing decrypted traffic.

Round robin gives targets turns. It is simple when capacities and request costs are similar. Weighted round robin adjusts shares for unequal capacity; equal request counts still do not guarantee equal CPU time.

Least-connections favors the target with fewer active connections. Power-of-two-choices samples two eligible targets and picks the less loaded by a chosen metric. Sampling reduces selection work. With HTTP/2 multiplexing, one connection may contain many requests, so choose the metric carefully.

Consistent hashing is useful for cache locality and affinity. It does not divide a single hot key across targets. Power-of-two-choices responds to load but may send the same key to different targets. Choose whether locality or dynamic balance matters more.

Active checks probe readiness on a schedule. Thresholds avoid flapping but delay removal. This example assumes negligible probe duration; timeouts and propagation add delay. A shallow health endpoint may pass while real requests fail, so test critical serving dependencies without making checks overload them.

Sticky sessions trade routing freedom for locality. Heavy users can overload one target. If state lives only in that process, affinity cannot save it after a crash. Keep required session state outside disposable instances, or explicitly accept session loss.

Draining separates admission from completion. Stop new assignments before terminating. Bound the wait so stuck requests cannot block deployment forever. Long-lived WebSockets and streams need an explicit reconnect or shutdown policy; they may outlast a normal drain timeout.