Skip to main content
Architecture & Engineering

Load Balancing

Load balancing distributes incoming requests across multiple servers so no single instance becomes a bottleneck, and so the failure of one instance does not take the service down. The balancer also health-checks its targets, removing unhealthy instances from rotation automatically.

Beyond distributing load, the health-checking behaviour is what turns a set of servers into a resilient service. An instance that stops responding is taken out of rotation without human involvement, which converts a class of incidents into a graceful degradation.

The design decision that catches teams out is session state. If a user's session lives in one server's memory, requests must keep returning to that server, which undermines both balancing and failover. Storing session state externally — in Redis or a database — makes any instance able to serve any request, which is the property that makes horizontal scaling actually work.

Codazz builds this in production — Performance & Scaling.

FAQ

Load Balancing
FAQ.

Common questions about load balancing.

Ask Us Anything

Sticky sessions bind a user to the server that first handled them, so in-memory session data remains available. It works, but it undermines even distribution and means a server failure logs those users out. Externalising session state to a shared store is almost always the better answer.