Designing for Scale (Load Balancing & Statelessness)

How to take on more load without falling over: scale out instead of just up, make your servers stateless so any box can handle any request, put a load balancer in front, and push the parts that can't be cloned - sessions, the database, the cache - out to the edges.

1. Scale Up vs Scale Out, and Why Statelessness Matters Scaling up means a bigger box - simple, but capped and a single point of failure. Scaling out means more boxes - the real answer for big scale. The property that makes scaling out possible is statelessness: if any server can handle any request, you can add servers freely.
2. Load Balancing A load balancer sits in front of your pool of identical servers and spreads incoming requests across them, checks which ones are healthy, and stops sending traffic to the ones that aren't. The sticky-session trap re-introduces statefulness; externalizing state is the cleaner fix.
3. Scaling the Stateful Bits Statelessness lets you clone your app servers freely - but some things can't be cloned: sessions (move them to a shared store like Redis), the database (the usual bottleneck), and load you can shed with caching. Scaling is mostly the art of pushing state out to the edges so the middle can be cloned.