A traffic director that sits in front of several servers and spreads incoming requests across them.
Why it exists
Once traffic outgrows one server you run several, but clients still need one address and no single server should be swamped while others sit idle. A load balancer exists to sit in front and spread incoming requests across the healthy servers, rerouting automatically when one fails.
How it actually works
Once you have many servers, something must decide which one handles each request. The load balancer is that front door: every request hits it first, and it forwards each to whichever server has room, spreading the load evenly.
It also notices when a server dies and stops sending traffic there, so one failed machine doesn't take the product down. Users never see it; they just reach a healthy server every time.
Kill a server and predict where its traffic goes.
Server 1
0 handled
Server 2
0 handled
Server 3
0 handled
dispatched 0 · routing to 3 of 3 servers · round robin
A load balancer spreads requests so no single server is overwhelmed, and reroutes to the survivors when one dies.
A senior PM walks you through it
A junior PM, stuck
One of our app servers died during Friday lunch rush and nobody noticed until hours later. Now I have to explain in the incident review why customers felt nothing, and I don't actually know. Ops sent me the access logs from all three servers and to me they are just three files of timestamps.
Those three files are the whole answer, and reading them takes about five minutes. Each app server writes a line for every request it handles, so laid side by side the three logs show you exactly how the balancer spread the lunch rush, and exactly when server-2 went quiet. I pulled the same stretch from all three, starting at the 12:41 peak, and trimmed it so we can read every line.
Access logs from the three app servers, Friday 2026-03-06, 12:41 to 12:44
server-1
12026-03-06T12:41:02Z INFO orders POST /api/orders 201 41ms
22026-03-06T12:41:05Z INFO orders GET /api/products 200 35ms
32026-03-06T12:41:09Z INFO checkout POST /api/orders 201 47ms
42026-03-06T12:41:14Z INFO orders GET /api/products 200 33ms
52026-03-06T12:41:21Z INFO orders POST /api/orders 201 44ms
62026-03-06T12:44:07Z INFO orders POST /api/orders 201 52ms
72026-03-06T12:44:09Z INFO orders GET /api/products 200 38ms
82026-03-06T12:44:12Z INFO checkout POST /api/orders 201 55ms
server-2
12026-03-06T12:41:03Z INFO orders GET /api/products 200 36ms
22026-03-06T12:41:07Z INFO checkout POST /api/orders 201 44ms
32026-03-06T12:41:12Z INFO orders GET /api/products 200 34ms
42026-03-06T12:41:18Z INFO orders POST /api/orders 201 43ms
52026-03-06T12:41:26Z INFO orders GET /api/products 200 37ms
62026-03-06T12:44:02Z INFO orders GET /api/products 200 39ms
server-3
12026-03-06T12:41:04Z INFO orders POST /api/orders 201 45ms
22026-03-06T12:41:08Z INFO orders GET /api/products 200 32ms
32026-03-06T12:41:13Z INFO checkout POST /api/orders 201 46ms
42026-03-06T12:41:19Z INFO orders GET /api/products 200 34ms
52026-03-06T12:41:24Z INFO orders POST /api/orders 201 42ms
62026-03-06T12:44:08Z INFO orders POST /api/orders 201 49ms
72026-03-06T12:44:10Z INFO orders GET /api/products 200 36ms
82026-03-06T12:44:13Z INFO checkout POST /api/orders 201 51ms
Click a step to see the lines it points at.
Mistakes I've seen
Reading only the dead server's log and concluding "nothing looks wrong here." Of course it doesn't; a dead server's log is clean right up to the silence. The evidence lives in the other servers' files.
Hearing "a server died" and announcing an outage to stakeholders. Behind a balancer those are different events; check whether any customer request actually failed before you use the word outage.
Treating "nobody noticed" as pure success in the review. Customers not noticing is the design working; the team not noticing for hours is a missing alert, and those are two separate findings.
Assuming the balancer fixes overload as well as failure. It spreads traffic, it does not add capacity; if all three servers are saturated, evening out the pain does not remove it.
In the review, say: "Server-2 stopped logging at 12:44 and the balancer's health check pulled it out of rotation; server-1 and server-3 absorbed its traffic and every request after that is still a 200 or 201, so no customer saw a failure. The gap is that we had no alert on a server going silent." You just read a system across three log files instead of one, and that is a skill most PMs never pick up.
Where a PM meets this
"Add more servers behind the load balancer" is the standard answer to a traffic surge; knowing this lets you follow capacity conversations.
Load balancers enable zero-downtime deploys (drain one server, update it, return it) and graceful failure, both product-relevant reliability wins.
Hear it in a meeting
"The balancer's spreading traffic across eight servers, we've got headroom."