How a system copes with more load. Vertical means a bigger machine; horizontal means more machines working together.
Why it exists
A design that works for a hundred users can fall over at a hundred thousand, and the fix is rarely just a faster computer. Scalability exists as the property of handling more load by adding capacity, and as the vocabulary for whether you grow a machine (vertical) or add machines (horizontal).
How it actually works
Your one server handled 50 users fine. At 5,000 it chokes. There are two ways out. Vertical scaling: replace it with a more powerful machine, more CPU and memory. Simple, but there's a ceiling, and the biggest machine is expensive and still a single point of failure.
Horizontal scaling: add more ordinary machines and split the load across them. This scales much further and survives one machine dying, but it introduces a new problem, something must decide which machine each request goes to. That something is a load balancer.
Subtopics
A senior PM walks you through it
A junior PM, stuck
Marketing booked a Friday flash sale and engineering came back with one question: do I want to scale up or scale out. I said I would get back to them because I genuinely do not know the difference, or which one keeps the sale from falling over. I do not want to pick the wrong one and find out live.
Up versus out is a real, readable choice, and five minutes of numbers settle it. I captured the same one-minute flash-sale load two ways: our single server carrying it today, and the same traffic spread across three servers behind the balancer. Read them side by side and the two options stop being jargon.
One flash-sale minute, before and after scaling out, Friday 2026-03-06
today: one server at the flash-sale spike
112:41 server-1 cpu 97% mem 88% req/min 2100 p95 1400ms
212:41 server-1 503 Service Unavailable 47 requests dropped
312:41 server-1 cpu 100% no headroom left
scaled out: the same load across three servers behind the balancer
112:41 server-1 cpu 58% req/min 720 p95 120ms 0 dropped
212:41 server-2 cpu 61% req/min 700 p95 118ms 0 dropped
312:41 server-3 cpu 55% req/min 680 p95 121ms 0 dropped
412:41 balancer 2100 req/min spread three ways, all 200 OK
Click a step to see the lines it points at.
Mistakes I've seen
Answering "up or out" as if either just means "make it faster." Up buys a bigger single machine with a ceiling and a single point of failure; out adds machines and needs a balancer and stateless code. The trade-offs are real, there is no free default.
Promising a launch is safe because "we scaled the server." Vertical has a ceiling; if the sale exceeds the biggest machine you can rent, scaling up alone still falls over.
Forgetting horizontal needs stateless. If carts or sessions live in one server's memory, adding servers scatters users and breaks them; that is a code change, not just an infra one.
Treating a load balancer as extra capacity. It spreads load across servers you added; it adds no compute of its own, so putting a balancer in front without more servers does nothing for a spike.
Tell engineering: "For the flash sale let us scale out, not just up. One server hits 97% CPU and starts dropping requests at peak, and a bigger single machine still has a ceiling and one point of failure. Three servers behind the balancer keep each near 60%, as long as the app is stateless. Is it?" You named the difference between up and out and asked the one question that decides whether out is safe.
Where a PM meets this
"Can we handle the campaign traffic?" is a scaling question. The answer shapes whether a big launch is safe or risky.
Horizontal scaling requires the app to be stateless (see below); if it isn't, scaling out breaks things, which is why engineers care about that early.
Hear it in a meeting
"We'll scale horizontally for the sale, spin up ten servers behind the balancer."