A small, very fast storage layer holding copies of answers already computed, so the next identical request skips the work.
Why it exists
Some answers are expensive to produce: a database query that scans millions of rows, or a call to a slow outside service. When the same answer is requested thousands of times a minute and the underlying data has not changed, paying that full cost every single time is pure waste. A cache exists to hold a copy of a recent answer somewhere fast to read, so repeat requests come back in a couple of milliseconds instead of hundreds, and the expensive work runs only when the data actually changes.
How it actually works
Reading from a database means disk and query work; reading from memory is hundreds of times faster. A cache exploits this. When a request arrives ("give me product 1042"), the backend checks the cache first. If the answer's there (a cache hit), it returns in under a millisecond. If not (a miss), the backend does the real work, returns it, and stores a copy so the next identical request hits.
The hard part is invalidation: if product 1042's price changes, the cache still holds the old price until it expires or is deleted. Teams handle this by expiring quickly, by deleting the entry when data changes, or by accepting brief staleness where it's harmless. Which one is a real product decision, trading freshness against speed and cost.
Request the same product twice. Predict how long the second one takes.
hits 0 · misses 0 · hit rate 0%
No requests yet.
Repeat reads are served from cache in a couple of milliseconds. The first read, and every read after the data changes, still pays the full database cost.
Subtopics
A senior PM walks you through it
A junior PM, stuck
A restaurant updated Kacchi's price in their dashboard an hour ago, but customers still see the old price on the menu. The engineer just shrugged and said "it's cached." I do not want to spec a fix until I understand why a saved price change does not show up.
A cache is a fast copy of a recent answer that the server keeps so repeat requests skip the database. The catch is that the copy does not know when the real data changed. You do not need the theory; you need to read one header. Here are two identical requests plus a third after the price change, and the X-Cache line tells you which answer is fresh work and which is a stale copy.
GET /api/products/1042 three times: a miss, a hit, and a stale hit after the price changed
Reading a stale price as a lost write. An X-Cache: HIT with the old number means the save worked and the cache never heard about it; the fix is invalidation, not re-saving.
Filing it as a frontend bug. The wrong price is coming straight from the response body, so the screen is faithfully showing stale cached data, not drawing it wrong.
Asking to "make it always real-time." That throws away the hit rate that keeps the menu fast and the database alive; the right lever is a shorter expiry or an invalidation rule on price changes.
Not asking how long stale is tolerable. A five-minute-old price on a rarely-changed item is fine; a stale price during a live promo is not. That tolerance is a decision only the PM can make.
Tell the engineer: "So the save landed but the cached copy still serves 649 on a HIT. Can we invalidate product 1042 the moment its price changes, or set a short expiry, so a price edit shows up within seconds instead of when the cache happens to age out?" You read the header, named it as invalidation, and asked for the real fix.
Where a PM meets this
A user updates their address and the old one shows for a minute: cache staleness, fixed with an invalidation rule, not a redesign.
An engineer resisting "make it fully real-time" is protecting a high hit rate that keeps the page fast and the database alive.
Hear it in a meeting
"Homepage is 99% cache hits; it's the pricing API hammering the database."