An incident is a live production problem, handled with urgency; a postmortem is the honest, blameless write-up afterward.
Why it exists
When a serious outage happens, reacting is not enough; without learning from it the same failure returns, and blame fixes nothing. Incidents and postmortems exist to handle a live problem and then examine it honestly afterward, so the team turns each failure into a concrete change that prevents the next one.
How it actually works
When production breaks, it becomes an incident: severity is set, people gather, communication goes out, and the priority is restoring service (often via rollback or a flag). Once it's over, the team writes a postmortem: what happened, why, and what will prevent it next time.
The best postmortems are blameless, focused on fixing the system, not punishing a person, because the goal is learning, and blame just makes people hide mistakes.
A senior PM walks you through it
A junior PM, stuck
The postmortem for Friday's checkout incident just closed and my lead asked me to capture the action items as roadmap tickets. There are four of them and I cannot tell which ones actually prevent a repeat and which are just cleanup. If I file all four as equal tickets I know that is wrong, but I cannot defend cutting any of them either.
That action item list is really three different kinds of work wearing one heading: a fix, some system changes, and a piece of paperwork. There is a one-question test that separates them in about two minutes, and the doc itself gives you everything the test needs. This is the actual postmortem from Friday, the whole doc, nothing paraphrased. Read it top to bottom once, then take the steps with me.
Postmortem: checkout 500s after v2.4.1, Friday 2026-03-06
Summary
Severity: high. Checkout failing during the lunch peak, traffic at about 6x normal.
Impact window: 13:04 to 13:19 on Friday 2026-03-06, 15 minutes of checkout 500s.
Status: resolved by rollback to v2.4.0. Corrected release planned as v2.4.2.
Owner: Nadia (on-call).
Timeline
12:30lunch peak begins, orders climbing toward 38 per minute
13:04v2.4.1 reaches production
13:04checkout error rate spikes, a wall of 500s
13:04Nadia in the incident channel: "checkout is throwing 500s, roll it back"
13:07 to 13:15customer u_88231 charged twice, orders o_5512 and o_5513, 649 taka each; tracked in a separate root-cause thread
13:19rollback to v2.4.0 completes, error rate back to normal
14:00lunch peak ends with no further errors
Root cause and action items
Root cause: checkout's calls to the payment gateway timed out under peak load and surfaced as 500s.
The timeouts began the minute v2.4.1 reached production and stopped when the v2.4.0 rollback completed.
Contributing factor: the full rollout landed at 13:04, mid-peak, with traffic at about 6x normal.
AI-1. Ship the corrected checkout change as v2.4.2 through the normal 10% canary. Owner: Rafi.
AI-2. Freeze full rollouts during the lunch peak, 12:30 to 14:00. Owner: Nadia.
AI-3. Alert automatically when checkout's error rate spikes, instead of relying on whoever is watching the channel. Owner: Rafi.
AI-4. Write a plain-language summary of the incident for stakeholders. Owner: product.
Click a step to see the lines it points at.
Mistakes I've seen
Filing all four action items as one "postmortem follow-ups" ticket. The easy cleanup gets done, the deploy freeze quietly never ships, and the next mid-peak release replays Friday.
Reading severity as duration. Fifteen minutes looks minor until you see it sat inside the lunch peak at 6x traffic; the same failure at 3:07am, when nobody is watching, runs far longer before anyone notices.
Accepting "be more careful with releases" as an action item. No owner, no system change, nothing to ticket, and by the next incident it never existed.
Skipping the timeline because the summary already says what happened. The timeline is the proof; when someone pushes back on the peak-hours freeze, 13:04 sitting next to the release line is your whole argument.
In the ticket triage, say: "AI-2 and AI-3 go on the roadmap, they are the ones that change the system. The v2.4.2 fix is tracked with the release, and I will handle the stakeholder summary as a task." You just sorted action items by whether they would have changed Friday, and that test works on every postmortem you will ever read.
Where a PM meets this
PMs are often part of incident response, especially communication to stakeholders and customers.
Postmortem action items feed the roadmap; "we're adding a check so this can't recur" is real work you'll help prioritize.
Hear it in a meeting
"Postmortem's Thursday, let's capture the action items as tickets."