When an interviewer says “Tell me about a production incident you were involved in,” most candidates tell a debugging story: the alert, the logs, the clever find, the fix. That’s part of it. For any role with on-call duties, the interviewer is also listening for something else: while you were debugging, who knew what was happening, and how did they find out?
Why communication is half the incident
During an outage, people outside the response need information as much as the responders need focus. Support is fielding confused customers. Account managers want to know whether to warn their biggest clients. Leaders want to know how bad it is. If nobody tells them, they ask the responders directly, which interrupts the debugging. Or they guess, which is worse.
An incident handled with good communication doesn’t feel calmer because it’s simpler. It feels calmer because everyone knows what’s affected, what’s being done and when they’ll hear more.
Roles that keep responders and stakeholders apart
Google’s SRE book describes separate roles for a serious incident. The incident commander holds the overall picture and assigns work. The operations lead is the person actually changing the system. A communications role is “the public face” of the response, issuing periodic updates and keeping the incident document current. A planning role handles longer-term tasks such as filing bugs and arranging handoffs.
Most teams don’t staff four roles for every outage. But the core idea holds even with two people: the person debugging shouldn’t also be the person answering everyone’s questions. In an interview, saying which role you held, and when you handed one off, shows you understand that.
What a good status update contains
A useful update answers four questions, in plain language:
- Impact. What’s broken, for whom, and since when, in terms a non-engineer understands.
- Known versus suspected. What you’ve confirmed and what’s still a theory.
- Current action. What the team is doing right now.
- Next update. When people will hear from you again, even if nothing changes.
Here’s an illustrative sequence for a checkout incident, posted to a company-wide incident channel:
14:10, investigating. Since about 13:50 UTC, roughly 1 in 5 card payments at checkout are failing with a payment error. PayPal checkouts look normal. We’re investigating; a deploy at 13:45 is one possible cause. Next update by 14:30.
14:28, mitigating. We’ve rolled back the 13:45 deploy and the failure rate is falling (8% at 14:25). We don’t know yet whether the deploy caused this or coincided with a database slowdown we’re also seeing. Next update by 14:45, sooner if anything changes.
14:52, monitoring. Card payment failures have been at normal levels (under 0.5%) for twenty minutes. The likely cause is a connection-pool setting in the 13:45 deploy; we’ll confirm in the review. For support: customers who saw a payment error between 13:50 and 14:30 were not charged, confirmed against the payment provider’s logs. Final update at 15:30.
Each update separates what’s confirmed from what isn’t, and none of them promises a resolution time. The support line in the third update answers the question customers are actually asking.
Worked example: telling the story in an interview
Here’s the same incident told as an interview answer, first weakly, then with the communication visible. Both are illustrative; build yours from an incident you were really part of.
Weak version (illustrative):
We had a big outage on checkout. I jumped on it, found the bad deploy, rolled it back and had it fixed within the hour. The business was really happy with how fast we moved.
There’s no role, no sense of who else was involved, and “fixed within the hour” skips the part where anyone outside engineering found out what was going on.
Stronger version (illustrative; adapt to your own incident):
I was on call when card payment failures spiked to about 20% of checkouts. I declared an incident within ten minutes, because it was clearly customer-visible, and paged a second engineer. When she joined, I handed her the investigation, since she knew the payment service better, and took over coordination and updates.
I posted the first update in the incident channel within a few minutes: what was failing, that PayPal was unaffected, that a recent deploy was one theory, and when the next update would come. I kept to roughly twenty-minute updates and was careful to separate what we’d confirmed from what we suspected, because the database team was also seeing slowness and I didn’t want anyone blamed before we knew. When the head of support asked whether customers had been charged, I got the payment provider’s logs checked before answering, rather than guessing.
After the rollback, the failure rate dropped, but I waited until it had been at normal levels for twenty minutes before calling it resolved. Afterwards I wrote the timeline for the incident review, which led to a check on connection-pool settings in our deploy pipeline.
The candidate’s role is clear, they handed off the work they weren’t best placed to do, and the communication choices (cadence, known versus suspected, checking before answering, waiting before declaring) are described as decisions, not luck.
Facts, theories and the word “fixed”
Three habits separate good incident communicators from nervous ones.
Don’t name a cause until it’s confirmed. “Possibly related to the 13:45 deploy” is honest. “The deploy broke it” can turn out to be wrong, and if another team gets blamed in a company-wide channel, that’s hard to take back.
Don’t say “fixed” until you’ve verified it against a clear standard, such as a metric back to normal for a set period. Systems often look better for a few minutes and then fail again.
Promise the next update, not the resolution. “We expect this resolved in thirty minutes” becomes a broken promise if it isn’t. “Next update at 14:30” is a promise you can always keep.
Mistakes that weaken the answer
- A pure debugging story. For on-call roles, leaving out communication misses half the question.
- Long silences. An hour without an update makes people assume the worst.
- Blaming early. Naming another team or person before the review looks careless.
- Declaring victory too soon. Then having to reopen the incident.
- Too much detail for the audience. Stack traces belong in the responders’ channel, not the company update.
Follow-up questions to expect
“When do you declare an incident?” The SRE book suggests some simple tests: does it need a second team, is it visible to customers, or is it still unsolved after an hour of focused work? It also says declaring early and closing quickly is better than starting the process hours too late.
“How did you handle someone senior asking for updates every five minutes?” Point them to the channel and the next update time, and if you’re the communications person, absorb those questions yourself so the responders can keep working.
“What changed after the review?” Name one concrete change, such as a pipeline check or a runbook update.
Interview practice: write the first update
At 10:05 you’re alerted that password-reset emails aren’t arriving. About 30% of reset requests since 09:40 have no delivery record, and the email provider’s status page shows no problems. Write the first update for the company incident channel.
Answer and reasoning
An update along these lines covers what’s needed:
10:12, investigating. Since about 09:40, around 30% of password-reset emails are not being delivered. Customers who request a reset may not receive it; they can retry, but some retries will also fail. Sign-in for customers who know their password is not affected. The email provider reports no problems, so we’re checking our own sending service first. Next update by 10:35.
It describes impact in customer terms and says what isn’t affected, which stops people from assuming everything is down. It states the one confirmed fact about the provider without claiming a cause. It gives support something to tell customers (retrying may help, but not always). And it commits to a next update time instead of a fix time.
What would weaken it: guessing at a cause (“the provider is down”), technical detail nobody outside engineering needs, or no next update time, which guarantees a stream of direct messages to the responders.
Further reading on incidents
- Behavioral and HR interview questions
- Behavioral and HR multiple-choice questions
- Behavioral Interviews: Failure and Specific Learning, for when the incident was partly your mistake
- Behavioral: Communicating on Remote Teams
- Google SRE book, Chapter 14: Managing Incidents