A circuit breaker wraps calls to a dependency and keeps a running verdict on whether that dependency is healthy. While things are fine, calls pass through. When failures cross a threshold, the breaker “opens” and rejects calls immediately for a while, without touching the network. After that cool-down, it lets a few test calls through to see whether the dependency has recovered.
Why bother? Because the alternative is every caller waiting for its full timeout on every request to a service that’s down. If an order service calls inventory with a 2-second timeout and inventory is dead, every order request takes 2 seconds and then fails. Threads pile up, the order service’s own latency goes through the roof, and its callers start timing out too. A breaker turns 2 seconds of waiting into a 50-microsecond rejection, which protects the caller and gives the struggling dependency room to recover instead of being hammered by retries.
Three states, and what moves between them
- Closed. Normal operation. Calls go through, and each result is recorded in a sliding window (the last N calls, or the calls in the last N seconds).
- Open. The failure rate in the window crossed the threshold. Every call fails fast for a fixed wait duration. No traffic reaches the dependency.
- Half-open. The wait expired. A small, fixed number of probe calls go through. If they succeed, the breaker closes and the window starts fresh. If one fails, it opens again for another full wait.
The detail candidates often miss is the minimum number of calls. Without it, the very first request after a deploy fails, the failure rate is 100%, and the breaker trips on one error. Every serious implementation, Resilience4j included, waits until the window has enough calls to make the rate meaningful.
A breaker you can run, with a fake clock
Here’s a count-based breaker in about 40 lines of JavaScript. The clock is injected so the run is deterministic and the timeline is readable:
class CircuitBreaker {
constructor({ windowSize = 10, minCalls = 5, failureRate = 0.5, openMs = 30_000, probes = 2, now = Date.now }) {
Object.assign(this, { windowSize, minCalls, failureRate, openMs, probes, now });
this.state = 'CLOSED'; this.window = []; this.openedAt = 0; this.probeResults = []; this.inFlightProbes = 0;
}
async call(fn) {
if (this.state === 'OPEN') {
if (this.now() - this.openedAt < this.openMs) throw new Error('circuit open: fail fast');
this.transition('HALF_OPEN');
}
if (this.state === 'HALF_OPEN') {
if (this.inFlightProbes >= this.probes) throw new Error('circuit half-open: probe slots busy');
this.inFlightProbes++;
}
try { const v = await fn(); this.record(true); return v; }
catch (e) { this.record(false); throw e; }
}
record(ok) {
if (this.state === 'HALF_OPEN') {
this.inFlightProbes--; this.probeResults.push(ok);
if (!ok) return this.trip();
if (this.probeResults.length >= this.probes) { this.window = []; this.transition('CLOSED'); }
return;
}
this.window.push(ok);
if (this.window.length > this.windowSize) this.window.shift();
const failures = this.window.filter((x) => !x).length;
if (this.window.length >= this.minCalls && failures / this.window.length >= this.failureRate) this.trip();
}
trip() { this.openedAt = this.now(); this.probeResults = []; this.transition('OPEN'); }
transition(to) { console.log(` t=${String(this.now() / 1000).padStart(4)}s ${this.state} -> ${to}`); this.state = to; }
}The script makes one call per second to a fake inventory service. Three succeed, then inventory starts returning 503. Then it jumps the clock forward to show the open period, a failed probe, and a successful recovery. Output from Node 22 (indented lines are state transitions):
t= 0s call: success
t= 1s call: success
t= 2s call: success
t= 3s call: 503
t= 4s call: 503
t= 5s CLOSED -> OPEN
t= 5s call: 503
t= 6s call: circuit open: fail fast
t= 7s call: circuit open: fail fast
t= 20s call: circuit open: fail fast
t= 39s OPEN -> HALF_OPEN
t= 39s HALF_OPEN -> OPEN
t= 39s probe: 503
t= 70s OPEN -> HALF_OPEN
t= 70s probe: success
t= 70s HALF_OPEN -> CLOSED
t= 70s probe: success
t= 71s call: successWalk through it the way you would at a whiteboard:
- At t=3s and t=4s the window holds 3 successes and 2 failures: 40%, below the 50% threshold. Still closed.
- At t=5s the third failure makes it 3 of 6, which is 50%, and the window has at least 5 calls. The breaker trips.
- From t=6s to t=20s, calls fail fast. Inventory receives nothing, which is the point: if it’s overloaded, this is how it gets breathing room.
- At t=39s the 30-second wait has passed (it opened at 5s, so 35s would have been enough). The first call becomes a probe. Inventory is still broken, so the breaker reopens and the wait restarts from 39s.
- At t=70s two probes succeed, the breaker closes with an empty window, and normal traffic resumes.
Choosing what counts as a failure
This is where real configurations go wrong. A breaker that counts the wrong errors either never trips or trips for nonsense.
- Count: connection refused, timeouts, 502/503/504, and 500s that indicate the dependency is unhealthy. Also count calls slower than a threshold. Resilience4j has
slowCallRateThresholdfor this, because a dependency that answers in 9 seconds is functionally down even if it eventually returns 200. - Don’t count: 400, 404, 409, 422. A user sending an invalid coupon code isn’t evidence that the coupon service is broken. If those count, a burst of bad input from one client opens the breaker for everyone.
In Spring Boot with Resilience4j, that intent looks like this (illustrative):
resilience4j:
circuitbreaker:
instances:
inventory:
slidingWindowType: COUNT_BASED
slidingWindowSize: 20
minimumNumberOfCalls: 10
failureRateThreshold: 50
slowCallDurationThreshold: 2s
slowCallRateThreshold: 80
waitDurationInOpenState: 30s
permittedNumberOfCallsInHalfOpenState: 3
ignoreExceptions:
- com.example.inventory.InvalidSkuExceptionWhat the caller does when the breaker is open
An open breaker throws (in Resilience4j, CallNotPermittedException). You have to decide what that means for the user, and the right answer differs per call:
- Optional data: return a default. No recommendations, an empty “recently viewed” list.
- Cached data: serve the last known value with a staleness marker. Product details from 10 minutes ago are usually fine.
- Required for correctness: fail the request clearly. If the payment breaker is open, return 503 with
Retry-After. Never invent a “payment succeeded” fallback. - Deferrable work: accept the request and queue it. “Your order is placed, confirmation will follow” if the downstream step can happen later.
A fallback that quietly returns wrong data is worse than an error. “Stock unknown, assume available” can oversell a product nobody has.
Misconceptions that come up in interviews
“The breaker lives in the dependency.” No, it lives in the caller. Each calling service (often each instance) keeps its own view. That means 30 order-service pods each have their own breaker and trip at slightly different times, which is normal.
“Breakers replace retries.” They work together. Retries handle brief blips; the breaker stops retries from continuing to hammer a dependency that’s clearly down. Put the retry inside the breaker’s view of the world carefully: if each retry counts as a separate call, three retries per request will trip the breaker three times as fast, which may or may not be what you want.
“Half-open lets all traffic through.” It shouldn’t. If 2,000 queued requests rush a recovering service, you knock it over again. Limit the probes.
“One breaker per service is enough.” Often you want one per dependency per operation. If inventory’s /reserve endpoint is broken but /availability is fine, a single shared breaker blocks both.
Interview practice: the breaker that never opened
“Your order service has a circuit breaker on calls to the pricing service with a 50% failure threshold. During an incident, pricing took 25 to 30 seconds per request but eventually returned 200. The breaker stayed closed the whole time and the order service ran out of threads. Why, and how would you fix it?”
Answer
The breaker only counted errors, and pricing wasn’t returning errors. Slow successes looked healthy, so the failure rate stayed near zero. Meanwhile each order request held a thread for 25+ seconds and the pool drained.
Two changes. First, a call timeout well below 25 seconds, chosen from pricing’s normal p99, say 800 ms. A timeout converts a slow call into a failure the breaker can count, and it frees the thread. Second, a slow-call threshold on the breaker itself (for example, calls over 1 second count as slow, open if 80% are slow), so the breaker trips on degradation even if the timeout is generous. I’d also put a bulkhead on pricing calls so even before the breaker trips, pricing can’t take every thread, and decide on the fallback: probably cached prices with a short TTL, since quoting a stale price for a few minutes is safer than failing every checkout.