“We have a 30-second timeout” sounds like a complete answer until someone asks which timeout. A single HTTP call goes through several phases, and a server can hang in any of them: it might never accept the connection, accept it and never reply, send headers and stall halfway through the body, or keep sending one byte at a time. Different timers catch different cases, and some cases slip past a timer that looks like it should have caught them.
Interviewers use this topic to see whether you can reason about failure rather than recite settings. Everything below was run against a small Node server on localhost built to stall in each way.
The phases of one request
- DNS lookup. Usually fast and cached. A broken resolver can add seconds per attempt.
- TCP connect. One round trip when the host answers. When packets are silently dropped, the kernel keeps retrying the SYN; on Linux, the default gives up after roughly two minutes.
- TLS handshake. Another round trip plus certificate checks.
- Waiting for response headers. The request is sent and the server is working. This is where slow database queries, lock waits and overloaded threads show up.
- Reading the body. Headers arrived, but the rest can stall or crawl.
On top of all that sits the operation: what your caller actually cares about, which can include several attempts.
A connect timeout is the easiest to see. Pointed at a documentation address that’s never routed:
curl -sS --connect-timeout 2 http://192.0.2.1/curl: (28) Failed to connect to 192.0.2.1 port 80 after 2006 ms: Timeout was reachedWithout --connect-timeout, the same command waits for the operating system to give up.
Three ways a connected server can hang
This server accepts connections normally and then misbehaves in three different ways:
import http from 'node:http';
http.createServer((req, res) => {
if (req.url === '/stall-headers') return; // accept the request, never answer
if (req.url === '/stall-body') {
res.writeHead(200, { 'content-type': 'text/plain' });
res.write('first part\n'); // then nothing
return;
}
if (req.url === '/drip') {
res.writeHead(200, { 'content-type': 'text/plain' });
const timer = setInterval(() => res.write('.'), 500); // one byte every 500 ms, forever
req.on('close', () => clearInterval(timer));
return;
}
res.end('ok\n');
}).listen(8087, '127.0.0.1');curl’s --max-time is a total limit on the whole transfer, and it catches all three:
for p in stall-headers stall-body drip; do curl -sS -o /dev/null --max-time 2 http://127.0.0.1:8087/$p; donecurl: (28) Operation timed out after 2006 milliseconds with 0 bytes received
curl: (28) Operation timed out after 2002 milliseconds with 11 bytes received
curl: (28) Operation timed out after 2004 milliseconds with 3 bytes receivedThe byte counts tell you which phase hung. Zero bytes means the server never answered: it’s stuck before writing anything. Eleven bytes means it started a response and stopped, which often means a streaming job or a downstream call failed partway. Three bytes in two seconds means something is alive but far too slow.
Idle timers versus deadlines
Most client libraries offer an idle (or socket) timeout: fire if no bytes arrive for N milliseconds. Node’s http.request timeout option is one of these. Here it is with one second, against each endpoint:
const req = http.get({ host: '127.0.0.1', port: 8087, path, timeout: 1000 }, onResponse);
req.on('timeout', () => req.destroy(Object.assign(new Error('idle'), { code: 'IDLE_TIMEOUT' })));idle 1000 ms, /stall-headers: IDLE_TIMEOUT after 1005 ms
idle 1000 ms, /stall-body : IDLE_TIMEOUT after 1003 ms
idle 1000 ms, /drip : still running after 6000 msThe drip endpoint sends a byte every 500 ms, so the socket is never idle for a full second and the timer keeps resetting. The test script gave up at six seconds; the request itself would have run forever. A slow, steady trickle (a struggling upstream, a congested link, or a deliberately slow client) defeats idle timeouts completely.
A deadline bounds the whole operation. With fetch, that’s an abort signal, and it covers reading the body as well as the headers:
const res = await fetch(`http://127.0.0.1:8087${path}`, { signal: AbortSignal.timeout(2000) });
const text = await res.text();deadline 2000 ms, /drip : TimeoutError: The operation was aborted due to timeout (2010 ms)
deadline 2000 ms, /stall-body: TimeoutError: The operation was aborted due to timeout (2004 ms)A third kind is a minimum transfer rate, which curl offers directly:
curl -sS -o /dev/null --speed-limit 100 --speed-time 3 http://127.0.0.1:8087/dripcurl: (28) Operation too slow. Less than 100 bytes/sec transferred the last 3 secondsThat’s useful for large downloads, where a total deadline would have to be generous but a stall should still fail fast.
The misconception interviewers listen for is assuming a library’s “timeout” setting means a total limit. Often it’s idle-only, or covers only the headers. Defaults vary too: Node’s http.request has no timeout at all, and the fetch built into Node waits up to five minutes for headers and five more between body chunks. Read the docs for the exact client you use, and add your own deadline.
Choosing the numbers
Timeouts should come from two measurements: what the caller can afford, and how the dependency normally behaves.
- Connect: a healthy connect takes about one round trip. Inside a data centre that’s under a millisecond, so a connect timeout of a second or less is generous and fails fast when a host is gone.
- Waiting for headers: start from the dependency’s p99 or p99.9 latency and add margin. If it answers in 120 ms at p99, a 30-second timeout only means you’ll wait 30 seconds when it’s broken.
- Total deadline: from the caller’s budget. If the user-facing endpoint promises an answer in 2 seconds, nothing beneath it can wait longer than what’s left of those 2 seconds.
Retries have to fit inside the same budget. Three attempts with a 10-second timeout each means a caller can wait 30 seconds, and during an outage every client retrying at once triples the load on a dependency that’s already failing. Pass the remaining time down instead: compute a deadline at the edge, give each downstream call what remains (minus a little for your own work), and only retry if enough time is left for the attempt to succeed. gRPC does this natively by sending the deadline with each call.
A timeout isn’t a failure report
When the client gives up, the server usually doesn’t know. A POST /transfers that times out at the client after 2 seconds might commit at 2.3 seconds. If the client treats the timeout as “failed” and retries, the money moves twice. Treat a timeout as “outcome unknown”: retry only idempotent requests, or send an idempotency key so the server can recognise the repeat, or check the operation’s status before trying again.
Servers need their own timeouts for the opposite reason. A client that sends headers one byte at a time can hold a connection open indefinitely, which is the idea behind slowloris attacks. Node’s server has headersTimeout (60 seconds by default) and requestTimeout (5 minutes) to cap that.
Interview practice
Your product page calls an inventory service. During an incident, inventory responses slowed from 80 ms to over a minute. Your service’s connection pool and worker threads all filled up waiting, and the whole site went down, including pages that don’t need inventory. Design the timeout and retry policy you’d put in place.
Answer
The outage happened because waiting was unbounded, so a slow dependency used up resources every page shared. I’d give the inventory client a connect timeout of about 500 ms and a total deadline per call derived from the page’s budget. If the page should render within 1.5 seconds and inventory’s p99 is 150 ms, a deadline of around 400 ms leaves room for one retry and for the rest of the page. The deadline must cover the body, not just the headers, so a trickling response can’t hold a thread.
I’d allow at most one retry, only for idempotent reads, only if the remaining budget covers another attempt, and with jitter so clients don’t retry in sync. I’d also cap concurrent calls to inventory with a small dedicated pool, so even if every call times out, other pages keep their resources. Finally I’d decide what the page does when inventory fails: show “check availability” instead of erroring. Timeouts protect the caller, but only a fallback turns a slow dependency into a degraded page instead of a broken one.