Measuring latency in Node looks like a two-line job: record a time before, record a time after, subtract. The numbers you get are precise and often wrong, not because the arithmetic fails but because of what you chose to measure, how you summarised it, and how the load was generated. Advanced interviews probe this because a performance claim (“the change made p99 40% faster”) is only as good as the measurement behind it, and most measurement mistakes make systems look better than they are.
This article walks through the measurement chain from the clock up to the load generator, using node:perf_hooks on Node 22.
Pick a clock that only moves forward
Date.now() returns wall-clock time in whole milliseconds since 1970. Wall-clock time can be adjusted while your program runs: NTP corrections slew or step it, an administrator can set it, and VM migrations can make it jump. A duration computed from two Date.now() calls can come out negative or wildly long.
performance.now() is monotonic: it’s milliseconds since the process started, with sub-millisecond precision, and it never goes backwards. process.hrtime.bigint() is the same idea in nanoseconds as a BigInt.
const a = Date.now(), b = performance.now();
console.log(typeof a, a, '|', typeof b, b.toFixed(3));number 1791394934674 | number 448.624Use Date.now() (or new Date()) for timestamps you log or store, and performance.now() or hrtime for anything you subtract.
For timing named phases, performance.mark and performance.measure record entries you can collect with a PerformanceObserver, which keeps instrumentation separate from the code being measured:
import { performance, PerformanceObserver } from 'node:perf_hooks';
new PerformanceObserver((list) => {
for (const e of list.getEntries()) console.log(`measure ${e.name}: ${e.duration.toFixed(1)} ms`);
}).observe({ entryTypes: ['measure'] });
performance.mark('load-start');
await sleep(30); // waiting on I/O
performance.mark('load-end');
performance.measure('load-user', 'load-start', 'load-end');measure load-user: 31.3 msA 30 ms sleep measured as 31.3 ms. That extra millisecond is real (timer scheduling isn’t exact) and is a reminder that very short operations need many repetitions before a single measurement means anything. In a long-running server, clear marks you no longer need with performance.clearMarks(); the timeline keeps them otherwise.
Summarise with percentiles
Latency distributions are skewed. Most requests are fast, a few are very slow, and the slow ones are what users notice and what causes timeouts upstream. A mean blends them into a number that describes no actual request. Record into a histogram and report percentiles: p50 for the typical request, p99 for the bad ones, and max because a single 400 ms stall matters if your client’s timeout is 300 ms.
perf_hooks.createHistogram() gives you an HDR-style histogram that records integers (nanoseconds work well) and answers percentile(n), mean, min and max cheaply, so you can record every request without storing them.
Two more summarising mistakes are worth naming. Averaging percentiles across instances (p99 of server A averaged with p99 of server B) doesn’t give the fleet’s p99; you need to merge the histograms or compute from raw data. And a p99 from a 30-second test with 200 requests is computed from the two slowest samples, which is noise.
The load generator can lie: coordinated omission
This is the measurement error interviewers most like to hear about, because it’s subtle and very common. The demo server below answers in about a millisecond, but every 2 seconds something blocks its event loop for 400 ms, standing in for a garbage collection pause or a big synchronous JSON.stringify:
import http from 'node:http';
setInterval(() => { const end = Date.now() + 400; while (Date.now() < end) {} }, 2000);
http.createServer((req, res) => res.end('ok')).listen(3300);The client measures it two ways for 6 seconds each, both aiming for one request every 10 ms. The closed loop sends a request, waits for the response, records the time, sleeps 10 ms, and repeats. The open loop schedules request number i for start + i × 10 ms regardless of what happened before, and measures latency from that intended start time:
// Closed loop: send the next request only after the previous one returns.
while (performance.now() < stop) {
const t = process.hrtime.bigint();
await get(); h.record(process.hrtime.bigint() - t); n++;
await sleep(INTERVAL);
}
// Open loop: requests are scheduled at fixed times; latency counts from the scheduled time.
for (let i = 0; i * INTERVAL < DURATION; i++) {
const intended = start + i * INTERVAL;
const wait = intended - performance.now();
if (wait > 0) await sleep(wait);
pending.push(get().then(() => { h.record(Math.max(1, Math.round((performance.now() - intended) * 1e6))); n++; }));
}Same server, same 6 seconds:
closed loop (one at a time) n= 403 mean 4.0 p50 0.9 p99 5.8 max 398.2
open loop (fixed schedule) n= 600 mean 41.8 p50 1.6 p99 378.0 max 397.7The closed loop reports a p99 of 5.8 ms. The open loop reports 378 ms. Both are measuring the same server.
The closed loop is wrong. During each 400 ms stall, it had one request outstanding and politely waited, so it recorded one slow sample and then simply didn’t send the 39 other requests it would have sent in that window. Those missing requests are exactly the ones real users would have sent, and every one of them would have waited for the stall. The generator coordinated with the system under test and omitted the bad samples; the term “coordinated omission” comes from Gil Tene, who built HdrHistogram and the wrk2 load tool around fixing it. Note also that the closed loop managed only 403 requests instead of 600: it silently reduced the load.
Real users don’t wait for each other. Production traffic is open-loop, so measure it that way: drive a constant arrival rate and measure from the intended send time. wrk2 and k6’s constant-arrival-rate executors do this. A tool that runs a fixed number of connections each sending back-to-back is a closed loop per connection; it’s fine for finding maximum throughput, but don’t trust its tail latencies.
Timing inside the handler misses the queue
A common in-app metric starts a timer at the top of the request handler and stops it when the response finishes. That measures your handler. It doesn’t measure the time the request sat in the socket buffer waiting for the event loop to get to it. During a 400 ms stall, a request that arrives and then runs in 1 ms is recorded as 1 ms, while the client waited 401.
So pair handler timing with two loop-level signals. monitorEventLoopDelay samples how late a timer fires, which directly measures how long callbacks are being held up; the blocking work note shows it in action. Event loop utilization (ELU) measures the fraction of time the loop was busy running code rather than idle waiting for events:
const idle = performance.eventLoopUtilization();
await sleep(200);
const idleElu = performance.eventLoopUtilization(idle);
const before = performance.eventLoopUtilization();
const until = Date.now() + 200; while (Date.now() < until) {}
await sleep(0);
const busyElu = performance.eventLoopUtilization(before);
console.log(`ELU while waiting: ${idleElu.utilization.toFixed(2)}, while computing: ${busyElu.utilization.toFixed(2)}`);ELU while waiting: 0.01, while computing: 0.99ELU is often a better saturation signal for Node than CPU percentage. Process CPU includes garbage collection and libuv threads, and on a multi-core machine a pegged event loop shows up as maybe 100% of one core out of eight. An ELU near 1 means the single JavaScript thread has no spare time, and queueing (and therefore latency) will climb sharply with any more load.
Client-side measurement, ideally from the load balancer’s logs or a synthetic client, remains the ground truth for what users experience. In-process metrics tell you why.
Interview practice
A teammate benchmarks a change with a tool that opens 10 connections and sends requests back-to-back on each. Before the change: mean 12 ms, p99 30 ms. After: mean 11 ms, p99 28 ms, so they conclude the change is a small improvement. In production, after deploying, p99 rises from 80 ms to 250 ms during peak traffic. Give at least two reasons the benchmark could mislead, and how you’d measure instead.
Answer
First, coordinated omission. Ten back-to-back connections form a closed loop: whenever the server stalls, each connection waits, sends nothing, and records one slow sample instead of all the requests that would have arrived during the stall. If the change introduced periodic stalls (for example more garbage collection from extra allocation, or a synchronous step that occasionally runs long), this setup would barely show it. Second, the load level is wrong. Ten connections self-limit to whatever the server can handle, so the benchmark never reproduces peak traffic arrival rates, where queueing makes small per-request costs grow nonlinearly. Third, the benchmark probably used a warm cache, one payload size and no concurrent background work, while production has a mix.
I’d rerun with a constant-arrival-rate tool (wrk2, or k6 with a constant arrival rate) at production peak rates and above, recording into a histogram and comparing p50, p99 and max before and after across several runs. On the server I’d capture event loop delay and ELU during the run, plus GC activity with --trace-gc or the perf_hooks GC entries. If loop delay spikes line up with the latency tail, I’d take a CPU profile under load to find the new cost the change introduced.