Network Latency Versus Bandwidth

A classic interview prompt goes something like: “Users on a fast fibre connection say the app is slow. Their bandwidth is great. What’s going on?” The answer the interviewer hopes for is that bandwidth and latency are different limits, that most web requests are limited by latency, and that you can tell which one you’re hitting by doing a bit of arithmetic.

Think of a network path as a pipe. Bandwidth is how wide it is: how many bits per second you can push once data is flowing. Latency is how long it is: how long one bit takes to get from one end to the other. A wider pipe doesn’t make it shorter.

Two numbers, two different limits

For one request, a decent first estimate is:

time ≈ (round trips × RTT) + (bytes ÷ bandwidth)

RTT is the round-trip time. The first term is set by distance and by how many back-and-forth exchanges your protocol needs. The second is set by the size of the data and the capacity of the slowest link.

Latency has a floor you can’t buy your way under. Light in optical fibre covers roughly 200 km per millisecond. London to New York is about 5,600 km, so a round trip can never be faster than about 56 ms, and real routes, which don’t follow a straight line, usually come in around 70 ms. No upgrade at either end changes that. The only fix is to make fewer round trips or to move the servers closer, which is what a CDN does.

Bandwidth, by contrast, keeps getting cheaper. That’s why the latency term so often turns out to be the one that matters.

Counting the round trips in one request

Picture a phone fetching a 100 KB JSON response over a fresh HTTPS connection to a server 70 ms away:

TCP doesn’t send at full speed straight away. It starts with a congestion window of about ten segments (roughly 14.6 KB) and roughly doubles it every round trip while things go well. So 100 KB takes about three round trips of data on a fresh connection, regardless of whether the link is 50 Mbit/s or 1 Gbit/s.

The total is around 400 ms, and at 1 Gbit/s the raw transfer time for 100 KB is under a millisecond. Bandwidth barely appears in the sum. Doubling it would save almost nothing; halving the RTT would halve the time.

A simulation: chatty calls versus one batched call

To see the difference without a real long-distance link, this Node server waits 80 ms before every response (standing in for one round trip) and throttles file downloads to 1 MB/s:

const DELAY_MS = 80;              // pretend every request costs one 80 ms round trip
const BYTES_PER_SEC = 1_000_000;  // and the link carries 1 MB/s

const server = http.createServer(async (req, res) => {
  const url = new URL(req.url, 'http://x');
  await new Promise((r) => setTimeout(r, DELAY_MS));
  if (url.pathname === '/item') return res.end(JSON.stringify({ id: url.searchParams.get('id') }));
  if (url.pathname === '/items') return res.end(JSON.stringify(url.searchParams.get('ids').split(',').map((id) => ({ id }))));
  // /file streams `bytes` bytes in 10 KB chunks, pausing between chunks to hold 1 MB/s
});

The client fetched ten items three different ways, then two files:

for (const id of ids) await (await fetch(`${base}/item?id=${id}`)).json();          // one after another
await Promise.all(ids.map(async (id) => (await fetch(`${base}/item?id=${id}`)).json())); // all at once
await (await fetch(`${base}/items?ids=${ids.join(',')}`)).json();                     // one batched call

10 requests, one after another     891 ms
10 requests, all at once           97 ms
1 batched request for 10 items     86 ms
20 KB file                         109 ms
2 MB file                          2315 ms

The sequential loop paid the 80 ms ten times. Running the requests in parallel or batching them paid it once. The data was tiny in every case, so bandwidth didn’t matter at all.

The two downloads show the other regime. The 20 KB file took barely longer than a single round trip: latency-bound. The 2 MB file took about 2.3 seconds (2 seconds of transfer at 1 MB/s, one round trip, and a little timer overhead from the throttle): bandwidth-bound. For that one, cutting round trips would save 80 ms out of 2,300. Compression or a faster link would help a lot.

This is the core of the diagnosis. Work out which term dominates before you pick a fix.

There’s a catch the parallel version hides. It only works when the calls are independent. If call two needs an ID from call one’s response, and call three needs something from call two, you can’t run them concurrently. Each level of dependency adds a full round trip. Mobile apps that load a screen through four layers of dependent calls pay four RTTs before anything renders, and nothing on the client fixes that. The fix is on the server: a batched endpoint, a backend-for-frontend that makes the chatty calls inside the data centre (where an RTT is well under a millisecond), or an API shape like GraphQL that fetches related data in one request.

When bandwidth does matter, and the window that caps it

Large transfers over long distances bring in a third factor. TCP can only have a limited amount of unacknowledged data in flight: the smaller of the receiver’s window and the sender’s congestion window. That caps one connection’s throughput at:

max throughput = window ÷ RTT

Without window scaling, the largest window is 65,535 bytes. Over a 70 ms path that’s 65,535 ÷ 0.07, about 936 KB/s or 7.5 Mbit/s, on any link speed. To keep a 1 Gbit/s link full at 70 ms, you need about 8.75 MB in flight. That figure is the bandwidth-delay product, and it’s why window scaling exists and why transfer tools tune socket buffers or open several parallel streams for big copies across oceans.

A few related misconceptions are worth heading off in an interview:

• Ping time is not page time. A 20 ms ping can still mean a 400 ms page if the page needs twenty round trips.
• Latency isn’t fixed. When a link is saturated, packets queue in router buffers and the RTT climbs, sometimes by hundreds of milliseconds (bufferbloat). A large upload can make every other request on the same connection feel slow.
• More parallelism isn’t free. Browsers cap HTTP/1.1 connections per origin (six is typical), servers have their own limits, and opening many new connections means paying for more handshakes and more slow starts. HTTP/2 multiplexing over one warm connection usually beats it.

Interview practice

A dashboard loads in 600 ms for staff in the office next to the Frankfurt data centre and in about 4 seconds for customers in Sydney, who have fast 5G. The screen makes twelve API calls, several of which depend on earlier responses. What’s happening, and what would you change?

Answer

The numbers point to latency, not bandwidth. Sydney to Frankfurt is around 280 ms per round trip, against a few milliseconds in the office, and fast 5G doesn’t shorten the distance. I’d capture the request waterfall in the browser’s network panel to count how many round trips sit on the critical path: new connections (TCP and TLS each cost one), plus every level of dependent calls. If there are four dependent levels plus connection setup, that alone is roughly six round trips, or 1.7 seconds, before counting server time or slow start on larger responses.

Then I’d fix the critical path, not the bandwidth. Collapse the dependent calls into one backend-for-frontend endpoint so the chatty part runs inside the data centre. Make sure the client keeps connections warm and uses HTTP/2, so independent calls share one connection instead of each paying for a handshake. Cache static and rarely changing data at a CDN edge near Sydney. If a large share of customers are in Australia, serving them from a nearby region removes most of the remaining RTT. I’d measure again after each change, because the goal is fewer round trips times a smaller RTT, and both can be checked.

Next steps

• Networking interview questions
• Networking MCQs
• TCP connections and the handshake
• TCP flow control
• Reference: MDN: HTTP