Networking: L4 vs L7 Load Balancing

“Layer 4 or layer 7?” sounds like a question about OSI diagrams, but the practical difference is simple: does the load balancer read your HTTP requests or not? Everything else, including what it can route on, how evenly it spreads load, what it costs and who terminates TLS, follows from that.

The clearest way to see it is to build both in a few lines of Node and point curl at them.

Two balancers in sixty lines

The setup has two backends on ports 9001 and 9002 that report which backend answered, the path, the TCP port of whoever connected to them, and any X-Forwarded-For header. In front of them sit two balancers.

import http from 'node:http';
import net from 'node:net';

for (const port of [9001, 9002]) {
  http.createServer((req, res) => {
    res.end(`backend=${port} path=${req.url} peer=${req.socket.remotePort} xff=${req.headers['x-forwarded-for'] ?? '-'}\n`);
  }).listen(port, '127.0.0.1');
}

// Layer 4: pick a backend per TCP connection, then copy bytes both ways.
let next4 = 0;
net.createServer((client) => {
  const port = [9001, 9002][next4++ % 2];
  const upstream = net.connect(port, '127.0.0.1');
  client.pipe(upstream).pipe(client);
  client.on('error', () => upstream.destroy());
  upstream.on('error', () => client.destroy());
}).listen(8001, '127.0.0.1');

// Layer 7: parse each request, route /api/ to 9002, round-robin the rest.
let next7 = 0;
const agent = new http.Agent({ keepAlive: true });
http.createServer((req, res) => {
  const port = req.url.startsWith('/api/') ? 9002 : [9001, 9002][next7++ % 2];
  const headers = { ...req.headers, 'x-forwarded-for': req.socket.remoteAddress };
  const up = http.request({ host: '127.0.0.1', port, path: req.url, method: req.method, headers, agent }, (r) => {
    res.writeHead(r.statusCode, r.headers);
    r.pipe(res);
  });
  up.on('error', () => { res.writeHead(502); res.end('bad gateway\n'); });
  req.pipe(up);
}).listen(8002, '127.0.0.1');

Look at what the layer-4 code knows. It gets a socket and makes a decision before a single byte of HTTP arrives. After that it’s a pipe. It couldn’t route on the path even if you wanted it to, because it never parses one. The layer-7 code receives a parsed request with a URL and headers, so routing on /api/ is one startsWith call.

Keep-alive pins a client to one backend at layer 4

curl reuses a single connection when you give it several URLs on the same host. Here’s what each balancer does with three requests on one connection:

$ curl -s http://127.0.0.1:8001/a http://127.0.0.1:8001/b http://127.0.0.1:8001/c
backend=9001 path=/a peer=54389 xff=-
backend=9001 path=/b peer=54389 xff=-
backend=9001 path=/c peer=54389 xff=-

$ curl -s http://127.0.0.1:8002/a http://127.0.0.1:8002/b http://127.0.0.1:8002/c
backend=9001 path=/a peer=54397 xff=127.0.0.1
backend=9002 path=/b peer=54398 xff=127.0.0.1
backend=9001 path=/c peer=54397 xff=127.0.0.1

The layer-4 balancer sent all three requests to 9001, because it balances connections and there was only one. Run three separate curls, each with its own connection, and it alternates as you’d expect (9002, 9001, 9002). The layer-7 balancer spread the three requests across both backends on a single client connection, because it makes a decision for every request.

This is the most common real-world surprise with layer-4 balancing. A service with a few clients that hold long-lived connections, such as other microservices using HTTP keep-alive or gRPC over HTTP/2, ends up badly unbalanced: one backend handles a busy client’s entire traffic while another sits idle. It also explains why adding a backend doesn’t help right away. Existing connections stay where they are, and only new ones reach the new instance. Teams running gRPC on Kubernetes hit this with plain Service objects (which balance connections) and fix it with an L7 proxy, a service mesh, or client-side balancing.

The backend sees the balancer, not the client

Notice the peer values. In both cases the backend saw a connection from the balancer’s own ephemeral port on 127.0.0.1, not from curl. A proxy that opens its own connection to the backend replaces the client’s address. On a real network the backend’s logs and rate limiter would see the load balancer’s IP for every request.

The two layers solve this differently:

• Layer 7 adds a header. Our proxy set X-Forwarded-For: 127.0.0.1, which reached the backend as xff=127.0.0.1. The standardised alternative is Forwarded: for=.... The backend must only trust these headers when the request came from its own balancer, since any client can send a fake X-Forwarded-For.
• Layer 4 can’t add headers to a byte stream it doesn’t understand. It either preserves the client’s IP at the network level (some cloud network load balancers do this, which also means backend security groups must allow client addresses), or it prepends a small PROXY protocol header to the connection that the backend reads before the HTTP request. The backend has to be configured for it: a server that doesn’t expect PROXY protocol sees PROXY TCP4 ... where it wanted GET / and returns a 400.

What each layer can do, and what it costs

Because a layer-7 balancer terminates the client’s connection and parses HTTP, it can:

• route by host name, path, header or cookie (/api/* to one service, /static/* to another)
• terminate TLS, inspect the request, and re-encrypt to the backend if needed
• retry an idempotent request on another backend after a failure, return a clean 502 or 503, and apply rate limits
• set sticky sessions with a cookie, and make health checks that look at real responses (GET /healthz must return 200)

Layer 4 works with any TCP or UDP protocol: databases, MQTT, SMTP, game traffic, raw TLS. It doesn’t buffer requests, holds very little per-connection state, and can pass TLS through untouched so the backend owns the certificate. It’s typically cheaper and handles far more connections per machine. AWS draws the line as Network Load Balancer (layer 4) versus Application Load Balancer (layer 7); HAProxy offers both as mode tcp and mode http; NGINX has its stream module for layer 4 alongside its usual HTTP proxying.

A common production shape uses both. A layer-4 balancer at the edge spreads connections across a fleet of layer-7 proxies (Envoy, NGINX, HAProxy), which terminate TLS and do the request-level routing. That gets you the scale of layer 4 and the intelligence of layer 7.

A misconception worth correcting in an interview: “layer 4 is faster” is too broad. It adds less latency per request, but a layer-7 balancer that reuses pooled connections to backends can make the backend’s life easier, since it absorbs slow clients and connection churn. Pick on what you need to route on, not on assumed speed.

Practice question: the unbalanced gRPC service

An orders service runs six pods behind a layer-4 load balancer. Its two callers are other services that each open one gRPC connection at startup. Dashboards show two pods at 90% CPU and four nearly idle. Scaling to ten pods changed nothing. Explain and fix it.

Answer and reasoning

gRPC runs over HTTP/2, which multiplexes every call over one long-lived TCP connection. A layer-4 balancer picks a backend when a connection opens, so each caller’s single connection landed on one pod and stays there. Two callers, two hot pods. New pods only get new connections, and the callers don’t open any, so scaling had no effect. The same thing we saw with curl’s three requests on one connection, at larger scale.

Fixes, roughly in order of effort:

1. Put a layer-7 proxy that understands HTTP/2 (Envoy, or a gRPC-aware ingress) between callers and pods, so each call is balanced separately.
2. Use client-side balancing: the caller resolves all pod addresses (for example through a headless Kubernetes Service) and opens a connection to each, spreading calls with gRPC’s round-robin policy.
3. As a partial fix, cap connection age on the server (gRPC’s MaxConnectionAge), so clients periodically reconnect and get rebalanced. It smooths things out but doesn’t balance individual calls.

I’d also note what not to do: increasing pod size or count treats the symptom, and it’s why scaling didn’t help.

Where to go next

• Networking interview questions
• Networking quiz questions
• Proxies and forwarded headers
• Load balancing in system design
• X-Forwarded-For on MDN