System Design CDNs and Cacheable Content

A content delivery network keeps copies of your responses in data centres close to users, so a visitor in Sydney gets your JavaScript bundle from Sydney instead of from a server in Virginia. That saves a couple of hundred milliseconds of round trips per request and, more importantly for capacity, takes most of the traffic off your origin.

The design question isn’t “should we use a CDN?” (for public static content, yes). It’s which responses the CDN is allowed to share between users, for how long, and under which key. Get that wrong in one direction and the CDN does nothing. Get it wrong in the other and it serves one user’s account page to another.

What a CDN may share

An edge cache stores a response under a cache key, usually the host and path plus selected query parameters and headers. The next request with the same key gets the stored copy. That rule gives you the test for every endpoint: is the response identical for everyone who sends this key?

Walk through a typical e-commerce site with that question:

A few directives do most of the work. max-age is how long any cache may consider the copy fresh. s-maxage overrides it for shared caches like CDNs, so you can let the edge cache for 60 seconds while browsers revalidate. private means only the user’s own browser may store it. no-store means nobody stores it. stale-while-revalidate=30 lets the edge serve the expired copy for up to 30 more seconds while it fetches a fresh one in the background, which hides origin latency and blunts stampedes.

The dangerous misconception is that a CDN “knows” what’s personal. Many CDNs skip caching responses that carry Set-Cookie or Authorization by default, but defaults vary between providers and are easy to override with a broad “cache everything” rule. If /account/orders is ever cached under its URL alone, the second user to visit sees the first user’s orders. The origin should state its intent explicitly on every response, and personal responses should say private, no-store.

Versioned URLs instead of purges

How do you ship a new JavaScript bundle when browsers and edges hold the old one for a year? You don’t overwrite it. The build puts a content hash in the file name (app.3f9a1c.js becomes app.8b20e4.js), the HTML references the new name, and the old file simply stops being requested. That’s why hashed assets can carry immutable and a year-long max-age: the URL changes whenever the content does.

The alternative, overwriting app.js and purging it from the CDN, works but has rough edges. Purges take time to reach every edge location, some providers rate-limit or charge for them, and browsers that cached the old file aren’t affected by a CDN purge at all. Keep purges for content whose URL can’t change, like the home page HTML after a breaking story, and give that content a short s-maxage so a missed purge costs a minute, not a day.

Hit ratio and the long tail

The value of a CDN shows up as hit ratio: the share of requests answered from the edge. To see what drives it, a Python simulation sends 3 million requests across 50,000 product images, with popularity following a Zipf distribution (a few images are very popular, most are rarely viewed), through an LRU cache of different sizes:

def hit_ratio(keys, capacity):
    cache, hits = OrderedDict(), 0
    for k in keys:
        if k in cache:
            hits += 1; cache.move_to_end(k)
        else:
            cache[k] = True
            if len(cache) > capacity: cache.popitem(last=False)
    return hits / len(keys)

50,000 distinct objects, 3,000,000 requests, Zipf alpha 1.0
  edge cache holds   1% of objects: hit ratio 47.6%, origin gets 52.4% of requests
  edge cache holds   5% of objects: hit ratio 64.5%, origin gets 35.5% of requests
  edge cache holds  20% of objects: hit ratio 80.0%, origin gets 20.0% of requests
  5% cache, half the URLs carry ?utm_source=<one of 30>, key includes it: hit ratio 39.7%
  5% cache, same traffic, parameter stripped from the key:             hit ratio 64.5%

Two lessons come out of this. First, a long tail of rarely requested objects limits how high the hit ratio goes. Holding the top 1% of images already serves nearly half the requests, but going from 5% to 20% of the catalogue only moves the hit ratio from 64.5% to 80%. Static bundles get hit ratios above 99% because there are only a handful of them and everyone requests them; a big product catalogue doesn’t.

Second, the last two lines show the cache key problem. When half the links arrive with a marketing parameter like ?utm_source=newsletter and the CDN includes the full query string in the key, each image gets cached up to 31 times under different keys. The hit ratio fell from 64.5% to 39.7% with identical content. The fix is configuring the CDN to drop parameters that don’t change the response (or allowlist the ones that do, such as ?w=800 for an image width).

Vary causes the same fragmentation. Vary: Accept-Encoding is fine, a handful of variants. Vary: User-Agent creates a variant per browser build, thousands of them, and Vary: Cookie effectively makes every user’s copy separate, which is the same as not caching.

Origin protection and tiered caching

A CDN has hundreds of edge locations, and each one misses independently. If 300 locations each cache /api/products/42 for 30 seconds, the origin can see up to 300 requests per 30 seconds for that one product instead of one. Most CDNs offer a regional or “shield” tier: edges that miss ask a mid-tier cache first, and only the mid-tier goes to the origin. That collapses those 300 misses into one or a few.

Also plan for the day the CDN can’t help. If the edge serves 95% of 100,000 requests per second, the origin is sized for 5,000. A purge of everything, a misconfigured cache key or a cold new region can send it 50,000 in an instant. Keep request collapsing on at the edge, use stale-while-revalidate and stale-if-error so the edge can keep serving old copies when the origin struggles, and don’t purge the whole site at peak.

Interview practice: a news home page with a logged-in header

“A news site wants its home page on a CDN. The page shows the latest headlines, plus ‘Hi, Priya’ and a notification count in the header for logged-in readers. Editors need breaking stories visible within a minute. How would you cache it?”

Answer

The page mixes shared content (headlines, the same for everyone) with personal content (name, notifications). If I cache the whole HTML by URL, either nobody gets personal content or everyone gets Priya’s. So I’d split it. The HTML shell with the headlines is public: Cache-Control: public, s-maxage=30, stale-while-revalidate=30, with no Set-Cookie on that response. The personal header loads separately, from /api/me with private, no-store, filled in by a small script after the page renders (or with an edge-side include if the CDN supports it).

The 30-second s-maxage means a new headline appears within about a minute even if nothing else happens, which meets the editors’ requirement. For a major breaking story, editors can also trigger a purge of /, but correctness doesn’t depend on the purge arriving. I’d strip tracking parameters from the cache key so newsletter links don’t fragment it, enable a shield tier so hundreds of edges don’t each hit the origin every 30 seconds, and add stale-if-error so readers still get the last good page if the origin goes down during a traffic spike.

Further reading

• System design interview questions
• System design MCQs
• CloudFront caching behaviour
• Cache-aside and stale data
• MDN: Cache-Control header reference