CDN & Edge Caching

Start here: one server, users everywhere#

TL;DRthe 30-second version

A CDN is a network of caches spread across hundreds of cities. A user's request is served by the nearest one instead of by your origin server, so it travels a few kilometres instead of thousands.
Each cache location is a point of presence (PoP): a set of edge servers close to users. The first request for a file misses and is fetched from your origin once; every request after that is a hit served locally.
You steer each user to the nearest PoP with anycast (one IP address announced from every location, and the internet routes to the closest) or with DNS-based routing (the CDN's nameserver hands back a nearby address).
The number that decides everything is the cache hit ratio: the fraction of requests answered at the edge. A 96% hit ratio means your origin sees 1 request in 25 — that is the origin offload a CDN buys you.
You keep copies fresh with a time-to-live (TTL) set by the Cache-Control header, and you force an early update by purging the cache or by changing the URL when the content changes.

Say your origin server — the one machine that holds the real, canonical copy of your site — sits in a data center in Virginia. A reader opens your homepage from Sydney. Their request for `/assets/hero.jpg`, a 2 MB image, has to travel from Sydney to Virginia and the response has to travel back. That round trip is about 16,000 kilometres each way, and even at the speed of light through fibre that's roughly 160 milliseconds of pure propagation — closer to 200 ms in practice once handshakes and queuing are added — before the first byte lands. The image itself, at 2 MB, then takes several more round trips to transfer. The reader waits, and the page feels slow.

Now multiply that by your audience. If a million readers in Australia load that homepage, all million requests cross the Pacific to the same server in Virginia. The distance is paid a million times over on latency, and your origin has to push the same 2 MB image out a million times, which is two terabytes of bandwidth for one popular image on one day. The obvious fix is to make the origin bigger — more CPU, more machines behind it. But a bigger server in Virginia is still in Virginia. It does nothing about the 200 milliseconds of distance, and it still pushes every byte across the ocean. Scaling the origin is answering the wrong question.

Distance is the cost you can't scale awayTwo costs are hiding here, and only one of them responds to a bigger origin. The first is throughput: how many requests and how many bytes the origin can serve. You can scale that with more machines. The second is latency: the time the request spends in flight, set by the physical distance to the server and the speed of light. No amount of origin capacity shortens that trip. The only way to cut the distance is to move the answer closer to the person asking. That single move is the whole idea behind a CDN.

Put a copy near the user#

So put a copy of the image close to the reader. Rent space in a data center in Sydney, run a cache server there, and serve Australian readers from it. The first Sydney reader to ask for `/assets/hero.jpg` finds nothing in the local cache — a cache miss — so the Sydney server fetches the image from your origin in Virginia one time, stores it, and serves it. The next Sydney reader is a cache hit: the image is already there, a few kilometres away, and comes back in about 5 milliseconds instead of 200. The million-reader crowd now pays the long trip to Virginia exactly once, not a million times.

One cache in Sydney helps Australia. To help everyone, you do the same thing in hundreds of cities. That fleet of cache locations is the content delivery network. Each location is called a point of presence, or PoP: a cluster of edge servers — the caches that sit at the edge of the network, close to users — in one city. Your origin stays where it is and stays the single source of truth; the CDN wraps a layer of caches around it, one near every population center. A request now hits the closest PoP, and only a miss travels the rest of the way to the origin.

Reader in SydneyGET /assets/hero.jpg

request

Sydney PoP (edge cache)hit: serve in ~5 ms

miss → fetch once, then store

Origin in Virginiaonly reached on a miss

The request path (a hit stops early)

That leaves one question the picture skips: how does the Sydney reader's request reach the Sydney PoP and not the one in Frankfurt? There are two common answers. The first is anycast. The CDN announces the same IP address — say `104.16.0.1` — from every PoP at once, using BGP, the protocol routers use to advertise which networks they can reach. Because hundreds of locations all claim that one address, the internet's own routing naturally delivers each request to the nearest location that answers for it. The reader does nothing special; they connect to one IP, and the network hands them the closest copy. Cloudflare and Fastly route this way.

The second answer is DNS-based routing. Here the CDN's nameserver does the steering: when the reader's browser resolves your hostname, the CDN looks at where the request came from and returns the IP address of a nearby PoP. This is the DNS resolution step you already know (see the DNS simulator at /dns/sim), with one twist — the answer depends on the asker's location. Amazon CloudFront routes this way. Either method lands the reader on a close-by edge; anycast decides at the network layer, DNS decides at name-resolution time.

What counts as "the same file": the cache keyAn edge cache has to decide whether two requests are asking for the same thing. It builds a cache key from the request, and by default the key is the hostname plus the path plus the query string. So `example.com/assets/hero.jpg?v=7` and `example.com/assets/hero.jpg?v=8` are two different keys and two separate cached copies, while two readers both asking for `?v=7` share one. The key is why a versioned URL forces a fresh fetch — a new query string is, to the cache, a brand-new file it has never seen. Hold on to that; it's the trick behind cache busting later.

Keeping copies fresh: TTL, purge, and versioned URLs#

A cached copy can go stale. Your origin changes the image, but the Sydney PoP is still holding the old one. So each copy carries a time-to-live, or TTL: how many seconds the edge is allowed to treat it as fresh before checking back. The origin sets the TTL with an HTTP response header called Cache-Control. This is the same freshness machinery from the caching-strategies topic (the TTL and stampede behaviour is runnable at /cachestrategy/sim); a CDN just applies it across the whole edge fleet at once.

The Cache-Control header carries a few directives that matter at the edge. Here are the ones worth knowing by name:

Directive	What it tells the caches
max-age=600	Any cache may treat this as fresh for 600 seconds (this applies to the browser's own cache too).
s-maxage=600	The shared cache (the CDN) may treat it as fresh for 600 seconds; overrides max-age for the CDN only, so you can give the edge and the browser different TTLs.
public	Any shared cache is allowed to store this response.
private	Only the user's own browser may cache it; the CDN must not store it (used for per-user content).
no-store	Never cache this anywhere — every request goes to the origin.
stale-while-revalidate=30	After freshness expires, the edge may serve the stale copy for up to 30 more seconds while it fetches a fresh one in the background.

A TTL handles content that changes on a schedule. But sometimes you change something now and cannot wait for the TTL to expire — you shipped a broken stylesheet and need the fix live immediately. There are two ways to force an early update, and they pull in opposite directions.

Purge (also called invalidation): you tell the CDN to drop a cached copy right now, by URL or by a tag you attached to a group of files. The next request for it misses and re-fetches from the origin. Purges are direct but not instant — the command has to reach every PoP, which takes a moment, and a purge of many files at once briefly raises origin load as edges refetch.
Versioned URLs (cache busting): instead of deleting the old copy, you stop asking for it. Rename the file `hero.7.jpg` → `hero.8.jpg`, or change the query string `?v=7` → `?v=8`. Because the cache key changes, the CDN has never seen the new URL, so it's a guaranteed miss and a fresh fetch — no purge command, no waiting for it to propagate. The old copy simply ages out on its own TTL, ignored.

The pattern almost everyone lands onStatic assets — images, JavaScript bundles, CSS — get a very long TTL (a year is common) plus a versioned URL. Because the filename changes whenever the content changes, you can cache it forever and never purge: a new version is a new URL, an old version is harmless. Content that changes unpredictably (an HTML page, an API response) gets a short TTL, or a short TTL plus stale-while-revalidate so readers never wait on a refresh. Purge is kept for the emergency you didn't plan for.

The number that matters: cache hit ratio#

A CDN's whole value is measured by one ratio: the cache hit ratio, the fraction of requests answered at the edge without touching the origin. Everything you tune — TTLs, cache keys, shielding — is really tuning this number up. The reason it dominates is that the requests it doesn't catch, the misses, are the expensive ones: they cross the network to your origin, burn origin CPU, and pull bytes out of your origin's bandwidth, which is the part of the bill that grows with traffic.

The flip side of the hit ratio is origin offload: the share of load the CDN absorbs so your origin never sees it. A 96% hit ratio means the origin handles 4% of requests, so it's offloading 24 requests for every 1 it lets through. Push the ratio higher and the origin's share drops fast — the last few percent are worth chasing because they cut origin load by large multiples, not small ones.

PredictYour news site serves 100 million requests a day. The CDN reports a 96% cache hit ratio. How many requests actually reach your origin — and what happens to that number if you tune the hit ratio up to 98%?

Hint: Miss fraction × total requests = origin load. Compute it at 96%, then at 98%, and compare — the interesting part is the ratio between them.

At 96%, the origin sees the 4% that miss: 4% of 100 million is 4 million requests a day, roughly 46 per second on average. Without the CDN it would be all 100 million (about 1,160 per second) — so the CDN is offloading 96 million requests, and your origin can be a fraction of the size it would otherwise need. Now tune the ratio to 98%: misses drop to 2% of 100 million, or 2 million a day. Going from 96% to 98% doesn't feel like much, but it halves origin load (4 million → 2 million). That's why hit ratio is the metric CDN tuning obsesses over: near the top of the range, a two-point gain is a 2x reduction in everything your origin has to do. The one-line answer: 4 million reach the origin at 96%, and 98% cuts that in half.

What drags a hit ratio down? Mostly cache keys that split one logical file into many cached copies — a tracking query string that varies per user, or per-user cookies folded into the key, so no two readers ever share a hit. And short TTLs: a copy that expires every few seconds spends most of its life being re-fetched. Raising the ratio is mostly the discipline of not fragmenting the key and not expiring copies sooner than the content actually changes.

When the content can't be shared: static vs dynamic at the edge

A CDN caches a copy and serves it to everyone. That works perfectly when the response is the same for every reader — a logo, a video segment, a JavaScript bundle. This is static content, and it's the easy, high-hit-ratio case. The hard case is dynamic content: a response that differs per reader, like a logged-in dashboard showing your name and your orders. You can't hand one reader's dashboard to another, so a naive edge cache can't share it at all.

The trap is concluding the CDN is useless for dynamic content. It isn't, for two reasons. First, even an uncacheable response still benefits from the edge: the reader's connection (the TCP and TLS handshakes) terminates at the nearby PoP instead of at the far origin, and the CDN carries the request to the origin over its own tuned backbone, which is often faster than the public internet. Second, a lot of "dynamic" content is really slow-changing and shared. A news headline block that updates every few seconds can be cached with a one-to-two-second TTL — micro-caching — so a spike of readers in that window is one origin fetch, not thousands. The honest framing: cache what's shared, set the TTL to how stale you can tolerate, and let the edge accelerate the connection even when the body can't be cached.

Content	Edge treatment	Why
Images, video, JS/CSS bundles	Long TTL + versioned URL	Identical for everyone; changes only on redeploy
Public HTML pages, API reads	Short TTL, often + stale-while-revalidate	Shared but changes on a schedule; tolerate seconds of staleness
Hot, fast-changing shared data	Micro-cache (1–2 s TTL)	One fetch absorbs a burst; brief staleness is acceptable
Per-user / logged-in responses	private / no-store; edge still terminates the connection	Not shareable, but the nearby handshake and CDN backbone still help

Protecting the origin: shielding and tiered caches

There's a failure mode hiding in a large edge fleet. Suppose a popular file's TTL expires, or you purge it, at the moment a big audience wants it. Now every PoP holding readers — hundreds of them — misses at once, and all of them fetch the same file from your origin in the same instant. Your origin, which the CDN was supposed to protect, gets hit by hundreds of simultaneous requests for one file. This is a cache stampede (the same thundering-herd problem the caching-strategies topic simulates at /cachestrategy/sim), amplified by the number of PoPs.

The fix is to add a middle tier. Instead of every edge going straight to the origin on a miss, the edges miss to a smaller set of regional caches — the shield tier — and only the shield goes to the origin. When a hundred edges miss for the same file, they converge on one regional shield, which fetches from the origin once and fans the copy back out. The origin sees one request where it would have seen a hundred. This is origin shielding (CloudFront's Origin Shield, Cloudflare's Tiered Cache), and it typically cuts origin load for popular content by ten to a hundred times.

Many edge PoPs misshundreds, same file, same instant

miss → ask the shield

Regional shield cachededuplicates the misses

shield misses once

Originsees one fetch, not hundreds

A shield tier collapses many misses into one

The two routing choices, side by side

The one design fork worth being able to argue is how a user is steered to a PoP. Anycast and DNS-based routing both get the reader to a nearby edge, but they decide it at different layers and fail differently.

	Anycast	DNS-based routing
How it steers	One IP announced from every PoP; BGP routes to the nearest	The CDN's nameserver returns a nearby PoP's IP per query
Where the decision happens	The network layer, per packet	At name resolution, then cached for the DNS TTL
Reacts to a PoP failing	Fast — BGP withdraws the route, traffic reroutes automatically	Slower — clients hold the old IP until the DNS record's TTL expires
Granularity of "nearest"	Network topology (shortest BGP path), not always geographically closest	Can use geo/latency data to pick a good PoP deliberately
Used by	Cloudflare, Fastly	Amazon CloudFront

Neither is strictly better. Anycast reroutes around a dead PoP in seconds because it rides the routing table itself, but "nearest by network path" isn't always the closest city. DNS routing can steer with richer latency data and deliberate policy, but a client that already resolved the name keeps using the old address until its DNS TTL runs out, so failover is as slow as that TTL. Most large CDNs lean on anycast for its fast, automatic failover.

In the wild

Cloudflare runs edge servers in more than 330 cities and routes with anycast, so one IP is served from everywhere and BGP handles the steering. Its Tiered Cache feature is the shield tier that keeps origin load low for popular content.
Akamai, the oldest CDN, takes the opposite footprint bet: over 4,100 points of presence across 130-plus countries, pushed deep inside ISP networks to sit as physically close to users as possible.
Fastly runs far fewer, more powerful PoPs (on the order of a hundred), also anycast-routed, and is known for near-instant purges and programmable edge logic, which suit use cases that need to invalidate content in a hurry.
Amazon CloudFront uses DNS-based routing tied into Route 53, has crossed 600 points of presence (including cache nodes embedded inside ISP networks), and offers Origin Shield as an explicit regional tier in front of your origin.
The footprint numbers make a tempting scoreboard, but they mostly measure coverage breadth. A network with 330 PoPs can match one with several thousand on the latency a reader actually feels on a cache hit; what a big PoP count buys is reaching users in more remote networks, not a faster hit.

Pitfalls & gotchas

My cache hit ratio is terrible even though the content is public. Why?

Almost always the cache key is fragmented. A tracking query string (`?utm_source=…`) or a per-user cookie folded into the key makes every request look like a unique URL, so no two readers ever share a cached copy and everything misses to the origin. The fix is to normalize the key: strip the query parameters that don't change the response, and don't vary the cache on cookies for content that's the same for everyone.

Is a purge instant?

No. A purge is a command that has to reach every PoP, so there's a short propagation delay before the last edge drops the file — usually seconds, sometimes longer. And a large purge briefly spikes origin load, because a wave of edges all miss and re-fetch at once. When you can, prefer versioned URLs: a new URL is a guaranteed miss with no command to propagate and no purge storm.

I accidentally cached a logged-in user's page and served it to someone else. How?

The response was cacheable when it should have been marked private or no-store. If per-user content comes back with a public, cacheable Cache-Control header, a shared cache will store one user's response under a shared key and hand it to the next reader. Per-user responses must set `private` (browser-only) or `no-store`, and must never be keyed in a way that lets two users collide. This is the most dangerous CDN misconfiguration.

Why did my page keep showing the old version long after I updated it?

A copy is fresh for its full TTL, and nothing you do at the origin reaches an edge that isn't checking back yet. If you set `max-age=86400` and then change the content, edges keep serving the old copy for up to a day. Either set a TTL that matches how often the content really changes, or purge, or (best for assets) use a versioned URL so the new content is a new key the edge is forced to fetch.

QuizYou deploy a new version of your site's main JavaScript bundle at the same URL, `/app.js`, which is served with `Cache-Control: max-age=31536000` (one year). Some users load the new HTML but the old `/app.js`, and the site breaks for them. What's the clean fix?

Lower the TTL on /app.js to a few seconds so it refreshes constantly.
Purge /app.js on every deploy and hope the purge reaches all PoPs before users load the page.
Give the bundle a versioned URL (e.g. /app.8f3a.js) that changes when the content changes, so the new HTML references a new key the edge must fetch.
Set the bundle to no-store so it's never cached.

Show answer

Give the bundle a versioned URL (e.g. /app.8f3a.js) that changes when the content changes, so the new HTML references a new key the edge must fetch. — The problem is a mismatch: the HTML now points at a bundle whose old copy is still cached under the same key. Versioning the URL solves it cleanly — when the content changes, the filename changes (`/app.8f3a.js`), so the new HTML references a URL the edge has never seen and is forced to fetch, while the old bundle harmlessly ages out. That's why you can keep the year-long TTL and never purge. Lowering the TTL (choice 1) throws away the whole benefit of caching a static asset and still races the deploy. Purging (choice 2) works but isn't instant and causes a refetch storm on every deploy. no-store (choice 4) means the bundle is fetched from the origin on every single page load — the worst outcome for a file that rarely changes.

Under the hood: anycast and cache-key normalization

Go deeperHow anycast actually picks a PoP (and where it's fuzzy)

Anycast leans entirely on BGP, the protocol by which networks advertise the address ranges they can deliver to. Every PoP announces the same range (say `104.16.0.0/13`) to its upstream networks. Each router on the internet then has many paths to that range and picks one using BGP's own preference rules — shortest AS-path, local policy, and so on. The result is that a packet flows to whichever announcing PoP is closest by the routing table's notion of distance.

The subtlety is that BGP's "closest" is topological, not geographic. A reader might be routed to a PoP a few network hops away that is, in map terms, farther than another PoP their network happens to have a worse path to. It's usually close enough, and it's why anycast failover is automatic: when a PoP stops announcing (because it died or was drained), routers simply pick the next-best path with no DNS change and no client action. The cost is that you don't get fine per-user control over which PoP serves whom — the routing table decides.

Go deeperCache-key normalization: the lever on hit ratio

The default cache key is hostname + path + query string, but you rarely want it raw. Two requests that differ only by a marketing parameter — `?utm_campaign=spring` versus no parameter — are the same file, yet a raw key treats them as two, halving reuse. So CDNs let you normalize the key: drop query parameters that don't affect the response, sort the ones that do, and decide explicitly which headers or cookies are allowed to vary a cached copy.

The trade-off runs in both directions. Normalize too aggressively and you collapse responses that really are different (serving a mobile layout to a desktop reader because you dropped the header that distinguishes them). Normalize too little and you shatter the hit ratio into per-user fragments. The `Vary` response header is the honest tool here: it names exactly which request headers legitimately produce a different response (for example `Vary: Accept-Encoding`, so a gzip copy and a Brotli copy are cached separately and correctly), instead of caching one and serving it to the wrong client.

In an interview

A CDN shows up in almost every design that serves content to a wide audience (it's the front tier of the video-streaming and news-feed designs on this site). The interviewer isn't testing whether you know the acronym — they're testing whether you can reason about what caches, what doesn't, and how the origin is protected. Anchor your answer on the hit ratio and the static-versus-dynamic split.

Lead with the problem, not the product: one origin can't be close to everyone, and distance is a latency cost you can't scale away. A CDN moves copies to the edge so most requests are local hits.
Name the metric: cache hit ratio drives everything, and origin offload is its consequence. Be ready to compute the origin load from a hit ratio and a request rate, and to note that the last few points of hit ratio are worth the most.
Split static from dynamic: static assets get a long TTL plus versioned URLs (cache forever, never purge); dynamic and per-user content gets short TTLs, micro-caching where it's shared, or private/no-store where it isn't — and the edge still accelerates the connection either way.
Know the two freshness levers and their trade: TTL for scheduled change, and for an early update either a purge (direct, but propagates and can spike the origin) or a versioned URL (a guaranteed miss with no purge storm).
Name the origin-protection story: a shield or tiered cache collapses a stampede of edge misses into a single origin fetch. That's the answer to "what happens when a popular file expires everywhere at once?"
If pushed on routing, contrast anycast (fast automatic failover via BGP, less per-user control) with DNS-based routing (deliberate steering, but failover is bounded by the DNS TTL).

PredictThe interviewer says: "Your homepage HTML is the same for all logged-out users but changes a few times an hour, and it's getting hammered during a traffic spike. How do you use the CDN?"

Hint: Same for everyone + tolerates minutes of staleness = cacheable. Which directives give you a short freshness window without making readers wait on a refresh during the spike?

Recognize that the page is shared (same for every logged-out user), so it is cacheable — the only question is staleness. Give it a short TTL that matches how fresh it needs to be: if a minute of staleness is fine, `s-maxage=60`. Add stale-while-revalidate so that when the copy expires, readers are served the slightly-old copy instantly while the edge refreshes in the background, instead of waiting on the origin. During a spike, that combination means the flood of readers is absorbed by the edge and the origin sees roughly one fetch per PoP per TTL window, not one per reader. If the spike is huge, add a shield tier so even those per-PoP misses collapse into a single origin fetch. The key insight to state out loud: "changes a few times an hour" plus "same for everyone" means cacheable with a short TTL — the fix is a freshness policy, not turning caching off. If any part of the page is per-user, split that fragment out and mark it private, so the shared shell still caches.

References

MDN — Cache-Control — The authoritative directive reference: max-age, s-maxage, public/private, no-store, stale-while-revalidate.
MDN — HTTP caching — Freshness, revalidation, and the Vary header, from the source.
Cloudflare — What is a CDN? — PoPs, edge caching, and anycast routing explained by a CDN operator.
Cloudflare — Tiered Cache — The shield-tier design that deduplicates edge misses to protect the origin.
AWS — CloudFront Origin Shield — A named regional shield tier and how it lowers origin load.
Fastly — How caching and CDNs work — Cache keys, TTLs, purging, and edge behaviour from a second operator.

Up nextThe Page Cache

← Back to the learning path

Feedback on this topic →