“I changed the DNS record an hour ago and half the office still hits the old server.” Nearly every engineer says some version of this once. Nothing is broken. DNS is a cached system by design, and the number that decides how long the old answer survives was chosen by whoever set the record’s TTL, often years before the change you’re making now.
This note works through what the TTL actually controls, using real dig output against public resolvers, and then turns that into a cutover plan you can defend in an interview.
What the TTL number means to each cache
Every DNS record carries a time to live in seconds. The authoritative server for the zone hands out the configured value. A recursive resolver that fetches the record stores it and starts counting down. When a client asks the resolver again, it gets the remaining time, not the original.
You can watch this. Asking one of Cloudflare’s authoritative servers for example.com directly (with +norecurse, so it answers only from its own zone data) shows the configured TTL:
$ dig @hera.ns.cloudflare.com example.com A +norecurse +noall +answer
example.com. 300 IN A 104.20.23.154
example.com. 300 IN A 172.66.147.243Now ask public resolvers, a few seconds apart:
$ dig @1.1.1.1 example.com A +noall +answer
example.com. 95 IN A 104.20.23.154
example.com. 95 IN A 172.66.147.243
$ dig @8.8.8.8 example.com A +noall +answer
example.com. 197 IN A 172.66.147.243
example.com. 197 IN A 104.20.23.154
$ dig @1.1.1.1 example.com A +noall +answer
example.com. 105 IN A 104.20.23.154Three things are visible here. The resolvers report less than 300 because they cached the answer earlier and are counting down. Google and Cloudflare have different remaining times because they fetched at different moments. And the second query to 1.1.1.1 came back with more time left than the first. That isn’t a bug: 1.1.1.1 is anycast, so consecutive queries can land on different cache instances that each fetched the record independently. “The resolver’s cache” is really many caches.
The practical consequence: after you change a record, the worst-case time before every resolver sees it is roughly the old TTL, measured from the moment of the change. A resolver that cached the old value one second before your edit will keep serving it for almost the full 300 seconds.
Negative answers are cached as well
People often forget this one, and interviewers like to ask about it. If a name doesn’t exist, the resolver remembers that too. RFC 2308 defines how long: the response carries the zone’s SOA record in the authority section, and the negative TTL is the smaller of that SOA record’s own TTL and its last field, MINIMUM.
$ dig @a.iana-servers.net stacktrick-missing-5.iana.org A +noall +comments +authority
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 51150
iana.org. 3600 IN SOA sns.dns.icann.org. noc.dns.icann.org. 2026101901 7200 3600 1209600 3600The last field is 3600, and so is the SOA’s TTL, so a resolver may remember “this name does not exist” for an hour. Here’s the classic failure that follows. Someone deploys a service, a health check or a curious engineer looks up api-v2.example.com before the record is created, the resolver caches NXDOMAIN, and then the record is added. For the next hour, clients using that resolver are told the name doesn’t exist, even though the authoritative server would happily answer.
Resolvers don’t all treat that number the same way. Asking the same question of four servers moments apart:
@a.iana-servers.net NXDOMAIN SOA TTL 3600
@1.1.1.1 NXDOMAIN SOA TTL 3600
@8.8.8.8 NXDOMAIN SOA TTL 1800
@9.9.9.9 NXDOMAIN SOA TTL 3600Google Public DNS returned 1800 on a fresh name, which suggests it caps negative caching below what the zone asks for. Repeating the query three seconds later showed 1797, so it was counting down from its own cap. Resolvers are allowed to shorten TTLs, and many also apply a ceiling to very long positive TTLs. You can’t assume the zone’s number is what every user experiences.
One more wrinkle: not every “missing” answer is NXDOMAIN. Querying a nonexistent name under example.com through 1.1.1.1 returned status: NOERROR with zero answers and an SOA in the authority section, because some DNSSEC-signing providers answer missing names as “exists, but has no records of this type”. It’s cached the same way, using the same SOA rule, but if your monitoring only alerts on NXDOMAIN it will miss it.
Caches outside the resolver
The recursive resolver is the cache everyone talks about, but a lookup from an application can pass through several others:
- The browser. Chrome keeps its own host cache (visible at
chrome://net-internals/#dns, which also has a button to clear it). How long it keeps an entry depends on whether it used its built-in resolver or the OS one, so it won’t always match the record’s TTL. - The operating system. macOS caches in
mDNSResponder; on Linux,systemd-resolvedornscdmay cache, or nothing does. On macOS,sudo dscacheutil -flushcache; sudo killall -HUP mDNSResponderclears it. - The language runtime. The JVM caches successful lookups according to
networkaddress.cache.ttl; older JDKs with a security manager cached forever. Node’sdns.lookup()does no caching of its own and goes through the OS resolver on each call, which surprises people in the other direction when they see high lookup volumes. - Connection pools. This is the one that bites during migrations. A pool that opened connections to the old IP keeps using them until they close. DNS might have flipped minutes ago, but a service with long-lived keep-alive connections never looks up the name again.
When someone says “DNS hasn’t propagated”, it’s worth asking which of these layers is actually holding the old address. Often it’s the last one.
Planning a cutover with TTLs
Say app.example.com has an A record with a TTL of 86400 (one day) and you’re moving it to a new load balancer on Saturday at 10:00. Here’s the order of operations and why each step sits where it does.
- Thursday 09:00 or earlier: lower the TTL to 60. This change is itself subject to the old TTL. Resolvers that cached the record with 86400 seconds left won’t ask again, and won’t learn about the new short TTL, for up to a day. Lowering it on Saturday at 09:55 does nothing useful. The rule is to lower it at least one full old-TTL period before the switch, plus a margin.
- Saturday 10:00: change the address. By now every compliant cache holds the record with at most 60 seconds left, so within about a minute most traffic moves.
- Keep the old target alive. Resolvers that extend TTLs, clients with pinned connections, and devices with odd stub resolvers will trickle in for hours. Leave the old server serving (or proxying to the new one) until its access logs go quiet.
- Once stable, raise the TTL again, to something like 3600. A permanent 60-second TTL multiplies query load on your authoritative provider and means every cold cache adds a lookup to the user’s first request.
The trade-off is the whole point of the TTL: long values reduce lookups and protect you from short authoritative outages, because resolvers keep serving cached answers; short values let you change things quickly. Many teams settle on minutes to an hour for records they might fail over, and longer for things like MX and NS that rarely move.
Interview tip
To confirm what a specific resolver is serving, query it by address with
dig @<resolver> name. Querying through your laptop’s default resolver mixes in your OS cache and whatever your network hands you.
Interview exercise: the record that won’t appear
You create a new record, reports.example.com, at 14:00. A colleague says it works for them, but your CI runners get NXDOMAIN until about 15:00. The zone’s SOA is ... 7200 3600 1209600 3600 with a TTL of 3600. What happened, and how would you avoid it next time?
Answer and reasoning
The CI runners’ resolver almost certainly cached a negative answer before 14:00. Perhaps a pipeline step or a health check queried the name before the record existed. Under RFC 2308 that NXDOMAIN is cached for the smaller of the SOA’s TTL (3600) and its MINIMUM field (3600), so up to one hour, which matches the 15:00 recovery. Your colleague uses a different resolver that had never asked, so it fetched the fresh record.
I’d confirm it by querying the authoritative server directly with +norecurse (it answers correctly) and then the CI resolver (NXDOMAIN, with the SOA TTL counting down in the authority section). To avoid it: create records before anything references them, and if the zone’s negative TTL is long, consider lowering the SOA MINIMUM to a few minutes. That only helps future lookups, because existing negative entries keep their old remaining time. If the CI resolver is one you run, you can also flush that single name from its cache.
A weaker answer says “DNS propagation takes time” and stops. The interviewer is listening for negative caching, the SOA rule, and the fact that different resolvers cached at different moments.