DNS questions show up in almost every networking interview, usually inside “what happens when you type a URL into the browser?” The basic answer (a name becomes an IP address) is easy. The follow-ups are where it gets interesting: who asks whom, where answers get cached, why your laptop sees a new server while a colleague still sees the old one, and why “DNS resolves fine” doesn’t mean the site works.
This note walks through all of that with dig against real public DNS, plus a small Node script.
The chain of servers behind one lookup
Your application doesn’t talk to the whole DNS system. It asks a stub resolver in the operating system, which forwards the question to a recursive resolver: your ISP’s, your company’s, or a public one like 1.1.1.1 or 8.8.8.8. The recursive resolver does the legwork, starting from the root if it has nothing cached.
dig +trace performs that walk itself, so you can watch each step:
dig +trace +nodnssec example.com. 508770 NS a.root-servers.net.
com. 172800 NS l.gtld-servers.net.
example.com. 172800 NS hera.ns.cloudflare.com.
example.com. 300 A 172.66.147.243
example.com. 300 A 104.20.23.154
;; Received 836 bytes from 193.0.14.129#53(k.root-servers.net) in 44 ms
;; Received 359 bytes from 192.41.162.30#53(l.gtld-servers.net) in 138 ms
;; Received 72 bytes from 173.245.58.162#53(hera.ns.cloudflare.com) in 8 ms(Output trimmed to one line per step.) A root server said “ask the .com servers”. A .com server said “example.com is served by Cloudflare’s nameservers”. Cloudflare’s nameserver, which is authoritative for the zone, gave the actual A records.
That’s three round trips and nearly 200 ms from cold. In practice it almost never happens that way, because each referral has its own TTL. The .com delegation is cached for 172,800 seconds (two days), so a busy resolver already knows where example.com’s nameservers are and only has to ask them.
Authoritative answers versus cached ones
Ask the authoritative server directly, with recursion turned off, and look at the flags:
dig +norecurse example.com @hera.ns.cloudflare.com;; flags: qr aa; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1
example.com. 300 IN A 172.66.147.243
example.com. 300 IN A 104.20.23.154aa means authoritative answer, and the TTL is the full 300 seconds the zone owner configured. Ask a recursive resolver and you get ra (recursion available) instead, with a TTL that counts down as the cached copy ages:
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1
example.com. 291 IN A 172.66.147.243Here’s a detail that surprises people. Querying both public resolvers four times, two seconds apart, and printing the TTL each one returned gave this:
1.1.1.1: 213 8.8.8.8: 300
1.1.1.1: 75 8.8.8.8: 300
1.1.1.1: 247 8.8.8.8: 298
1.1.1.1: 131 8.8.8.8: 300If 1.1.1.1 were one cache, the number would fall by two each time. Instead it jumps around, because a public resolver is many machines behind one anycast address, each with its own cache. So “the resolver has cached the old address” is really “some of the resolver’s caches have”. During a change, the same user can get the old answer, then the new one, then the old one again until every cache has expired.
Negative answers are cached too. Asking for a name that doesn’t exist returned no answer plus the zone’s SOA record, ending in 1800. That number tells resolvers how long they may remember “there’s nothing here”. If you query a hostname before you create it, you can be stuck waiting up to half an hour for resolvers to notice it exists.
Your program may not be asking DNS at all
There are more caches between your code and the recursive resolver. Node makes this visible because it has two different functions:
import dns from 'node:dns/promises';
for (const name of ['localhost', 'example.com']) {
const viaOs = await dns.lookup(name, { all: true }).catch((e) => e.code);
const viaDns = await dns.resolve4(name, { ttl: true }).catch((e) => e.code);
console.log(name, JSON.stringify(viaOs), JSON.stringify(viaDns));
}localhost
lookup (getaddrinfo, hosts file, OS cache): [{"address":"127.0.0.1","family":4},{"address":"::1","family":6}]
resolve4 (asks the DNS server directly): "ENOTFOUND"
example.com
lookup (getaddrinfo, hosts file, OS cache): [{"address":"172.66.147.243","family":4},{"address":"104.20.23.154","family":4}]
resolve4 (asks the DNS server directly): [{"address":"104.20.23.154","ttl":274},{"address":"172.66.147.243","ttl":274}]localhost comes from the hosts file, which lookup reads and DNS servers know nothing about. http.get and fetch use lookup, so an /etc/hosts entry left over from testing silently overrides real DNS. Also note that lookup returned no IPv6 addresses for example.com even though it has AAAA records. This machine has no IPv6 route, so the OS filtered them. Another host with IPv6 would try a different address for the same name.
Above the OS there are runtime caches (the JVM’s networkaddress.cache.ttl, the browser’s own DNS cache) and, more important, connection pools. A pooled connection never looks the name up again. A service holding keep-alive connections to an old IP will keep using it long after the TTL ran out.
Walking through a server move
Say app.example.com is moving from 203.0.113.10 to 198.51.100.20 and the A record has a TTL of 3600. A careful move goes like this:
-
Lower the TTL first, to 60, and then wait at least the old TTL (an hour) so every cache picks up the short value. Lowering it at the same moment as the change does nothing for caches that already hold the hour-long copy.
-
Test the new server without DNS. curl can pin a name to an address, which checks TLS and the application at the new IP before anyone is sent there. Against example.com’s real address it looks like this:
curl -s -o /dev/null -w '%{http_code} via %{remote_ip}\n' \ --resolve example.com:443:104.20.23.154 https://example.com/Terminal 200 via 104.20.23.154Text -
Change every record. Forgetting the AAAA record is a classic: IPv6 clients keep going to the old server while IPv4 clients move.
-
Keep the old server running, ideally proxying to the new one, and watch its access logs until traffic dies down. Pooled connections and resolvers that ignore short TTLs will trail in for a while.
-
Raise the TTL again once you’re settled.
If the new address resolves but browsers report a certificate error, DNS is done. The problem is the certificate or the TLS setup on the new server, and changing DNS again won’t fix it.
Interview practice
An hour ago your team moved a website to a new server. Your laptop sees the new site, a colleague in another office sees the old one, and a customer reports a certificate warning. How do you work out what’s going on?
Answer
I’d separate three questions: what the authoritative servers say, what different resolvers have cached, and whether the new server actually works. First, dig +norecurse against the zone’s nameservers to confirm the new A and AAAA records are published. Then the same query through my resolver and through my colleague’s. If my colleague’s resolver returns the old address with a TTL still counting down, it’s caching and will clear on its own; if the record’s TTL was an hour and we didn’t lower it in advance, that’s the explanation. I’d check their hosts file and whether the app they use holds pooled connections, too.
The certificate warning is a separate problem. I’d pin the hostname to the new IP with curl --resolve (or openssl s_client -servername) and look at the certificate: does it cover this exact hostname, is the chain complete, has it expired? If the customer resolves to the old server, its certificate may have lapsed since nobody expected traffic there. DNS working tells you where clients go; it doesn’t tell you what they find when they arrive.