When a program reads a file, the kernel keeps a copy of the data in RAM in case anyone asks again. That copy is the page cache, and it’s the reason a second grep over a large directory finishes instantly, the reason your laptop’s memory always looks nearly full, and the reason so many I/O benchmarks report numbers no disk could produce. Interviewers ask about it because it separates people who’ve measured I/O from people who’ve only read about it.
Where the cache sits in the read path
Every ordinary read() goes through the cache. The kernel works out which pages of the file the request covers and looks each one up. On a hit, it copies the bytes from the cached page into your buffer: a memory copy, nothing more. On a miss, it allocates a page, asks the storage device for the data, puts the thread to sleep until the I/O completes, and then copies. It usually reads ahead too, fetching the next pages before you ask, because sequential access is common.
Memory-mapped files use the same pages. When you mmap a file, the page tables point straight at the cached pages, so there’s no copy at all; a miss shows up as a page fault instead of a slow read().
Writes land in the cache first as well. write() copies your data into cached pages, marks them dirty and returns; the kernel writes them to storage later. That’s why write() is fast and also why it doesn’t mean the data is safe, which is a separate topic.
The cache is the same pool of memory as everything else. File pages that nobody is using are first in line when a program needs memory, so the kernel evicts them, roughly least recently used first. Linux keeps active and inactive lists (newer kernels can use a multi-generational LRU); macOS tracks file-backed pages in its own way. The effect is the same: the cache grows into all unused RAM and shrinks when programs need it.
Cold and warm, measured
To measure a cold read you first need data that isn’t cached, which is harder than it sounds since writing a file caches it. On macOS, fcntl(fd, F_NOCACHE, 1) on the writing descriptor stops the pages from staying behind. mincore() then reports which pages of a mapping are resident, which lets the script check its own starting point:
import os, fcntl, mmap, time, ctypes
PATH, SIZE, CHUNK = "data.bin", 256 << 20, 1 << 20
libc = ctypes.CDLL(None, use_errno=True)
libc.mmap.restype = ctypes.c_void_p
libc.mmap.argtypes = [ctypes.c_void_p, ctypes.c_size_t, ctypes.c_int, ctypes.c_int, ctypes.c_int, ctypes.c_long]
libc.mincore.argtypes = [ctypes.c_void_p, ctypes.c_size_t, ctypes.c_char_p]
libc.munmap.argtypes = [ctypes.c_void_p, ctypes.c_size_t]
def write_uncached():
fd = os.open(PATH, os.O_CREAT | os.O_WRONLY | os.O_TRUNC, 0o644)
fcntl.fcntl(fd, fcntl.F_NOCACHE, 1) # macOS: don't leave these pages in the cache
block = os.urandom(CHUNK)
for _ in range(SIZE // CHUNK): os.write(fd, block)
os.fsync(fd); os.close(fd)
def resident():
"""Fraction of the file's pages in the page cache, via mincore()."""
fd = os.open(PATH, os.O_RDONLY)
addr = libc.mmap(None, SIZE, mmap.PROT_READ, mmap.MAP_SHARED, fd, 0)
pages = SIZE // mmap.PAGESIZE
vec = ctypes.create_string_buffer(pages)
libc.mincore(addr, SIZE, vec)
libc.munmap(addr, SIZE); os.close(fd)
return sum(b & 1 for b in vec.raw) / pages
def read_all():
fd = os.open(PATH, os.O_RDONLY)
t0 = time.perf_counter()
while os.read(fd, CHUNK): pass
dt = time.perf_counter() - t0; os.close(fd)
return SIZE / dt / 2**20Reading the 256 MiB file sequentially three times, then rewriting it uncached and doing 2,000 random 4 KiB pread calls twice:
after writing with F_NOCACHE: 0.0% of pages cached
first read (cold) 2395 MiB/s then 100.0% cached
second read (warm) 10093 MiB/s then 100.0% cached
third read (warm) 12541 MiB/s then 100.0% cached
rewritten uncached: 0.0% cached
2000 random 4 KiB reads, cold: 189.4 us each then 11.5% cached
2000 random 4 KiB reads, warm: 1.6 us eachAcross three runs, cold sequential reads came in at 2.3 to 2.6 GiB/s and warm ones at 10 to 16 GiB/s. The M1’s internal SSD is fast, so the sequential gap is only four to six times. Random access is where the cache really shows: 154 to 189 µs per cold read against 0.9 to 1.6 µs per warm one. A benchmark that runs the random workload twice and reports the second run is off by more than 100x.
There’s a detail in the last numbers. 2,000 reads of 4 KiB is about 8 MiB, yet 11.5% of the file, about 29 MiB, ended up cached. The cache works in whole pages, and on Apple Silicon a page is 16 KiB, so each 4 KiB read brought in four times what it asked for. On x86 Linux with 4 KiB pages the ratio would differ, and readahead can fetch more still.
Benchmarking without fooling yourself
The first rule is to say which state you measured: cold, warm, or a realistic mix. Production is usually in between, with a hot working set cached and a long tail that isn’t. Some ways to control the state:
- Drop the cache. On Linux,
sync; echo 3 > /proc/sys/vm/drop_cachesempties it system-wide (root only, and it affects everything on the machine). On macOS,purgedoes the same and also needs root, so I used theF_NOCACHEapproach here instead. - Evict one file.
posix_fadvise(fd, 0, 0, POSIX_FADV_DONTNEED)asks Linux to drop a single file’s clean pages, without root. - Bypass the cache.
O_DIRECTon Linux, orF_NOCACHEon macOS, sends I/O straight to the device. Tools likefioexpose this asdirect=1. - Outgrow it. Use a data set several times larger than RAM so the cache can’t hold it.
Watch for the opposite mistake too: a write benchmark without fsync measures how fast the kernel copies into memory, not how fast the disk writes.
Free memory that isn’t free
Because the cache fills spare RAM, a healthy machine shows little free memory. On Linux, free -h reports it under buff/cache, and the available column estimates how much could be handed to programs by evicting cache. Alarms on “free memory” alone fire constantly for no reason. On macOS, vm_stat lists File-backed pages, and Activity Monitor calls them Cached Files.
Containers complicate it. Linux charges page cache to the cgroup whose process first touched the page, so reading big files raises a container’s memory usage toward its limit. That cache is reclaimable, which is why Kubernetes reports a “working set” figure that subtracts inactive file pages from total usage. A container that reads a lot of files can look close to its limit without being in danger, but cache that’s actively used (the active file list) is counted in the working set too.
When to bypass the cache
Two situations call for going around it. Databases that manage their own buffer pool, such as InnoDB configured with O_DIRECT, would otherwise keep every page twice: once in their pool and once in the kernel’s. (PostgreSQL takes the other approach and deliberately relies on the OS cache alongside a smaller buffer pool.) And one-pass bulk reads, like backups, scans or log shipping, pull in data that will never be read again and evict pages other programs needed, a problem called cache pollution.
Interview exercise: the backup that slows the database
Every night at 02:00 a backup job reads about 500 GB of files on a database host with 128 GB of RAM. For roughly an hour afterwards, the database’s read latency is several times higher than usual, then it recovers. Explain and fix it.
Answer
The backup streams 500 GB through the page cache, and every page it reads is the most recently used, so the kernel evicts older pages to make room, including the database’s hot data files. When the backup ends, the database finds its working set gone. Every query that would have been a memory copy is now a storage read, a gap of more than 100x in the random-read test above, until the hot pages are read back in. The hour of slow queries is the cache warming up again.
To confirm, I’d watch the database’s read I/O and the host’s cache size during and after the backup: disk reads should jump right after 02:00 and decay as latency recovers.
Fixes, from least to most invasive: have the backup tool drop its pages as it goes with posix_fadvise(POSIX_FADV_DONTNEED) on each file once read, which many backup tools and rsync-style utilities support or can be patched to do; read with O_DIRECT so the backup never enters the cache; or run the backup in a cgroup with a memory limit so its page cache is charged to it and reclaimed within its own limit instead of evicting the database’s pages. Lowering its I/O priority (ionice) helps the database’s I/O during the backup, but doesn’t stop the eviction, so it isn’t the fix on its own. Running backups from a replica avoids the problem entirely.
Further reading on storage
- Operating Systems interview questions
- Operating Systems MCQs
- Paging, Page Faults and Memory Access
- Operating Systems: Journaling and Durability
- posix_fadvise(2) on man7.org, for per-file cache hints such as POSIX_FADV_DONTNEED