Paging, Page Faults and Memory Access

Candidates often hear “page fault” and think “crash”. That’s backwards. A page fault is the hardware saying “I can’t finish this memory access with the page tables as they are; kernel, please sort it out.” Usually the kernel sorts it out in a microsecond or so and the program carries on, unaware. Your process probably took thousands of faults just starting up. The interview skill is knowing which kind of fault you’re looking at, what it costs, and how to measure it.

What a page table entry records

Virtual memory is divided into fixed-size pages (16 KiB on Apple Silicon, 4 KiB on most x86-64 systems), and physical memory into frames of the same size. For each virtual page, the page table entry holds the frame number plus a handful of bits. The ones that matter here are:

• present (valid): is there a frame behind this page right now?
• permissions: read, write, execute, and whether user mode may access it
• accessed and dirty: set by hardware when the page is used or written, and read by the kernel when it decides what to evict

The MMU checks these on every access. If the page isn’t present, or the access breaks a permission, the CPU stops before the instruction completes and traps into the kernel. That trap is the page fault.

Three outcomes of a fault

The kernel looks up the faulting address in its own record of the process’s memory regions (Linux calls them VMAs) and ends up in one of three places.

A minor fault can be resolved without touching storage. The classic case is the first write to freshly allocated memory: malloc and mmap only reserve address space, so the first touch of each page faults, and the kernel hands over a zeroed frame. Other minor faults: the page already sits in the page cache because another process read the file, or a copy-on-write page needs copying after fork.

A major fault needs I/O. The page belongs to a file that isn’t in memory yet, or it was swapped out, so the thread blocks until storage returns the data. That’s tens of microseconds on a fast SSD and milliseconds on a disk or network filesystem.

An invalid fault has no fix. The address isn’t in any region, or the access isn’t permitted (writing to read-only code, say). The kernel sends SIGSEGV, and that’s the crash people think of. A close cousin is SIGBUS, which you get when touching an mmap of a file beyond its current end.

For the first two, the kernel updates the page table entry and returns to the same instruction, which runs again and now succeeds. That restart is why demand paging is invisible to programs. All they notice is time.

Counting minor faults

This Python script maps 64 MiB of anonymous memory, writes one byte per page, then does it again. resource.getrusage reports the process’s fault counters:

import mmap, resource, time

def faults():
    r = resource.getrusage(resource.RUSAGE_SELF)
    return r.ru_minflt, r.ru_majflt

PAGE = mmap.PAGESIZE
SIZE = 64 << 20
m = mmap.mmap(-1, SIZE)
print(f"page size {PAGE} bytes, mapping {SIZE >> 20} MiB = {SIZE // PAGE} pages")

for label in ("first touch", "second touch"):
    f0 = faults(); t0 = time.perf_counter()
    for off in range(0, SIZE, PAGE):
        m[off] = 1
    t = time.perf_counter() - t0; f1 = faults()
    print(f"{label:12}: minor faults {f1[0]-f0[0]:5}  major {f1[1]-f0[1]}  {t*1000:6.1f} ms")

page size 16384 bytes, mapping 64 MiB = 4096 pages
first touch : minor faults  4096  major 0     6.2 ms
second touch: minor faults     0  major 0     4.8 ms

One fault per page, exactly as predicted, and none the second time because the entries are now present. The Python loop itself is slow enough to hide the cost, so I wrote the same test in C over 256 MiB. Across five runs the first pass cost between 0.9 and 4 µs per page (allocating and zeroing a 16 KiB frame, plus the trap), and the second pass between 0.2 and 1 µs per page with no faults at all. The second-pass cost is TLB and cache misses, since every access lands on a different page.

One run was different. Its second pass took 5,993 faults, and vm_stat showed about 11,500 decompressions system-wide during that run. This 8 GB machine was holding roughly 12 GB of compressed pages at the time, so the kernel was most likely compressing some of the test’s pages between passes and faulting them back in. “Touched once, resident forever” only holds while memory is plentiful.

Paying for I/O: a major fault

To see major faults, the file has to be absent from the page cache. On macOS, fcntl(fd, F_NOCACHE, 1) while writing keeps the data from staying cached. The script writes 128 MiB that way, then maps it read-only and reads one byte per page, twice:

fd = os.open(path, os.O_RDONLY)
m = mmap.mmap(fd, SIZE, access=mmap.ACCESS_READ)
for label in ("first pass", "second pass"):
    f0 = faults(); t0 = time.perf_counter(); s = 0
    for off in range(0, SIZE, mmap.PAGESIZE):
        s += m[off]
    t = time.perf_counter() - t0; f1 = faults()
    print(f"{label:11}: minor {f1[0]-f0[0]:5}  major {f1[1]-f0[1]:5}  {t*1000:7.1f} ms")

First run of the program, then a second run straight after:

first pass : minor     7  major  8192    265.8 ms
second pass: minor     0  major     0      1.9 ms

first pass : minor  8192  major     0      7.7 ms
second pass: minor     0  major     0      2.2 ms

The first run took 8,192 major faults, one per 16 KiB page, and 266 ms. By the second run the file was in the page cache, so the same 8,192 faults were all minor: the frames already held the data and just needed mapping into the new process. That run’s first pass took 7.7 ms, about 34 times faster, with identical code. When someone says “the second benchmark run was much faster”, this is very often why.

Where faults show up in real systems

Faults matter when they land on a latency-sensitive path. A freshly deployed service takes minor faults as its heap grows and major faults as it reads memory-mapped indexes for the first time. A database with a large buffer pool takes first-touch faults during its first hour under load. Garbage collectors walking a cold heap take them too.

Measure before you guess. getrusage gives counters from inside a process. On macOS, /usr/bin/time -l prints them under BSD names: minor faults appear as page reclaims and major faults as page faults. On Linux, /usr/bin/time -v labels them “Minor (reclaiming a frame)” and “Major (requiring I/O)”, and ps -o min_flt,maj_flt -p <pid> or perf stat -e page-faults give the same data (I ran the macOS tools only).

When first-touch cost is the problem, the fixes all move faults earlier. You can touch memory at start-up (the JVM’s -XX:+AlwaysPreTouch does this for the heap), map with MAP_POPULATE on Linux, ask for readahead with madvise(MADV_WILLNEED), or lock critical pages with mlock so they can’t be evicted later.

Interview practice: slow for ten minutes after every deploy

After each deploy, a search service’s p99 latency is three times normal for about ten minutes and then settles. CPU isn’t saturated. The service memory-maps a 6 GB index file. How might page faults explain this, and what would you do?

Answer and reasoning

A new process starts with no pages mapped. Every first access to a page of the index is a fault: minor if the old process’s pages are still in the page cache, major if they’ve been evicted or the deploy moved to a new host. Major faults block the request thread on storage, so the slowest requests are the ones that happen to touch cold pages. Once the hot part of the index is resident, faults stop and latency returns to normal, which matches the ten-minute shape.

To confirm it, I’d chart the process’s major fault rate (maj_flt on Linux, or getrusage from inside) alongside p99 after a deploy. If they fall together, that’s the cause.

Fixes, roughly in order of effort: warm the index before the instance takes traffic, either by reading it sequentially or with madvise(MADV_WILLNEED); send synthetic queries during a warm-up phase before joining the load balancer; and make sure the host has enough RAM for the hot set so pages aren’t evicted while the service runs. Pre-faulting only helps if the memory then stays resident.

Further reading on memory

• Operating Systems interview questions
• Operating Systems MCQs
• Virtual Memory and Address-Space Isolation
• Operating Systems: Swap and Thrashing
• OSTEP, Beyond Physical Memory: Mechanisms, the chapter on the present bit and page-fault handling