A context switch is the kernel taking a CPU core away from one thread and giving it to another. It happens thousands of times a second on any busy machine, mostly unnoticed. It becomes an interview topic when someone asks why a service with 2,000 threads is slow, or why vmstat shows a huge cs column, and the useful answer separates the cost of the switch itself from the things that cause switches in the first place.
What gets saved, and what gets lost
A thread’s execution state is a small set of values: the program counter, the stack pointer, the general-purpose registers, the floating-point and SIMD registers, and a few status registers. To switch, the kernel (already running because of a system call, an interrupt or a timer tick) stores the outgoing thread’s values in its kernel data structures, asks the scheduler which thread runs next, and loads that thread’s values. When the new thread belongs to a different process, the kernel also points the MMU at that process’s page tables.
That’s the direct cost, and on modern hardware it’s in the low microseconds at most. The indirect cost is harder to see. The incoming thread finds the CPU caches, the TLB and the branch predictor full of the previous thread’s state. Until it has pulled its own data back in, it runs slower. For a thread with a large working set, that warm-up can cost more than the switch.
Two related terms get mixed up. A mode switch is the transition from user mode to kernel mode, which every system call makes; it doesn’t change which thread is running. A context switch changes the running thread. A system call only causes a context switch if it blocks.
Voluntary and involuntary switches
Switches come in two kinds, and telling them apart is the first step in any diagnosis.
A voluntary switch happens when a thread can’t continue: it’s waiting for a disk read, a network packet, a lock, or a message on a pipe. It gives up the CPU, and the scheduler runs someone else. Lots of voluntary switches means lots of blocking.
An involuntary switch happens when the scheduler takes the CPU away from a thread that could have kept running: its time slice expired, or a higher-priority thread woke up. Lots of involuntary switches means more runnable threads than cores, or (in containers) the CPU quota running out.
macOS reports both in /usr/bin/time -l. A CPU-bound Python loop of about 1.3 seconds:
1.28 real 1.26 user 0.01 sys
1 voluntary context switches
608 involuntary context switchesThe loop never blocked, so nearly every switch was a preemption, about one every 2 ms on a machine with other work running. One platform quirk worth knowing: a Python script doing 1,000 time.sleep(0.001) calls showed 19 voluntary and 1,090 involuntary switches on this Mac. On Linux, sleeping is the textbook voluntary switch (I didn’t run the Linux case). Pipe reads, below, are counted as voluntary on both. Check how your platform classifies a wait before drawing conclusions from the split.
Measuring a handoff between two processes
The classic way to measure switching is ping-pong. Two processes share a pair of pipes; the parent writes a byte and waits for the reply, the child reads it and writes it back. Each process blocks on every round trip, so the kernel has to put one to sleep and wake the other each time:
#define N 100000
int main(void){
int ab[2], ba[2]; pipe(ab); pipe(ba); char c = 'x';
pid_t pid = fork();
if (pid == 0) { /* child: echo every byte back */
for (int i = 0; i < N; i++) { read(ab[0], &c, 1); write(ba[1], &c, 1); }
_exit(0);
}
double t = now();
for (int i = 0; i < N; i++) { write(ab[1], &c, 1); read(ba[0], &c, 1); }
t = now() - t;
waitpid(pid, NULL, 0);
/* print the time per round trip and getrusage switch counts for both sides */
}Two runs on the M1:
100000 round trips: 3.79 us each
parent: 99997 voluntary, 200 involuntary switches
child: 99991 voluntary, 175 involuntary switches
100000 round trips: 3.34 us each
parent: 99993 voluntary, 120 involuntary switches
child: 99989 voluntary, 113 involuntary switchesEach process made almost exactly one voluntary switch per round trip, as designed. A round trip of 3.3 to 3.8 µs contains four system calls and two block-and-wake cycles, so each handoff costs around 1.5 µs. On a multicore machine the two processes may sit on different cores, in which case part of that is the cost of one core waking another. Linux has this exact benchmark built in as perf bench sched pipe.
Put 1.5 µs in context. A design that hands every request through four queues between thread pools pays that four times per request, plus cache warm-up on each new thread. At 100,000 requests a second, that’s 600,000 handoffs and roughly a full core spent on switching alone.
More threads than cores is less scary than it sounds
A common belief is that running more threads than cores is a disaster because of context switches. For CPU-bound work, it mostly isn’t. This C program does a fixed amount of total work, each thread repeatedly sweeping its own 512 KiB buffer, split across 8, 64 or 512 threads on the 8-core M1:
8 threads: 0.44 s wall, 3567 involuntary switches
64 threads: 0.41 s wall, 3667 involuntary switches
512 threads: 0.51 s wall, 5187 involuntary switches
8 threads: 0.45 s wall, 2002 involuntary switches
64 threads: 0.43 s wall, 3741 involuntary switches
512 threads: 0.45 s wall, 5708 involuntary switchesGoing from 8 to 512 threads cost between nothing and 15%. The scheduler’s time slices are long compared with a switch, so a few thousand extra switches barely register. The real costs of oversubscription are elsewhere:
- Latency. With 512 runnable threads on 8 cores, each thread waits for many others’ slices before it runs again. Total throughput holds up while any single request gets slower.
- Memory. Every thread has a stack (8 MiB reserved on Linux by default, 512 KiB for secondary threads on macOS) and kernel structures.
- Lock holder preemption. If a thread is preempted while holding a lock, every thread that needs that lock waits for it to be scheduled again. With many runnable threads, that can take a whole round of time slices.
- Blocking handoffs. The ping-pong pattern, repeated per item, is where switch costs pile up.
So size CPU-bound pools near the core count to keep latency predictable, and handle many concurrent I/O waits with an event loop (epoll, kqueue) or async runtime, not one blocked thread per connection.
Reading the counters in production
On Linux, /proc/<pid>/status has voluntary_ctxt_switches and nonvoluntary_ctxt_switches per process (and per thread under /proc/<pid>/task/), pidstat -w 1 shows rates, and the cs column of vmstat 1 gives the system total. A rising involuntary rate on a latency-sensitive service suggests CPU contention or cgroup throttling. A very high voluntary rate suggests fine-grained blocking: tiny I/O operations, contended locks, or work bouncing between threads. Off-CPU profiling (for example perf sched or off-CPU flame graphs) shows where threads block.
Interview exercise: 400,000 switches a second
A Java service on a 16-core host runs a pool of 800 worker threads. Each request passes through three internal queues handled by different pools. At peak, vmstat shows 400,000 context switches per second, CPU is 70% busy with a lot of system time, and p99 latency is climbing. What do you look at, and what would you change?
Answer
First, split the switches. Per-thread voluntary and involuntary counts tell you whether threads are blocking or being preempted. With three queue hops per request, I’d expect mostly voluntary switches: every hop parks a thread on an empty queue and wakes one on the other side. At, say, 30,000 requests per second, three hops with a park and a wake-up each is already 180,000 switches a second before counting any I/O waits, which is the right order of magnitude for the symptom. The high system time is the kernel doing that waking and sleeping.
Changes, roughly in order: remove hops that don’t need separate pools, running those stages inline on the same thread so each request costs one handoff instead of three. Batch where a hop is required, draining several items per wake-up. Shrink the 800-thread pool toward what the workload needs; if threads exist only to wait on I/O, use non-blocking I/O instead. Check lock contention too, since a hot lock produces the same voluntary-switch signature.
Then measure again. If involuntary switches dominate instead, the problem is too many runnable threads or CPU throttling, and the fix is fewer threads or more CPU, not fewer queues.
Related notes
- Operating Systems interview questions
- Operating Systems MCQs
- CPU Scheduling and Response-Time Goals
- Operating Systems: Thread Pools
- OSTEP, Mechanism: Limited Direct Execution, which walks through the timer interrupt and the switch routine