“Should this use threads or processes?” is one of the most common Python system-design questions, and the honest answer starts with “what is the work waiting on?” CPython’s global interpreter lock (GIL) lets only one thread execute Python bytecode at a time, so threads and processes behave very differently depending on whether the work is computing or waiting. Rather than argue from theory, this guide measures it. All timings come from one run on an 8-core Mac with Python 3.14 (the standard build, GIL enabled); your numbers will differ, but the shape won’t.
Measuring the two kinds of work
Two functions stand in for the two workload types. One is a pure-Python loop (CPU-bound); the other sleeps, the way a network call waits for a response (I/O-bound).
import os, sys, time
from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutor
def cpu_task(n):
total = 0
for i in range(n):
total += i * i
return total
def io_task(_):
time.sleep(0.2)
return "ok"
def timed(label, fn):
start = time.perf_counter()
fn()
print(f"{label:<28}{time.perf_counter() - start:.2f}s")
if __name__ == "__main__":
print("cores:", os.cpu_count(), "| GIL enabled:", sys._is_gil_enabled())
N = 5_000_000
timed("cpu, sequential x4", lambda: [cpu_task(N) for _ in range(4)])
with ThreadPoolExecutor(4) as ex:
timed("cpu, 4 threads", lambda: list(ex.map(cpu_task, [N] * 4)))
with ProcessPoolExecutor(4) as ex:
list(ex.map(cpu_task, [10] * 4)) # warm up workers
timed("cpu, 4 processes", lambda: list(ex.map(cpu_task, [N] * 4)))
timed("io, sequential x20", lambda: [io_task(i) for i in range(20)])
with ThreadPoolExecutor(20) as ex:
timed("io, 20 threads", lambda: list(ex.map(io_task, range(20))))cores: 8 | GIL enabled: True
cpu, sequential x4 0.83s
cpu, 4 threads 0.84s
cpu, 4 processes 0.23s
io, sequential x20 4.06s
io, 20 threads 0.22sFour threads doing CPU work took exactly as long as doing it sequentially. Only one thread held the GIL at a time, so they took turns on one core. Four processes, each with its own interpreter and its own GIL, ran on four cores and finished in about a quarter of the time.
The I/O case flips. Twenty sleeps in a row took 4 seconds. Twenty threads took 0.22s, because time.sleep, like socket reads, file reads and most database driver calls, releases the GIL while it waits. While one thread waits, another runs. Threads are a perfectly good tool for I/O-bound Python.
Why the GIL doesn’t settle every question
Two misconceptions come up repeatedly in interviews.
“Python can’t do parallel CPU work.” Pure-Python code can’t, in threads, on the default build. But C extensions can release the GIL during heavy computation. NumPy matrix operations, hashlib on large buffers, zlib compression and many image libraries do this, so threads calling them can use several cores. Before deciding, check whether the hot loop is Python bytecode or native code.
“The GIL makes my code thread-safe.” It makes individual bytecode operations safe for the interpreter’s internals. It doesn’t make your sequences of operations atomic. A thread can be switched out between reading a value and writing back an update:
import threading
balance = 0
def fee(amount):
return amount + 1 # any function call between the read and the write
def deposit():
global balance
for _ in range(200_000):
current = balance # read
balance = fee(current) # write
threads = [threading.Thread(target=deposit) for _ in range(4)]
for t in threads: t.start()
for t in threads: t.join()
print("expected 800000, got", balance)Ten runs of this script printed 800000 five times, and 731004, 729372, 600000, 468544 and 733772 the other five. That’s the nasty part: it often works. A test that runs once can pass, and the bug shows up under production load. The interpreter checks whether to switch threads at certain points, such as function calls and loop back-edges, and the call to fee between the read and the write gives it a chance. Wrapping the read-modify-write in a lock (with lock: balance = fee(balance)) printed 800000 every time.
The general rule: any state shared between threads and modified by more than one of them needs a lock, or should be redesigned so it isn’t shared. queue.Queue is the usual way to pass work between threads without sharing mutable objects.
The free-threaded build
Python 3.13 introduced an optional free-threaded build (python3.13t, PEP 703) without the GIL, and 3.14 made it officially supported. On that build, the CPU-bound thread example above can use several cores. It’s still a separate build, many C extensions need updates to support it, and single-threaded code runs somewhat slower on it. sys._is_gil_enabled() tells you which you’re running. For interviews, it’s worth knowing it exists and that the default most teams deploy still has a GIL.
The cost of processes
Processes aren’t free speed. Each task’s arguments and results are pickled, sent through a pipe, and unpickled on the other side. When the work per item is tiny, that overhead dominates:
def double(x): return x * 2
data = list(range(200_000))
# in-process: [double(x) for x in data]
# pool: list(ex.map(double, data))
# pool, batched: list(ex.map(double, data, chunksize=10_000))in-process 0.011s
pool, chunksize=1 14.33s
pool, chunksize=10k 0.039sSending 200,000 tiny tasks one at a time was over a thousand times slower than a plain loop. Batching them with chunksize brought it close, but still slower than not using processes at all. Processes pay off when each task does a lot of computation relative to the data it moves: resizing an image, parsing a large file, running a simulation.
Two practical constraints follow from pickling. Functions passed to a process pool must be importable by name, so a lambda fails:
_pickle.PicklingError: Can't pickle <function <lambda> at 0x1060dd2d0>: it's not found as __main__.<lambda>And on macOS and Windows the default start method is spawn, which imports your main module in every worker. That’s why the example has an if __name__ == "__main__": guard; without it, each worker would try to start its own pool.
A decision procedure
When you’re asked to choose, walk through it in this order:
- Is the work waiting on I/O (HTTP calls, database queries, file reads)? Use threads (
ThreadPoolExecutor) or asyncio. Asyncio scales to thousands of concurrent connections with less memory per task, but every library in the path must be async; threads work with ordinary blocking libraries. - Is it CPU-bound in pure Python? Use processes (
ProcessPoolExecutor), sized to the number of cores, with batched inputs. - Is it CPU-bound in a native library that releases the GIL? Threads may be enough, and they avoid pickling large arrays.
- Is it a long-running background job that must survive restarts? None of the above; use a job queue (Celery, RQ, a cloud queue) so work isn’t lost when a process dies.
And regardless of the choice, bound concurrency to what downstream systems can handle. Fifty threads against a database that allows twenty connections just moves the queue somewhere less visible.
Interview exercise: the slow thumbnail service
A service downloads 500 images from S3, resizes each with Pillow, and uploads the thumbnails. It processes them one at a time and takes 10 minutes. Profiling shows about 70% of the time in network transfer and 30% in resizing. How would you speed it up?
Answer
Split the stages by workload. Downloads and uploads are I/O-bound, so a ThreadPoolExecutor with, say, 16 to 32 workers lets many transfers wait at once; that attacks the 70% directly and should cut it to a small fraction, limited by bandwidth and S3 request rates. Resizing is CPU work. Pillow releases the GIL for many of its core operations, so I’d first measure resizing with threads; if it scales across cores, a single thread pool can do the whole pipeline. If it doesn’t, move resizing to a ProcessPoolExecutor with one worker per core, passing file paths or bytes rather than large Python objects to keep pickling cheap. I’d bound each stage (a semaphore or bounded queue) so downloads can’t run far ahead and fill memory with unprocessed images, and I’d make each image’s processing independent and retryable so a single failure doesn’t stop the batch. Finally, I’d measure after each change, because the 70/30 split will shift once the network part gets faster.