Ch. 9 · Python

Python Threads, Processes and Workload Choice

Python Threads, Processes and Workload Choice. Learn the reasoning, a practical example, common mistakes and an interview exercise.

~2 min readadvancedupdated Oct 3, 2026

Concurrency choices depend on I/O, CPU work, runtime configuration and shared-state requirements. Do not infer universal parallelism guarantees from syntax.

Before you start

You should know Python functions, collections and exceptions. Use a small isolated script or interactive session to trace the example. Pay attention to when objects are created and when work executes; iteration, binding and mutation can happen at different points in a program.

The practical goal is to reason through this situation: Threads can overlap blocking I/O; separate processes can distribute CPU work with serialization overhead. Read the walkthrough first, then try the interview exercise before opening its answer. The important part is explaining the decision and its consequences, rather than remembering a definition alone.

Step-by-step walkthrough

Step 1: Classify the bottleneck

Separate waiting on I/O from sustained CPU computation.

Step 2: Inspect runtime and library

Interpreter build and native code affect actual parallel behavior.

Step 3: Benchmark bounded alternatives

Compare threads and processes including copying, startup and memory costs.

Worked scenario

Threads can overlap blocking I/O; separate processes can distribute CPU work with serialization overhead.

An image library may release interpreter constraints during native work, allowing threads to behave differently from a pure-Python calculation. Separate processes isolate state and can distribute CPU work but copy or serialize task data. Benchmark the actual operation rather than selecting execution from a universal slogan about the GIL.

Common mistake

GIL behavior varies by interpreter and build, and native libraries may release it.

Verify the behavior

Use identical datasets and worker limits; measure throughput, memory and transfer overhead.

Interview exercise

Choose execution for an image batch.

Answer and reasoning

Measure the library and runtime, then compare bounded threads or processes including transfer cost and memory use.

Continue learning

Compare the scenario with the Python interview questions and test your understanding with the Python MCQs. For terminology and implementation details, consult the reference material.

More in Python

read ✓Python · hard

Python asyncio.gather and Timeouts

Run coroutines concurrently with asyncio.gather, enforce timeouts, and handle partial failures and cancellation correctly.

~2 min readread →
read ✓Python · mid

Python Bounded Async Worker Queues

Use asyncio.Queue and a fixed worker pool to bound pending work. Trace backpressure, shutdown and failure ownership.

~3 min readread →
read ✓Python · mid

Python Asyncio and Blocking Functions

Python Asyncio and Blocking Functions. Learn the reasoning, a practical example, common mistakes and an interview exercise.

~2 min readread →
esc