Ch. 9 · Python

Python Generators and Single-Pass Iteration

Python Generators and Single-Pass Iteration. Learn the reasoning, a practical example, common mistakes and an interview exercise.

~2 min readbeginnerupdated Oct 3, 2026

Generators produce values on demand and preserve local execution state. They reduce eager storage but still require ownership of underlying resources.

Before you start

You should know Python functions, collections and exceptions. Use a small isolated script or interactive session to trace the example. Pay attention to when objects are created and when work executes; iteration, binding and mutation can happen at different points in a program.

The practical goal is to reason through this situation: Process log lines one at a time rather than loading the whole file. Read the walkthrough first, then try the interview exercise before opening its answer. The important part is explaining the decision and its consequences, rather than remembering a definition alone.

Step-by-step walkthrough

Step 1: Identify suspension points

Yield produces one value and preserves execution state for resumption.

Step 2: Own source lifetime

A file-backed generator must keep or release its file under an explicit scope.

Step 3: Plan repeated summaries

Accumulate together or recreate the source instead of reusing a consumed generator.

Worked scenario

Process log lines one at a time rather than loading the whole file.

A log generator reads one line at a time, so early termination avoids loading the entire file. However, stopping halfway means cleanup still matters. Computing count and errors in one pass avoids a second traversal; buffering trades memory for replayability when repeated access is genuinely required.

Common mistake

A consumed generator cannot be replayed without recreating it.

Verify the behavior

Test empty input, early stop and full exhaustion; verify resource closure.

Interview exercise

Compute two independent summaries.

Answer and reasoning

Use a single pass accumulating both, recreate the source, or deliberately buffer when repeat traversal is required.

Continue learning

Compare the scenario with the Python interview questions and test your understanding with the Python MCQs. For terminology and implementation details, consult the reference material.

More in Python

read ✓Python · hard

Python asyncio.gather and Timeouts

Run coroutines concurrently with asyncio.gather, enforce timeouts, and handle partial failures and cancellation correctly.

~2 min readread →
read ✓Python · mid

Python Bounded Async Worker Queues

Use asyncio.Queue and a fixed worker pool to bound pending work. Trace backpressure, shutdown and failure ownership.

~3 min readread →
read ✓Python · mid

Python Asyncio and Blocking Functions

Python Asyncio and Blocking Functions. Learn the reasoning, a practical example, common mistakes and an interview exercise.

~2 min readread →
esc