Ch. 22 · Software Testing

Flaky Tests and Root-Cause Evidence

Flaky Tests and Root-Cause Evidence. Learn the reasoning, a practical example, common mistakes and an interview exercise.

~2 min readintermediateupdated Oct 3, 2026

A flaky test changes outcome without an intended code change. Common causes include races, shared state and uncontrolled external dependencies.

Before you start

You should understand inputs, expected outputs and the boundaries of the component under test. State the risk a test should detect before choosing a tool or mock. Distinguish controlled dependencies from real integrations so the result does not imply more coverage than the test actually provides.

The practical goal is to reason through this situation: A test passes alone but fails in a suite because another test changes global configuration. Read the walkthrough first, then try the interview exercise before opening its answer. The important part is explaining the decision and its consequences, rather than remembering a definition alone.

Step-by-step walkthrough

Step 1: Capture variability

Record order, environment and timing when outcomes change.

Step 2: Control one suspected cause

Isolate shared state or completion order before adding retries.

Step 3: Fix ownership

Eliminate the race or leak rather than merely masking its visible failure.

Worked scenario

A test passes alone but fails in a suite because another test changes global configuration.

A test passes alone but fails after another test changes timezone configuration. Retrying may happen to run it under a restored state, hiding the dependency. Give each test owned configuration setup and cleanup or inject the relevant setting explicitly, then confirm both execution orders produce the same outcome.

Common mistake

Automatic retries can reduce noise while hiding the underlying fault.

Verify the behavior

Run varied orders and repeated executions with the suspected shared state controlled.

Interview exercise

Investigate one flake.

Answer and reasoning

Capture order, timing and environment, reproduce under controlled variation and fix the shared ownership or synchronization issue.

Continue learning

Compare the scenario with the Software Testing interview questions and test your understanding with the Software Testing MCQs. For terminology and implementation details, consult the reference material.

More in Software Testing

read ✓Software Testing · hard

Testing Time with an Injected Clock

Inject a clock instead of calling the system time, freeze it in tests, and remove the sleeps that make tests flaky.

~2 min readread →
read ✓Software Testing · mid

Code Coverage and Its Limits

Read line and branch coverage as a signal, not a goal, and avoid the tests that chase the number without checking behavior.

~2 min readread →
read ✓Software Testing · hard

Scoping End-to-End Tests

Reserve end-to-end tests for critical user journeys, use stable selectors, and avoid duplicating coverage the lower tests already have.

~2 min readread →
esc