A fixture is the known starting state a test runs against: a database with two accounts in it, a temporary directory with a config file, a clock set to a particular moment. Tests are only trustworthy when that starting state is the one the test author had in mind. The quickest way to lose that guarantee is to let tests share fixtures that they also modify.
Shared mutable fixtures produce a recognisable family of failures. A test passes when you run it alone and fails in the full suite. A test fails only on CI, where tests run in a different order or in parallel. Deleting an unrelated test makes another one start failing. All of them come from the same place: one test wrote something, and a later test read it.
Shared state turning into order-dependent failures
Here’s a tiny bank module backed by SQLite, and a test class that builds its database once for all tests, which is a tempting shortcut when setup feels slow:
# bank.py
SCHEMA = """
CREATE TABLE accounts (
id INTEGER PRIMARY KEY,
owner TEXT NOT NULL UNIQUE,
balance INTEGER NOT NULL CHECK (balance >= 0)
)
"""
def open_account(conn, owner, opening_balance=0):
return conn.execute(
"INSERT INTO accounts (owner, balance) VALUES (?, ?)", (owner, opening_balance)
).lastrowid
def total_balance(conn):
return conn.execute("SELECT COALESCE(SUM(balance), 0) FROM accounts").fetchone()[0]class SharedDatabase(unittest.TestCase):
@classmethod
def setUpClass(cls):
cls.conn = sqlite3.connect(":memory:")
cls.conn.execute(SCHEMA)
def test_account_opening_stores_balance(self):
open_account(self.conn, "ana", 100)
self.assertEqual(total_balance(self.conn), 100)
def test_empty_bank_has_zero_total(self):
self.assertEqual(total_balance(self.conn), 0)
def test_owner_names_are_unique(self):
open_account(self.conn, "ana", 0)
with self.assertRaises(sqlite3.IntegrityError):
open_account(self.conn, "ana", 0)Run the class, then run one of the failing tests on its own:
$ python3 -m unittest -v test_shared
test_account_opening_stores_balance ... ok
test_empty_bank_has_zero_total ... FAIL
test_owner_names_are_unique ... ERROR
ERROR: test_owner_names_are_unique
sqlite3.IntegrityError: UNIQUE constraint failed: accounts.owner
FAIL: test_empty_bank_has_zero_total
AssertionError: 100 != 0
Ran 3 tests in 0.003s
FAILED (failures=1, errors=1)
$ python3 -m unittest -v test_shared.SharedDatabase.test_empty_bank_has_zero_total
Ran 1 test in 0.000s
OKEach test is correct on its own. unittest runs methods in alphabetical order, so the account-opening test goes first and leaves ana with a balance of 100 in the shared database. The empty-bank test then sees 100. The uniqueness test is the nastier case: it errors on its first insert, outside the assertRaises block, because ana already exists. Reading that traceback, you might conclude the uniqueness constraint is broken, when the constraint is the only thing working correctly.
Alphabetical order also means the bug is stable here. Under a runner that shuffles test order (pytest with pytest-randomly, for instance) or runs tests in parallel, the same problem shows up as intermittent failures, which are far more expensive to diagnose.
A fresh fixture for every test
The straightforward fix is to give each test its own database. An in-memory SQLite database takes well under a millisecond to create, so there’s no performance argument for sharing it:
class FreshDatabasePerTest(unittest.TestCase):
def setUp(self):
self.conn = sqlite3.connect(":memory:")
self.addCleanup(self.conn.close)
self.conn.execute(SCHEMA)With the same three test methods, all pass in any order. Each test owns its data and can modify it freely.
Note the use of addCleanup instead of a tearDown method. Cleanups registered with addCleanup run in reverse order of registration, and they run even if setUp fails partway through. If SCHEMA had a syntax error, tearDown would never be called (unittest skips it when setUp raises), but the connection would still be closed. The habit worth building is: create a resource, register its cleanup on the next line. Then cleanup can’t be forgotten, and it can’t run for a resource that was never created.
When building the fixture is expensive: roll back instead
Not every fixture is cheap. A PostgreSQL schema with 80 tables and seed data can take seconds to build, and building it for each of 2,000 tests isn’t an option. The common pattern is to create the schema once, then run each test inside a transaction that’s rolled back at the end:
class RollbackPerTest(unittest.TestCase):
@classmethod
def setUpClass(cls):
fd, cls.path = tempfile.mkstemp(suffix=".db")
os.close(fd)
with sqlite3.connect(cls.path) as conn:
conn.execute(SCHEMA)
@classmethod
def tearDownClass(cls):
os.remove(cls.path)
def setUp(self):
self.conn = sqlite3.connect(self.path)
self.addCleanup(self.conn.close)
self.conn.execute("BEGIN")
self.addCleanup(self.conn.rollback)Because cleanups run in reverse, the rollback happens before the connection closes. Now add a function that does its own transaction handling, like most real data-access code:
def transfer(conn, src, dst, amount):
with conn: # commits on success, rolls back on error
conn.execute("UPDATE accounts SET balance = balance - ? WHERE id = ?", (amount, src))
conn.execute("UPDATE accounts SET balance = balance + ? WHERE id = ?", (amount, dst))A test that opens two accounts and transfers between them, followed by the empty-bank test:
test_a_transfer_moves_money ... ok
test_b_bank_starts_empty ... FAIL
AssertionError: 100 != 0with conn: called commit(), which committed the test’s own transaction, accounts included. The rollback in cleanup then had nothing to undo. This is the main weakness of rollback-per-test: it only isolates tests from code that never commits. Frameworks that use the pattern, such as Django’s TestCase and Rails’ transactional tests, work around it by nesting the application’s own transaction blocks inside the test’s transaction (as savepoints, or folded into the outer one), so they never really commit.
You can do the same in your own code. A savepoint nests inside whatever transaction the caller has open, and commits on its own when there isn’t one:
def transfer(conn, src, dst, amount):
conn.execute("SAVEPOINT transfer") # nests inside a caller's transaction
try:
conn.execute("UPDATE accounts SET balance = balance - ? WHERE id = ?", (amount, src))
conn.execute("UPDATE accounts SET balance = balance + ? WHERE id = ?", (amount, dst))
except Exception:
conn.execute("ROLLBACK TO transfer")
raise
finally:
conn.execute("RELEASE transfer")With this version both tests pass. Outside a test, the change still commits: a second connection sees [('ana', 70), ('ben', 30)] after a transfer of 30, and an overdraft raises CHECK constraint failed: balance >= 0 and leaves both balances untouched.
When you can’t change the code (or it runs on another connection, like a background worker), rollback isolation doesn’t apply. The alternatives are deleting or truncating the touched tables in cleanup, cloning a fresh database from a prepared template for each test or each worker (PostgreSQL’s CREATE DATABASE test_7 TEMPLATE app_template does this quickly), or giving every test unique data so leftovers can’t collide.
State that leaks without a database
Databases get the attention, but plenty of other state outlives a test:
- Environment variables. A test that sets
os.environ["FEATURE_X"] = "on"changes every later test in the process.unittest.mock.patch.dict(os.environ, {"FEATURE_X": "on"})restores the original when the test ends. - Module-level globals and singletons: configuration objects, caches, connection pools, registries. Reset them in cleanup or, better, inject them so each test can pass its own.
- Files. Write into
tempfile.TemporaryDirectory()with its cleanup registered, never into the repository or a fixed path like/tmp/output.jsonthat parallel runs will fight over. - Time and randomness. A frozen clock or a seeded random generator that isn’t restored affects everything after it.
- External services: a shared test tenant in a SaaS API, a queue on a shared broker. Use unique names per test run so concurrent pipelines don’t consume each other’s messages.
A good check during review: for each thing the test writes, find the line that undoes it. If you can’t, that’s where the next intermittent failure will come from.
Interview practice
Your integration suite passes reliably on developers’ laptops, but on CI, where it now runs with four parallel workers against one shared PostgreSQL database, about one run in five fails, always in different tests. Each test uses rollback-per-test. How would you investigate and fix it?
Answer
Rollback-per-test isolates a test from later tests on the same connection, but four workers run concurrently, so what one transaction does can still affect another while both are open. Unique constraints are a typical example: two workers inserting a user with the same email both try to take the same key, and one blocks or fails. Tests that count rows or read “the latest order” can also see committed data from code paths that commit, as with the with conn: example above. Different tests failing on each run is what you’d expect from timing-dependent interference like this.
To confirm, I’d run the suite with one worker on CI. If it’s stable, the problem is cross-worker sharing rather than the tests themselves. Then I’d grep the failures for the data they touch (fixed emails, fixed IDs, global counts). The fix I’d prefer is one database per worker, created from a template at the start of the run, so workers can’t see each other at all. On top of that, I’d generate unique values for anything with a uniqueness constraint (for example, by adding the worker ID and a counter to emails), and audit for code that commits inside tests, switching it to savepoints or giving those tests explicit cleanup.