Every change carries some risk of breaking something that used to work. Regression testing is how you manage that risk: tests that pin down existing behavior, and new tests added whenever a bug is fixed so the same bug can’t quietly return. On a small project you just run everything. On a large one, “everything” takes 40 minutes or four hours, and the question becomes which tests to run for this change, and what you’re giving up by not running the rest.
This article walks through a single bug fix from start to finish: writing the regression tests, looking for the risks next to the bug, selecting tests from the diff, and the moment that selection gets it wrong.
Start with a test that fails for the original bug
The bug report: on the product listing sorted by price, pre-order items (which have no price yet, stored as null) appear first, as if they were free. A second report a week later says that when several products share a price, some appear on both page 1 and page 2 while others never appear at all.
The current code:
export function sortByPrice(products) {
return [...products].sort((a, b) => a.priceCents - b.priceCents);
}In JavaScript, null - 1200 is -1200, because null converts to 0 in arithmetic. So unpriced products sort as the cheapest. The existing test, cheapest first, uses two products with prices and passes happily.
Before touching the code, write tests that describe the correct behavior and run them against the old code:
test('products without a price go last, not first', () => {
const names = sortByPrice([
{ name: 'Lamp', priceCents: 3999 },
{ name: 'Pre-order chair', priceCents: null },
{ name: 'Mug', priceCents: 1200 },
]).map((p) => p.name);
assert.deepEqual(names, ['Mug', 'Lamp', 'Pre-order chair']);
});
test('equal prices are ordered by name', () => {
const names = sortByPrice([
{ name: 'Tea towel', priceCents: 900 },
{ name: 'Coaster', priceCents: 900 },
]).map((p) => p.name);
assert.deepEqual(names, ['Coaster', 'Tea towel']);
});✖ products without a price go last, not first (1.229917ms)
[
+ 'Pre-order chair',
'Mug',
'Lamp',
- 'Pre-order chair'
]
✖ equal prices are ordered by name (0.343291ms)
[
+ 'Tea towel',
'Coaster',
- 'Tea towel'
]Seeing them fail matters. A regression test that passes on the buggy code isn’t a regression test; it’s a test of something else. The failure output also confirms you’ve reproduced this bug, unpriced items at the front, rather than a different one.
Look next to the bug, not only at it
The second report, items repeating across pages, isn’t about the comparator directly. It’s about what calls it. The listing paginates the sorted products:
export function listPage(products, { page = 1, size = 2 } = {}) {
const sorted = sortByPrice(products);
return sorted.slice((page - 1) * size, page * size).map((p) => p.name);
}JavaScript’s sort is stable, so products with equal prices keep whatever order they arrived in. Rows from a database query without a complete ORDER BY can arrive in a different order on each request. Page 1 and page 2 are separate requests, so each one sorts a differently ordered input and slices it. A test at the listing level reproduces exactly what users saw:
test('equal prices never repeat or vanish across pages, whatever order rows arrive in', () => {
const rows = [
{ name: 'Coaster', priceCents: 900 },
{ name: 'Tea towel', priceCents: 900 },
{ name: 'Napkin', priceCents: 900 },
];
const page1 = listPage(rows, { page: 1, size: 2 });
const page2 = listPage([...rows].reverse(), { page: 2, size: 2 }); // the database may return another order
assert.deepEqual([...page1, ...page2].sort(), ['Coaster', 'Napkin', 'Tea towel']);
});✖ equal prices never repeat or vanish across pages, whatever order rows arrive in (2.128916ms)
[
'Coaster',
+ 'Coaster',
- 'Napkin',
'Tea towel'
]Coaster shows up twice and Napkin never appears. This is why regression selection starts with the behavior that changed and then asks what depends on it. The fix belongs in one function, but the risk spreads to every caller that pages, exports, caches or compares its output.
The fix puts unpriced products last and breaks ties by name, so the order is the same whatever the input order:
export function sortByPrice(products) {
return [...products].sort((a, b) => {
const aMissing = a.priceCents == null;
const bMissing = b.priceCents == null;
if (aMissing !== bMissing) return aMissing ? 1 : -1; // unpriced products go last
if (!aMissing && a.priceCents !== b.priceCents) return a.priceCents - b.priceCents;
return a.name.localeCompare(b.name); // deterministic order for ties
});
}Selecting tests from what changed
In a big codebase you don’t want to run thousands of tests to check a comparator. The usual approach is to work out which tests can reach the changed code. Here’s a minimal version: take the files changed relative to main, follow import statements, and keep any test file that depends on a changed file, directly or indirectly.
const changed = execSync(`git diff --name-only ${base}`, { encoding: 'utf8' }).split('\n').filter(Boolean);
const importsOf = (file) =>
[...readFileSync(file, 'utf8').matchAll(/from '(\.[^']+)'/g)].map((m) => normalize(join(dirname(file), m[1])));
const dependsOn = (file, target, seen = new Set()) => {
if (file === target) return true;
if (seen.has(file)) return false;
seen.add(file);
return importsOf(file).some((dep) => dependsOn(dep, target, seen));
};
const selected = tests.filter((t) => changed.some((c) => dependsOn(t, c)));On the fix branch:
changed: src/sort.js test/listing.test.js test/sort.test.js
selected: test/listing.test.js test/sort.test.jsThe cart tests don’t import anything sorting-related, so they’re skipped. Running the selection gives tests 5, pass 5, fail 0. Real tools do the same thing more thoroughly: Jest’s --findRelatedTests and --changedSince, Nx and Turborepo’s “affected” commands for monorepos, and Bazel’s dependency graph, which knows every input of every test target.
Where selection goes blind
The repository also has a nightly CSV export, tested by running the export script as a separate process and comparing its output:
test('the nightly CSV export matches the agreed format', () => {
const csv = execFileSync('node', ['src/export.js'], { encoding: 'utf8' });
assert.equal(csv, 'Pre-order chair,\nMug,1200\nLamp,3999\n');
});src/export.js imports sortByPrice. The test file doesn’t import anything from src; it starts a child process. So the selection script never connects it to sort.js. Compare the two runs:
== selected run
ℹ tests 5
ℹ pass 5
ℹ fail 0
== full run
✖ the nightly CSV export matches the agreed format (31.436416ms)
+ 'Mug,1200\nLamp,3999\nPre-order chair,\n'
- 'Pre-order chair,\nMug,1200\nLamp,3999\n'
ℹ tests 7
ℹ pass 6
ℹ fail 1The selected run is green, the full run is red. Notice that the export test’s expected output was written while the bug existed, with the unpriced chair first, so it encodes the bug as the agreed format. Whether the fix should change the export is a real question: a downstream system might parse that file and depend on the old order. A selective run would have merged the change without anyone asking.
Child processes are one blind spot. Others are configuration and data files, dynamic imports with computed paths, SQL migrations, environment variables, shared fixtures, generated code, and anything reached over the network. Changing the global CSS, a feature flag default or a dependency version affects tests that import none of the changed lines.
A practical policy
Selection and full runs aren’t competitors; they belong at different points:
- While coding: the tests related to the files you’re editing, in seconds.
- On every push: affected tests plus a fast smoke suite of critical paths.
- Before merge to main: the full suite, or at least a full run that blocks the release if not the merge.
- Nightly: everything, including slow end-to-end and performance checks, with failures assigned to someone the next morning.
And each escaped bug gets a regression test at the lowest level that can see it, written to fail first, plus a short note on why existing tests and selection missed it. Those notes are how you find out which blind spots matter in your codebase.
Interview practice
Your monorepo’s full test suite takes 55 minutes, so the team wants to run only “affected” tests on pull requests. What would you put in place so that this doesn’t let regressions through?
Answer
I’d accept affected-only runs on pull requests, but treat them as an early signal rather than the last line of defence. First, I’d check how the affected calculation works and list what it can’t see: config and data files, environment variables, child processes, dynamic imports, database migrations and shared test fixtures. Changes to any of those, plus changes to dependency lockfiles or build configuration, should trigger the full suite automatically.
Second, I’d keep the full suite running somewhere that blocks shipping: on the merge queue before code lands on main, or at minimum nightly, with a clear owner for failures. Third, I’d measure it. Every time the full run catches a failure the affected run missed, that’s a blind spot to fix, either by improving the dependency graph or by adding the pattern to the “always run everything” list. If that happens rarely, the selection is working; if it happens weekly, the team is paying for faster PRs with broken mainline builds. I’d also look at speeding up the full suite itself, through parallel sharding and fixing the slowest tests, since a 15-minute full run makes most of this debate unnecessary.