Code Coverage and Its Limits

Coverage is the easiest testing metric to collect and the easiest to misread. A coverage tool records which lines, branches and functions executed while the test suite ran. That’s all. It doesn’t know whether a test asserted anything, whether the assertions were right, or whether the inputs exercised the cases that matter. An interviewer who asks “what coverage do you aim for?” usually wants to hear that you understand this gap, and that you can still use the number for something.

A function at 100% that still has a bug

Here’s a shipping calculator. The requirement, as written in the ticket: US orders of $50 or more ship free; otherwise US shipping is $5.99 and international is $14.99; express adds $10.

export function shippingCents({ subtotalCents, country, express = false }) {
  let cost = country === 'US' ? 599 : 1499;
  if (subtotalCents > 5000 && country === 'US') cost = 0;
  if (express) cost += 1000;
  return cost;
}

There’s a bug: > should be >=, so an order of exactly $50.00 pays shipping. Now two tests, in the style you find in real codebases under coverage pressure:

import { test } from 'node:test';
import assert from 'node:assert/strict';
import { shippingCents } from './shipping.js';

test('US order over the threshold ships free', () => {
  assert.equal(shippingCents({ subtotalCents: 7500, country: 'US' }), 0);
});

test('international express order', () => {
  shippingCents({ subtotalCents: 2000, country: 'DE', express: true });
});

Node 22’s built-in coverage (node --test --experimental-test-coverage) reports:

✔ US order over the threshold ships free (1.299875ms)
✔ international express order (0.213958ms)
ℹ file             | line % | branch % | funcs % | uncovered lines
ℹ shipping.js      | 100.00 |   100.00 |  100.00 |

Every line, every branch and every function: 100%. The second test has no assertion at all. It runs the express and international paths, so they count as covered, but it would pass if the function returned -1 or NaN. And the boundary bug is invisible, because no test uses 5000.

This is the core limit. Coverage answers “did this code run?” A test suite’s value comes from “would a test fail if this code were wrong?” The two questions have very different answers here.

Line coverage versus branch coverage

Drop the second test and keep only the first, and the report changes in an instructive way:

ℹ file        | line % | branch % | funcs % | uncovered lines
ℹ shipping.js | 100.00 |    50.00 |  100.00 |

Every line still ran, because each if and its body sit on one line, and the US test passes through all four lines. But only half the branches were taken: the ternary never chose 1499, and if (express) was never true. Line coverage counts lines that executed at least once; branch coverage counts each possible direction of each decision. Formatting can move line coverage around (put cost += 1000 on its own line and it shows as uncovered), whereas branch coverage reflects the structure of the code.

So if you only look at one number, look at branch coverage. Even that has gaps. V8, which Node uses, measures branches like if/else and the two sides of a ternary, but a condition such as subtotalCents > 5000 && country === 'US' can be fully “covered” without ever testing the case where the subtotal qualifies and the country doesn’t. And no coverage metric will ever ask for the value 5000, because the boundary is in the requirement, not in the code’s structure.

Tests that would catch it

What does a suite that actually protects this function look like? It’s built from the requirement, with inputs on both sides of every boundary and an assertion on every result:

import { test } from 'node:test';
import assert from 'node:assert/strict';
import { shippingCents } from './shipping.js';

test('US orders of exactly $50.00 ship free', () => {
  assert.equal(shippingCents({ subtotalCents: 5000, country: 'US' }), 0);
});

test('US orders one cent under $50.00 pay standard shipping', () => {
  assert.equal(shippingCents({ subtotalCents: 4999, country: 'US' }), 599);
});

test('international express adds the express fee to the international rate', () => {
  assert.equal(shippingCents({ subtotalCents: 2000, country: 'DE', express: true }), 2499);
});

test('free shipping does not apply outside the US', () => {
  assert.equal(shippingCents({ subtotalCents: 9000, country: 'DE' }), 1499);
});

✖ US orders of exactly $50.00 ship free (0.724666ms)
✔ US orders one cent under $50.00 pay standard shipping (0.075083ms)
✔ international express adds the express fee to the international rate (0.052708ms)
✔ free shipping does not apply outside the US (0.376625ms)

  AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:

  599 !== 0

The coverage number for this suite is the same 100% as the useless one. The difference is entirely in what the tests check, and you can only see it by reading them. The fourth test is also worth noting: it covers the && country === 'US' part of the condition, which branch coverage never asked for.

If you want a tool that measures assertion quality, that’s mutation testing: it changes the code (> to >=, 599 to 600, removes the express line) and checks that some test fails for each change. The assertion-free test above would let every mutant survive. It’s slower than coverage, but it measures what people mistakenly think coverage measures.

What the number is good for

None of this makes coverage useless. It’s a reliable signal in one direction: low coverage means untested code. If a payment module sits at 40% branch coverage, there are decisions in it that no test exercises, and the uncovered-lines column tells you exactly where. That’s the useful way to read a report: not as a score, but as a list of places to look.

Some practical guidance that holds up:

• Review the uncovered lines in the diff, not the total. A pull request that adds a new catch block with no test is worth a comment; a project total that moved from 83.1% to 82.9% usually isn’t.
• Use a threshold as a floor, not a goal. Node can enforce one. With only the first test and --test-coverage-branches=80, the run exits with code 1 and prints Error: 50.00% branch coverage does not meet threshold of 80%. That’s a good guard against a suite slowly rotting. Set it a little under the current value so it blocks regressions without forcing filler tests.
• Don’t set a high mandatory target. When a team must hit 90% to merge, the cheapest way to get there is the second test at the top of this page: call the code, assert nothing. Goodhart’s law applies: once the number is the target, it stops measuring test quality.
• Exclude what you genuinely don’t test, such as generated code or a thin main.js entry point, rather than writing ceremonial tests for it. An honest 80% is more useful than a padded 95%.
• Expect diminishing returns. The last few percent are often defensive branches for “impossible” states. Sometimes they deserve a test; often they deserve a second look at whether the branch should exist.

Martin Fowler’s short piece on test coverage makes the same point: coverage is useful for finding untested code, and a poor target for how good your tests are.

Practice question

Your team’s coverage is 92%, enforced at 90% in CI. A production incident traces back to a discount rule that applied twice when two coupons were combined. The code has a test file with 100% coverage. Your manager asks how this got past a 92% suite and whether the threshold should go to 95%. What do you say?

Model answer

Raising the threshold wouldn’t have helped, because the code involved was already fully covered. Coverage only says the lines ran; it doesn’t say any test combined two coupons or checked the resulting total. I’d look at the discount tests and expect to find single-coupon cases, or tests that call the function without asserting the final amount.

My proposal: first, add a failing test that reproduces the incident (two coupons, assert the exact total) and fix the bug against it. Second, review the discount tests against the business rules, listing the combinations and boundaries the rules mention (stacking, minimum spend, expiry) and making sure each has a test with a precise assertion. Third, to measure assertion strength on high-risk modules like pricing, run mutation testing there, perhaps weekly rather than on every commit. Finally, keep the 90% floor, since it still prevents untested code from piling up, but stop presenting the percentage as evidence of quality in reviews or reports. The question to ask in review is “which test fails if this line is wrong?”

Keep reading

• Testing interview questions
• Testing MCQs
• Mutation testing
• Boundary value analysis
• Martin Fowler: Test Coverage