Code coverage: the metric that lies hardest right when you trust it most

A team celebrates hitting 85% coverage. On the dashboard the bar turns green and everyone breathes easier. Two weeks later a trivial bug reaches production: a subtraction that was supposed to validate a balance returned the wrong sign. The strange part is not that the bug existed. The strange part is that the line with the error was covered. The tests ran it. And even so, nobody verified it.

The short answer to why this happens: coverage measures which code was executed during the tests, not which behavior was checked. Those are different things, and confusing them is the most expensive and most common mistake in quality management. Coverage is a speedometer that tells you how fast you drove through each stretch of road, but not whether you were watching the road at all.

01 · what it measuresRunning a line is not the same as testing it

Code coverage is a measure of how much of your source code was traversed while a test suite ran. An instrument watches the execution and records which lines, which branches and which conditions fired. In the end it divides what was traversed by the total and hands you a percentage. That is all it does: count visits.

The trouble shows up in the leap the mind makes on its own. Seeing "85% coverage" and reading "85% tested" is a mirage so natural that almost nobody notices it. A test can call a function, make every one of its lines run, and check nothing about the result. If there is no assertion saying "the balance should be 100", the line counts as covered even when it returns garbage.

Coverage counts lines visited. Quality depends on behaviors verified. They are not the same.

There are several types, and sorting them matters because they do not demand the same thing. Line coverage asks whether each line ran. Branch coverage asks whether each fork (each if) was taken down both paths, the true one and the false one. Condition coverage goes deeper still: inside an if with several conditions joined by and or or, it asks whether each individual condition was ever true and ever false. A 100% on lines can coexist with a 50% on branches: it is enough to have never tested the else path.

02 · the trapWhen the number climbs and quality does not budge

The biggest trap is not technical, it is human, and it has a name. The moment a percentage becomes a target, it stops working as a measure. This is Goodhart's law: when a measure becomes a goal, it ceases to be a good measure [1]. Applied to coverage, the result is predictable. If the organization demands 90%, the team will reach 90%. But not necessarily by testing better: sometimes by writing tests that run code without checking its result, purely to paint the bar green.

These tests wear an innocent face and are worse than having none. They give a false sense of safety. The code is "covered", so nobody suspects it, and when it fails, it fails in the zone everyone believed was safe. A suite that raises coverage without raising the ability to catch defects is pure theater with a maintenance cost.

Here it helps to flip the question. Coverage is excellent for what it does not tell you, and unreliable for what it does. A 0% of coverage in a module is a hard, trustworthy signal: that code is not tested, period. The absence informs. The presence, by contrast, is ambiguous: a 90% does not guarantee that those executions checked anything useful. That is why coverage is worth a lot as a gap detector and little as a quality certificate. Read it backwards from how almost everyone reads it.

03 · using it rightA diagnostic tool, never a management target

So, do you throw it in the trash? No. A tool that is used badly is not a bad tool. Coverage, in sensible hands, is valuable diagnosis. The key is changing the question you ask of it.

The wrong question is "what percentage do we have?". The right one is "which important parts are we not touching?". The coverage report, read by its red lines and not by the global number, is a map of your blind spots. It shows you the error handling nobody exercised, the configuration branch that never runs in tests, the forgotten edge case. That use is gold. The other, chasing a number, is Goodhart's law waiting to collect.

Do not ask how much coverage you have. Ask which important code was left untouched.

It is worth anchoring this in the vocabulary of the craft. ISO/IEC/IEEE 29119, the international software testing standard, treats coverage as a measure of test completeness against a criterion, not as an end in itself [2]. And the ISTQB glossary itself defines coverage as the degree, expressed as a percentage, to which a specified element has been exercised by a test suite [3]. Notice the word: exercised, not verified. The standard is honest about what it measures. It is we who ask it to mean more than it can.

04 · what to measure insteadVerified behaviors, not visited lines

If coverage is not the goal, what is? The real goal was always the same: catch defects before the user does. And for that it helps to watch signals that actually correlate with that ability. One is the quality of the assertions: not how many lines you run, but how many checkable claims you make about the result. Another, more demanding, is mutation testing: the tool deliberately injects small errors into your code (changing a > into a >=, a + into a -) and verifies whether your tests catch them. If the code mutates and your tests stay green, those tests were checking nothing, no matter how much coverage they report. Mutation measures what coverage only pretends to.

None of this means scorning coverage. It means putting it in its place. It is an honest diagnostic instrument about one narrow, concrete thing: what ran. Asking it to certify quality is like asking the speedometer to confirm you reached the right destination. It marks speed, not direction.

Next time you see a green bar at 85%, resist the automatic relief. Ask the uncomfortable question: of what actually ran, how much was truly checked? There, in that gap between visiting a line and verifying what it does, lives almost everything coverage hides from you right when you trust it most.