QA · Read 8 min

Regression testing: making sure fixing A does not break B

The most innocent change revives a bug you thought was dead. Regression testing is the safety net that warns you when the past breaks again.

A regression test is a test you run again after a change to confirm that whatever already worked still works. That is the whole idea behind the name: a regression is when the software "goes backward", when a feature that used to be correct starts failing again because of a recent modification. The regression test exists to catch that backslide before it reaches the user.

The scene is a classic. You fix bug A, you ship, and three days later someone reports that flow B, which nobody touched, stopped working. It is not bad luck: A and B shared something (a function, a piece of data, an assumption) and your fix moved the ground out from under B. Regression testing is the discipline of never trusting the words "I did not touch that".

01 · what it isRegression: when the past breaks again

The international testing vocabulary standard defines it plainly: a regression test is one run on a previously tested component or system, after a modification, to verify that no defects were introduced and that no earlier defects were exposed in the unmodified parts [1]. Notice the nuance: you are not out to test the new thing, you are out to defend the old one.

A regression defect almost never shows up where you worked. It surfaces off to the side, in code you never even opened. The reason is that software is full of invisible dependencies: a function that five others call, a global variable, a date format, an ordering of results that someone assumed would stay constant. You change one and the other five find out too late, in production.

Regression does not test what you did. It protects what already worked from what you just did.

That is why regression differs from other tests in its intent. A new test asks "does this thing I just built do what it should?". A regression test asks "does this thing that already existed still do what it used to?". The first one validates; the second one watches. And because the product keeps growing, the list of things to watch only gets longer.

Figure 1 · a change in A breaks B
You fix A intentional change B breaks nobody touched it hidden dependency shared function · common data · implicit assumption
The regression defect travels along the dependency you do not see. Fixing A should not touch B, but they share the same floor. The regression test is the alarm that goes off when that floor shifts.

02 · why it growsThe suite fattens up at the pace of the product

Here is the trap that catches every young team off guard. Every new feature you ship becomes, from that moment on, something you have to keep protecting in every release. The regression suite is not a fixed list: it is the sum of everything that ever worked and that you do not want to lose. It grows with the product, version after version.

At first you can do it by hand. Ten screens, twenty flows, one tester walks through them in a morning. But the product does not stay at ten screens. Two years in you have hundreds of flows, and "walking through all of them by hand before every release" becomes impossible: it would take days, nobody does it in full, and regression defects start slipping in precisely through the corners that got skipped for lack of time.

The cost of finding it late

A defect that slips through regression does not cost the same depending on when you catch it. Fixing a fault caught in production costs far more than fixing the same fault caught during development: you have to diagnose it on a live system, reproduce it, patch it under pressure, and sometimes apologize to the customer. The cheap regression run today prevents the expensive fire tomorrow.

This growth explains why regression is the number one candidate for automation. A regression test is, by definition, something you already know should pass and that you will run many more times without changes. It is repetitive, predictable, and dull: exactly the kind of work a machine does better than a person. Automating the new and exploratory is hard; automating what is already stable and just needs to be checked over and over is where automation truly pays off.

Manual regression scales with the team. Automated regression scales with the product. Only one of the two wins that race.

03 · what to rerunYou cannot run everything: you have to prioritize

Even if you automate, there comes a point where the full suite takes too long to run on every small change. That is when the central question of mature regression appears: out of everything that could be rerun, what do I rerun now? Running everything is always the safest choice, but also the slowest, and slowness has a cost: if regression takes hours, people stop running it.

The answer rests on two ideas. The first is selection by impact: you prioritize the tests that cover the areas your change may have affected, plus the highest-risk areas (what customers use the most, what moves the most money, what has broken the most times before). Not all tests are worth the same; one that protects payment weighs more than one that protects a help screen.

The second idea is suite hygiene. A regression that grows without pruning fills up with flaky tests (which fail for reasons that are not real bugs) and duplicate tests. Flaky tests are dangerous because they teach the team to ignore red failures, and an alarm that gets ignored protects nothing. Maintaining the suite means deleting the dead, fixing the unstable, and not letting it fatten up out of control.

Figure 2 · prioritizing what to rerun
area risk likelihood the change affects it rerun first high risk + high impact rerun if time allows rerun if time allows leave for the full suite
Not every test runs on every change. The high-risk and high-impact ones go first; the low-risk and low-impact ones can wait for the full nightly run. Prioritizing is admitting that time is finite.

04 · where it livesRegression fits into the CI/CD pipeline

Everything above clicks into place when regression stops being a manual event and becomes part of the continuous integration and delivery pipeline. The idea of continuous integration is that every change a developer pushes automatically triggers the project build and tests, so problems are caught in minutes rather than weeks [2]. Automated regression is the heart of that trigger.

The practical pattern is usually layered. On every change a fast, selective regression runs (the highest-risk and highest-impact tests, in a few minutes). Once a day, or before a release, the full suite runs, slower, checking everything else. That way the developer gets a quick answer and can keep working, and the product gets a complete safety net before it reaches the customer.

When this works, regression stops living in one person's head and becomes a property of the system: every time someone proposes a change, the pipeline answers "everything is still green" or "you broke this", and it does so without anyone having to remember to check. That is the difference between hoping the fix to A does not break B, and knowing it before you ship.

Let the idea stand firm: regression is not a rare or advanced kind of test. It is a stance toward change. Everything that works today is something a change tomorrow, however innocent, can break, and the only reasonable defense is to have a net that reverifies the usual, every time, automatically and by priority. The past breaks again all the time. Regression is what warns you when it does.

Sources

  1. ISTQB (International Software Testing Qualifications Board). Glossary of Testing Terms, entries "regression testing" and "confirmation testing". Based on the vocabulary aligned with ISO/IEC/IEEE 29119. glossary.istqb.org.
  2. Fowler, M. (2006, updated). Continuous Integration. martinfowler.com/articles/continuousIntegration.html. (Foundation for why tests, regression included, run automatically on every change.)
  3. ISO/IEC 25010. Systems and software engineering: Systems and software Quality Requirements and Evaluation (SQuaRE): System and software quality models. Reference model for the quality characteristics that regression helps preserve (functionality, reliability, maintainability).

Learn more about AI

See all Learn AI