Skip to content
Velloreth
Programming 6 min read

Regression Testing: Making Sure Bugs Stay Fixed

Regression tests guard against old bugs returning. Learn how to write them, what to cover, and how to keep the suite from growing forever.

By

What a regression test is

A regression test reproduces a bug that was previously fixed. It passes while the fix is in place, and fails if the bug returns.

The name comes from the idea of a regression: the code going backwards, losing a behaviour it once had.

// Bug report: "Deleting the last item in a cart throws an error"
// Fixed in commit abc123.

test('deleting the last cart item does not throw', () => {
  const cart = new Cart();
  cart.add({ id: '1', price: 10 });

  expect(() => cart.remove('1')).not.toThrow();
  expect(cart.items).toHaveLength(0);
});

The test encodes the bug report. If someone later changes remove and reintroduces the error, this test fails.

Why every fix needs one

A bug fix without a regression test is unverified and unprotected.

Unverified. You believe the fix works because you tried it once. A test proves it, and proves it continues to work.

Unprotected. Without a test, the next refactor can reintroduce the bug silently. The bug returns, and nobody notices until a user reports it again.

Documented. The test records the bug and its fix. A future developer reading it learns that this case was once broken, which is context they would otherwise lack.

The cost is small — usually a few lines — and the benefit compounds over the life of the code.

Write the test first

The order matters, and it is the same discipline as TDD.

  1. Write a test that reproduces the bug. Run it. It should fail.
  2. Fix the bug. Run the test. It should pass.
  3. Keep the test.

Step 1 is the important one. Watching the test fail proves it actually reproduces the bug. If you write the test after the fix, you cannot be sure it would have caught the problem — it might pass for an unrelated reason.

This is especially important for bugs that are hard to reproduce. If you cannot write a failing test, you do not fully understand the bug, and the fix is a guess.

What to cover

Not every bug needs the same treatment. Prioritise by cost.

High value:

  • Bugs that reached production.
  • Bugs that affected users visibly.
  • Bugs caused by a subtle interaction between components.
  • Bugs that were expensive to diagnose.
  • Bugs in security or payment code.
  • Bugs that have already recurred.

Lower value:

  • Typos in log messages.
  • Cosmetic issues in internal tools.
  • Bugs in code that is about to be removed.

The judgement is: how bad would it be if this returned? If the answer is “a user would notice and we would scramble”, write the test.

Types of regression test

Unit regression tests. The most common. A test that calls the function that had the bug with the input that triggered it.

Integration regression tests. For bugs that only appeared when components met — a serialisation mismatch, a schema change, a configuration error.

End-to-end regression tests. For bugs in a user flow, such as “checkout fails when the cart has a discount code”.

Visual regression tests. Screenshot comparisons that detect unintended UI changes.

Performance regression tests. Assert that an operation stays within a time or memory budget, catching accidental slowdowns.

Keeping the suite manageable

A regression suite grows with every bug fixed. Left unchecked, it becomes slow and redundant.

Prune periodically. Remove tests for behaviour that no longer exists. A test for a removed feature is dead weight.

Merge duplicates. If three tests cover the same code path with slightly different inputs, one may suffice.

Delete tests covered by broader tests. If an integration test covers a flow, the unit tests for the same flow may be redundant.

Watch the runtime. A suite that takes 20 minutes will not be run before every commit. Keep the fast tests fast.

Do not delete a regression test just because it is old. Age is not a reason. Redundancy is.

Regression testing and refactoring

Regression tests are what make refactoring safe. They are the safety net that lets you change structure without changing behaviour.

This is why the tests should assert on behaviour rather than implementation. A regression test that asserts on internal calls will break during a refactor even though the bug is still fixed.

// Brittle: breaks if the implementation changes
test('deleting the last item calls splice', () => {
  const spy = jest.spyOn(Array.prototype, 'splice');
  cart.remove('1');
  expect(spy).toHaveBeenCalled();
});

// Robust: asserts the behaviour
test('deleting the last item leaves an empty cart', () => {
  cart.remove('1');
  expect(cart.items).toHaveLength(0);
});

The second test survives a change from splice to filter. The first does not, and it will be deleted during the refactor, losing the protection.

A worked example

Bug report: “The discount is applied twice when a user applies a code and then updates their cart.”

1. Reproduce it in a test.

test('does not double-apply the discount after a cart update', () => {
  const cart = new Cart();
  cart.add({ id: '1', price: 100 });
  cart.applyDiscount('SAVE10'); // 10% off

  cart.updateQuantity('1', 2); // triggers recalculation

  expect(cart.total).toBe(180); // 200 - 10%
});

Run it. It fails, returning 162 (the discount applied twice). The bug is reproduced.

2. Fix the bug. The recalculation was re-applying the discount to an already-discounted total. The fix is to recalculate from the subtotal.

3. Run the test. It passes.

4. Keep it. The test now guards against the bug returning.

The test is four lines and it encodes a real bug. That is the value.

Practical guidance

  • Write a regression test for every bug fix.
  • Write the test first and watch it fail.
  • Assert on behaviour, not implementation, so the test survives refactoring.
  • Prioritise by cost — production bugs, user-visible bugs, security bugs.
  • Prune the suite periodically to keep it fast.
  • Do not delete old tests for being old — delete them for being redundant.
  • Add a comment linking to the issue when the bug has a ticket.
  • Consider visual and performance regression tests for UI and hot paths.

For the related practices, see unit testing basics and fixing flaky tests. For the broader context, refactoring basics and incident response for solo developers.

Quotable summary

A regression test reproduces a bug that was previously fixed, so it fails if the bug returns. Every bug fix should come with one, because without it the fix is unverified and unprotected, and the next refactor can reintroduce the bug silently. Write the test first and watch it fail, which proves it actually reproduces the problem; a test written after the fix might pass for an unrelated reason. Assert on behaviour rather than implementation, so the test survives refactoring and keeps protecting the fix. Regression suites grow with every bug fixed, so prune tests that duplicate coverage or cover removed behaviour, and prioritise by cost — production bugs, user-visible bugs, and security bugs deserve tests first.

Frequently asked questions

What is a regression test?
A regression test is a test that reproduces a bug which was previously fixed. It passes while the fix is in place and fails if the bug returns, so it protects against the same mistake being reintroduced.
Why write a regression test for every bug fix?
Because it proves the fix addresses the reported problem, and it prevents the bug from returning. A bug that has been fixed once and returns is a bug that was never tested.
Should I write the regression test before or after the fix?
Before. Write the test, watch it fail, then fix the bug and watch it pass. If you write it after, you cannot be sure it would have caught the bug.
How do I keep a regression suite from growing forever?
Prune periodically. Remove tests for behaviour that no longer exists, merge tests that duplicate coverage, and delete tests that are covered by a broader test. A suite that takes too long to run stops being run.
What is the difference between regression testing and retesting?
Retesting verifies that a specific fix works. Regression testing verifies that the fix did not break anything else, and that previously fixed bugs stay fixed. Retesting is narrower.
What is visual regression testing?
It compares screenshots of the UI before and after a change to detect unintended visual differences. Tools like Percy and Playwright's screenshot assertions automate it.