Regression automation: when it pays off and when it burns the budget

"Let's automate everything" is the most expensive phrase in testing. Automated tests pay off only on stable flows that run often and change rarely. Everything else is cheaper to check by hand — and more honest to tell the client before work starts.

Three conditions under which automation pays off

• Stability: the flow has not changed for several releases and its expected result is fixed. Automating what is reworked every week means writing tests that break before they start being useful.
• Frequency: the suite runs before every release or more often. A quarterly regression takes years to pay for automation.
• Determinism: the flow gives the same result with the same data. Payment gateways with an unstable test mode and screens with external ads automate poorly.

What to automate first

1. API and contracts: the most stable, fastest and cheapest-to-maintain tests — response schemas, error codes, mandatory fields.
2. Critical user paths: sign-up, sign-in, payment, checkout — one scenario per path, no variations.
3. Data checks: reconciliations, field validation, integrity after migrations — where manual checking is long and tedious.
4. Only then — variations and interface details, and only if the first three levels are stable.

What to keep manual

Exploration of new features, LQA, visual correctness on different devices, flows with external services in an unstable test mode, one-off checks before a promotion. Manual regression in these areas is no worse than automated and several times cheaper. Rule of thumb: if a flow will run fewer than ten times a year, automation will not pay off.

The hidden cost — maintenance

Writing an automated test is a quarter of the cost. The rest is maintenance: every interface change breaks some tests, and someone has to fix them. If nobody on the team owns that, the suite degrades within months: tests fail, get disabled, then stop being run. That is why we start automation with the question "who will fix the tests in six months", not with choosing a framework.

How we set it up

Manual regression comes first: the suite is assembled, stabilised over several releases, and only then are automation candidates selected by the three conditions above. Automation is a separate contract: it is development, not testing, and for clients in Russia it is taxed differently. The result is tests in your repository, scheduled or CI runs, a report to the team chat and documentation your developer fixes them from.