Insights · Quality engineering
Quality assurance (QA) checks whether software passed its tests. Quality engineering asks what those tests prove, measures that proof against the risks the business carries, and produces evidence someone outside the team can check. When artificial intelligence (AI) writes most of the code, the second question decides whether a release deserves trust.
| Quality assurance | Quality engineering | |
|---|---|---|
| Core question | Did the tests pass? | What do the tests prove, and could someone else check it? |
| Unit of measure | Test results and a code coverage percentage | Business journeys covered, with gaps ranked by consequence |
| Output | A pass or fail for each build | Evidence a third party can inspect |
| Where it sits | A phase before release | A discipline built into every change |
| With AI-written code | Pass rates can hold steady while what the tests check drifts | Each change shows test integrity and behavioural evidence before it merges |
On 22 April 2026, Alphabet's chief executive, Sundar Pichai, wrote: “Today, 75% of all new code at Google is now AI-generated and approved by engineers, up from 50% last fall.”
The number drew the headlines. The useful part is the phrase “approved by engineers”. It names review as the step Google's scale now rests on. The post leaves open what that approval checks. Quality assurance can confirm that a change was approved and its tests passed; quality engineering asks what the approval and the tests proved.
Quality assurance grew up when writing code was the slow, expensive part of delivery. Its core question, whether the software passed the tests written for it, fits that world.
That question still matters. Its limit is what a pass can tell you. A passing pipeline shows that the current tests passed. It cannot show whether those tests check what the business depends on, or whether they changed in the same commit as the code they test.
An AI agent widens that second gap. Asked to turn a failing build green, an agent can fix the code or quietly adjust the test. Both produce a green pipeline. The full Certance Standard is explicit about the second case: “A test that has been changed, skipped or deleted to get a green build is worse than a failing one, because nobody can see that the protection has gone.”
Quality engineering keeps the tests and changes the questions asked of them. Three questions do most of the work.
Coverage is measured against the business journeys that carry money, data and regulatory exposure. A code coverage percentage counts lines that ran. It cannot show a suite that checks a payment screen renders while nothing checks the amount on it.
The Standard defines quality this way: “Quality is the evidence you can hand to someone else. A colleague, a customer or a supervisor must be able to check it.”
In the Standard's words, “Every AI-assisted change carries its own proof and a named owner.” It sets five gates before a change merges. The first, provenance, applies to every change, with or without AI; the other four apply to every AI-assisted change:
The Standard puts the economics in two sentences: “Production is cheap; verification is the asset. Size your process for the review queue.”
Here is what that means in practice. If a team doubles the code it generates while reviewer time stays the same, one of two things happens: fewer changes get a careful check, or each check gets half the attention. Every change can still be approved, as at Google, and the pass rate looks the same either way. A dashboard built on pass rates will show a healthy team while its verification falls behind.
Quality engineering measures how much of what ships has been shown to do what the business needs, with evidence that outlives the person who reviewed it. Review capacity, the scarce resource in the Standard's sentence, is spent on that question.
In a regulated firm, the question “what does the suite prove?” eventually arrives from outside the team. The Digital Operational Resilience Act (DORA) requires financial entities, other than microenterprises, to ensure “at least yearly, that appropriate tests are conducted on all ICT systems and applications supporting critical or important functions” (Article 24(6); ICT means information and communication technology). Article 25(1) names appropriate tests such as gap analyses, scenario-based tests, end-to-end testing and penetration testing. The testing this article is about, automated functional tests of the business journeys behind those functions, is one part of that evidence; our guide to test evidence under DORA sets out what Articles 24 and 25 ask for and where it fits.
Whether a given firm's testing meets that expectation is for its own compliance function to judge. The engineering task is to make sure the evidence exists, traces to a release and is kept long enough for a later review.
Where an answer is unclear, start with the first question: list the journeys and the tests that cover each one. That list is where any coverage review begins.
A quality engineering consultancy gives an independent read of what a test suite proves, measured against the journeys the business depends on. Certance does this for regulated software teams, starting with financial services, in the three-day Coverage & Evidence Audit. It runs on exported artefacts, with no access to production systems and no client personal data.
See the Coverage & Evidence AuditTest automation is one tool quality engineering uses. Quality engineering decides what the automation must prove, measures it against business risk and turns the results into evidence.
It needs the same tests and stricter evidence about them: who or what changed each test, whether it still checks what it checked before, and what behaviour the change was shown to have.