Most firms can evidence their security testing. Far fewer can evidence that their tests cover the business functions the EU Digital Operational Resilience Act (DORA) exists to protect.
See the Coverage & Evidence AuditTwo articles carry the weight, and they are routinely conflated.
Financial entities must test every ICT system supporting critical or important functions at least yearly. Article 8 assumes you already know which functions those are.
Twelve methods are named, and penetration testing is only one. Three of them, gap analyses, scenario-based and end-to-end testing, ask whether the software does what the business needs; security is tested separately.
DORA has applied since 17 January 2025. In 2025, EU firms reported 3,383 major ICT incidents. System failure was the largest category at 51%; cyber, 10%. Supervisors have moved from asking firms to adopt DORA to checking that they follow it.
Supervision has shifted from reviewing plans to inspecting evidence that controls operate.
The ECB's supervisory priorities for 2026 to 2028 include targeted reviews of ICT change management: whether changes to critical systems were tested, and evidenced as tested, before they reached production. That is a question about your test suite.
DORA Article 24(4) requires that tests are undertaken by independent parties, whether internal or external. Independent assessment is written into the regime.
A pipeline dashboard falls short of that. A pass rate falls short of that.
A passing test suite shows that some tests ran and passed. Four patterns account for the gap between that and coverage.
A check that a balance is visible passes whether the number is right, stale or zero. It confirms the page rendered; whether the result is correct needs a separate check.
A portfolio valuation showing wrong figures fails a critical function with no error, no alert and no failed test. It is found when a client notices.
Tests written, then skipped or disabled, still reported as coverage. Six months of a skipped payments spec is a gap wearing a passing suite's costume.
A payment tested for success, with nothing asserted on timeout, rejection or reversal, which is exactly where incidents live.
For the difference between checking that tests passed and proving what they cover, see quality assurance vs quality engineering.
Which of your critical business functions are protected by automated tests, and where are you blind? Five steps, in order.
Start with the business, then the suite. List the journeys the business cannot afford to fail, then tier them by consequence of failure.
Classify each test: behavioural, functional, structural or unreliable. Only the first two count. Then give each journey a coverage state.
Consequence multiplied by detection delay. A silent Tier 2 gap can outrank a monitored Tier 1 flow. Out comes a ranked gap register.
The full method is in The Certance Standard, 6.1 and Chapter 13: read the Standard
An independent gap analysis: does your test suite cover the business-critical journeys, and is that coverage evidenced? The gap analysis named in Article 25(1).
Out of scope: penetration testing and threat-led penetration testing under Articles 26 and 27, a separate security discipline with a different provider. This is a risk view; the compliance conclusion stays with your compliance function.
Certance holds no referral or revenue-share arrangement with any vendor named in the report.
DORA does not mandate automation. Article 24(6) requires that appropriate tests are conducted at least yearly on all ICT systems and applications supporting critical or important functions, and Article 25(1) names the methodologies available. In practice, for systems that release frequently, evidencing that testing happened before each release is difficult without automation.
A percentage on its own answers no supervisory question, because it has no denominator a regulator recognises. Supervisors ask which critical or important functions were tested, with what scope, what the results were, and what was remediated. 'Three of six Tier 1 journeys have behavioural coverage' is a more useful and more honest statement than '77% coverage'.
Where supervisors focus and where incidents start are two different questions. In 2025, the first year DORA applied, cyber accounted for 10% of major incidents; system failure was the largest category at 51%, and almost a third originated with a third party. Two of the ECB's 2026 supervisory activities, third-party risk and ICT change management, sit in that larger, non-cyber majority. Testing the business functions behind your critical services addresses the half of the distribution that security testing leaves open.
Penetration testing is one of twelve methodologies named in Article 25(1). The article also names gap analyses, scenario-based tests and end-to-end testing. A programme of security testing alone addresses the 10% of major incidents that were cyber-related and leaves the 51% that were system failures untested.
Article 24 sets the general requirements for the testing programme, including the yearly frequency in 24(6) and the independent-tester requirement in 24(4). Article 25 governs the testing of ICT tools and systems and names the methodologies. Articles 26 and 27 cover threat-led penetration testing, which applies to significant entities on a roughly three-yearly cycle and is a separate discipline.
DORA Article 24(4) requires that tests are undertaken by independent parties, whether internal or external. A self-assessment answers the question differently from an external one, and a risk function knows this.
Certance runs a fixed three-day Coverage & Evidence Audit. It needs no access to client systems: it works from test artefacts a client's own engineer exports and reviews, with no account provisioning, no production data and no customer data.
The Coverage & Evidence Audit is a fixed three-day, founder-led review that maps your test suite to the business journeys it is supposed to protect, and produces a gap report an engineering leader can take to a board or a risk committee.
How the audit works