If AI cannot explain exam grades it is not ready for enforcement

In January, England’s qualifications regulator published a working paper on AI in marking that went further than most observers expected. 

More than simply urging caution, Ofqual said using AI as the sole or primary marker of high-stakes exams does not comply with its regulations. 

It explained why in terms that should concern regulators well beyond education.

The regulator’s chief, Sir Ian Bauckham, told MPs that current AI systems still make mistakes, are difficult to challenge, and operate inside what he called “a complex series of algorithms inside an impenetrable black box.”

Though he noted AI’s genuine potential for quality assurance and marker training, Bauckham concluded it’s nowhere near ready to take over high-stakes marking. 

The paper applies the UK government’s five AI principles – safety, transparency, fairness, accountability, and contestability – and tests them against the specific demands of assessment validity. It concludes that no current AI system meets them all and notes AI’s reasoning is often “unreliable, unavailable, or difficult to interpret” and that existing models lack “true semantic understanding” of student responses.

Conceptually, it has more in common with a supervisory finding than with a white paper: a structured assessment that says, in effect, this does not yet pass.

The enforcement test

Consider the Canadian Securities Administrators (CSA). Between June and November 2025, the CSA used machine learning to identify and deactivate 6,918 fake URLs across 3,961 fraudulent investment websites.

The system – procured by the Ontario Securities Commission (OSC) on behalf of the CSA and the Canadian Investment Regulatory Organization (CIRO) – scans millions of reports daily to detect fraud patterns in near real time.

In Australia, the Australian Securities and Investments Commission (ASIC) has been trialling Transparently, a Singapore-based AI fraud detection tool that flagged suspicious accounting activity among 32 large ASX-listed companies. ASIC has separately been removing roughly 130 scam websites per week.

The results are hard to dispute, yes. 

But basic information about the models – who built them, how they were trained, how often they err – is considerably harder to pin down.

The CSA’s own account of the programme offers no detail on model accuracy, false positive rates, or appeal mechanisms for entities flagged by the system. ASIC declined to comment specifically on its Transparently trial.

Beyond education

It’s a pattern not confined to one sector. 

The Financial Conduct Authority (FCA) now uses predictive AI in its Supervision Hub and launched an AI Live Testing programme in December 2025. The programme runs for approximately 12 months, with each cohort testing over roughly six months. The FCA will publish a public evaluation report at the end.

In Singapore, the Monetary Authority of Singapore (MAS) proposed mandatory AI risk management guidelines in November 2025, with consultation closing in January 2026. The proposed guidelines require board-level accountability, AI inventories, and explainability standards proportionate to impact.

The FCA’s approach is principles-based and built around transparency: firms test AI in a structured environment, and the regulator publishes what it learns. MAS goes further, requiring financial institutions to demonstrate that higher-impact AI systems meet explicit explainability and fairness thresholds. Both models assume the regulator must be capable of assessing what it oversees.

The capability question

Behind these examples lies a big capability gap. 

Many regulators now deploy or oversee AI systems without the internal expertise to evaluate them. Our recent analysis of Canada’s AI regulatory readiness found that while the CSA built enforcement technology that works, most Canadian regulators lack the skills, frameworks, and institutional capacity to govern AI systems they are expected to police. 

In Australia, the Australian Public Service’s (APS) AI Plan mandates capability training across the public service and sets a December 2026 assessment milestone – an implicit acknowledgement that government agencies are not yet where they need to be.

The Alan Turing Institute’s AI regulatory capability framework asks a question most governance frameworks avoid: are regulators themselves institutionally equipped to govern AI effectively? The answer, across most jurisdictions, is not yet.

“AI can inform decisions, but shouldn’t make them. Individuals making decisions are still accountable.”

– Nic Granger, chief financial and information officer,
North Sea Transition Authority

Too often, what is missing is not the willingness to use AI but the capacity to explain, challenge, and account for its outputs. As Nic Granger, chief financial and information officer at the UK’s North Sea Transition Authority, plainly put it in a recent conversation with TMR: “AI can inform decisions, but shouldn’t make them. Individuals making decisions are still accountable.”

The Ofqual standard

What stands out in Ofqual’s position is less the identification of problems with AI and more the way the regulator used a clear, public framework to test the technology. 

Most regulators have not done this. 

Many are adopting AI tools while the governance architecture around those tools remains incomplete.

The European Data Protection Supervisor has warned that without well-structured governance, “human oversight” risks becoming a procedural formality – an operator who rubber-stamps AI outputs rather than scrutinising them. 

The EU AI Act’s Article 14 now requires that high-risk systems be designed for effective human oversight, explicitly addressing the risk of automation bias. But mandating oversight and achieving it are two very different things.

Ofqual’s working paper is about exams. Its logic applies everywhere a regulator uses or oversees a system whose reasoning cannot be fully explained, independently tested, or meaningfully challenged. That describes a growing share of the regulatory toolkit.

The question for other regulators is straightforward: could you do what Ofqual did? Could you publish a principled assessment of the AI systems you deploy – and defend the result?