The confidence ladder

Ask a well-built AI system whether an application meets your disaster recovery requirement and it will tell you: yes, it does. Ask it the same question about a different application and it will tell you the same thing. One of those answers rests on a signed quarterly test report. The other rests on someone ticking a box in a questionnaire eighteen months ago because the deadline was that afternoon.

In every governance system I have seen, those two answers are stored identically. Both are green. Both roll up into the same compliance percentage. Both are presented to a regulator with the same degree of confidence, which is to say more than one of them has earned.

This is not a modelling problem or a data quality problem. It is a missing primitive. The system has no way to represent how much the answer is worth.

Verdict and basis are two different things

The fix is to make every verdict carry a second value alongside it: its basis — what the verdict actually rests on. Not a confidence percentage, which models produce cheerfully and calibrate badly, but a categorical statement about the kind of support underneath the claim.

Four rungs are enough, and they are ordered:

Evidenced means a document, test result or measurement proves it, and the system can point at the specific artefact. Corroborated means somebody declared it and independent observed data agrees — weaker than proof, considerably stronger than a claim alone. Attested means it was declared and nothing else. None means there is no evidence in either direction: nobody has looked.

The ordering is what makes it useful. It converts a flat set of answers into something with a gradient, and a gradient gives a programme a direction to move in.

The two rungs everybody gets wrong

The bottom two rungs are where the value is, and both are routinely mishandled.

Attestation is not comfort. It is unverified risk. An organisation that records four hundred attested controls as satisfied has not measured compliance; it has measured how many people answered a questionnaire. That is worth knowing, and it is a different thing. Once attested is visibly distinct from evidenced, the honest number usually surprises people — and the surprise is the most useful output of the first month.

Absence of evidence is a finding, not a blank. Systems that silently exclude controls they could not assess produce a compliance percentage calculated over the subset that happened to be measurable, which is a statistic about your tooling rather than your estate. The honest treatment is to count "no evidence" as an outcome, score it as exposure, and put the number on the same page as the good news. In one real portfolio, over half the applicable controls for a representative application sat at none. Under the previous process, all of them had been recorded as satisfied.

Why a model must not produce the score

A language model is an excellent reader. Given properly labelled evidence it will weigh a vendor audit report against a self-assessment sensibly, notice that a resilience register claims two-hour recovery while the host inventory shows a single region, and explain why that matters. It should do all of that.

It should not then produce a risk score. Ask it to and you get a number that is plausible, unstable across runs, and impossible to defend when someone asks how it was derived. The moment your governance record contains a number nobody can recompute, the record has stopped being evidence.

So split the work. The model returns a verdict, a basis, its reasoning and citations. Everything numeric happens afterwards in ordinary code: impact derived from control criticality, exposure derived from the verdict and its basis, risk as the product, rollups as plain arithmetic. The same evidence always produces the same number, and an auditor with a spreadsheet can reproduce it.

The model judges. The code scores. Every property that makes a governance system defensible lives on the second side of that line.

Climbing the ladder is the actual work

Once basis is explicit, the programme acquires an objective that is not "improve the score". The objective is to move verdicts up the ladder, and to escalate the high-criticality controls that refuse to climb.

This reframes what counts as progress. Finding a document that already exists and making it citable is progress. Connecting a data source so that an attested control becomes corroborated is progress. Neither changes the environment at all — and both are frequently more valuable than remediation work, because they tell you whether the remediation was necessary.

A real example. A resilience platform held vendor disaster-recovery test reports as attached documents that no analytical process had ever opened. Extracting their text and feeding it in as a citable evidence layer moved several very-high-criticality controls from attested to evidenced, and cut one application's measured risk from 0.42 to 0.33. The application was exactly as resilient the day before. The organisation had simply been holding the proof, unread, the entire time.

A meaningful share of what enterprises carry as risk is not risk. It is unexamined evidence. You cannot tell the two apart without a basis field.

Attestation decays

One refinement matters in practice: basis has a time dimension. Evidence dated last quarter is a stronger claim than the same evidence dated three years ago, and an attestation ages faster than a test result because nothing was ever verified in the first place.

Modelling that decay explicitly means risk rises when nothing happens — which is correct, and which is the behaviour that turns a governance record from a snapshot into something that actually prompts action.

What it costs to adopt

Less than expected, because it is a schema change and a discipline rather than a platform. Add a basis field to whatever already stores your verdicts. Populate it honestly, including the uncomfortable rungs. Stop computing compliance percentages that quietly exclude what could not be measured. Report coverage by basis next to the headline number.

The hard part is not technical. It is the first meeting where the honest number is on the slide and it is much worse than the number that came before it. That meeting is the entire point. Everything after it is a real conversation about risk, which is not something most governance programmes have had.