On Confidence Readings and Why They Matter

Every finding in a delivered audit carries a confidence reading from 50 to 99 percent. The reading is the discipline that distinguishes the audit from a feed of search results. It tells the client, for each finding, how strongly the desk holds the conclusion.

The bands are:

Fifty to sixty-nine percent. The desk leans toward the finding but the evidence is mixed. A counterparty serious enough to read at this depth would draw the same provisional conclusion the desk has drawn, but further reading could change the picture. Findings in this band are flagged as developing.

Seventy to eighty-four percent. The desk holds the finding with reasonable confidence. The evidence is consistent across multiple sources; further reading is unlikely to change the conclusion materially. Most findings in a delivered audit sit in this band.

Eighty-five to ninety-four percent. The desk holds the finding with strong confidence. The evidence is unambiguous across primary sources. A counterparty reading the same sources would draw the same conclusion.

Ninety-five to ninety-nine percent. The finding is effectively certain. Documentary evidence is exhaustive and consistent. A counterparty's research would produce the same conclusion as a near-certainty.

What the readings are not is probability in any rigorous statistical sense. They are not Bayesian posteriors. They are the editorial judgement of a practitioner who has read the sources and weighs them. The same finding, read by a different practitioner, might be expressed at a slightly different confidence; the discipline is in expressing it at all and in defending the expression on the basis of the sources read.

What the readings do enable, however, is differential attention. A client reading a delivered audit can attend first to the high-confidence findings, then to the developing ones, and use the confidence to inform how much weight to give to each. The reading is the difference between a report that ranks and a report that lists.

The discipline of expressing confidence is what most automated systems do not do. An automated query returns hits; it does not return rankings of how confident it is in each hit. The audit, by ranking, takes responsibility for the ranking; the responsibility is what the client is paying for.