The trust framework
Automation that can be checked
A method for building automation that can be explained, tested and traced, and for saying honestly what it can and cannot do.
Automation earns trust when a sceptical person can ask four questions and get a specific answer. Should this be automated, and are we allowed to? How do we automate it well? How do we know it is good enough? What happened in this case? Everything we build is designed around those four: Decide, Design, Demonstrate, Document.
01
Decide
Should this be automated, and are we allowed to?
The business case, the compliance position and the standard it must meet, agreed first and written down as a warrant.
02
Design
How do we automate it well?
Automate the effort, constrain the AI, escalate the interpretation to a person.
03
Demonstrate
How do we know it is good enough?
Judged against expert golden sets with a proper statistical framework, and every figure reported with a confidence interval.
04
Document
What happened in this case?
An audit trail for everything, from raw data to finished output.
Decide: should this be automated, and are we allowed to?
Most automation problems are settled, or created, before anything is built. So the first step is a decision, made deliberately and written down. It opens the warrant: the evidence that this automation is lawful, tested and defensible. Decide starts it with the case for automating. Design adds the specification, Demonstrate adds the test results, and Document completes it with the audit trail. It is what you keep, and what you show anyone who asks.
We run Decide as a structured interview with the people who are accountable, not as a checklist. It names the things the automation handles, the decisions that get made about them, who is accountable for each decision, and what counts as finished. Those answers become the specification Design builds to, the test set Demonstrate marks it against, and the record Document keeps. The people who disagree with their own colleagues about the definitions are usually the reason to do it.
- Purpose and business case. What problem it solves, for whom, and what it is worth, including what it would cost to do nothing and what it would cost to get wrong.
- Lawful basis and compliance. Whether the data may be used this way and what is needed to do it properly: data protection and impact assessments, sector rules, the transparency obligations that apply, and whether the rules on decisions made solely by automated means affect people here. We prepare the facts; we do not give legal advice.
- An accountable owner. A named person who answers for the outcome.
- Boundaries. What it will touch, what it will not, and which decisions stay with people.
- Risks accepted. The ways it could fail, who has agreed to live with them, and what would make us stop.
- What good enough means. The standard it must meet, set before it is built, so that testing later cannot quietly move the goalposts.
Sometimes the answer is not to automate. A warrant that says so is a good outcome, and a cheaper one than finding it out later.
Design: how we automate it well
The principle is simple: automate the effort, constrain the AI's judgement, escalate the interpretation to a person. Every task an automation touches is one of three kinds, and each kind is handled differently.
- Deterministic steps: automate the effort. Cleaning, matching, calculating, formatting: work where there is a right answer and rules can find it. These are automated completely and tested like any other software. If a rule can do it, a model is not used.
- Routine judgement: constrain the AI. Categorising, extracting or summarising against a framework a person has already set. This is where AI earns its place, but on a short lead, as described below.
- Decisions about meaning: escalate to a person. What a theme is, what a finding implies, whether an exception applies, what to recommend. These go to a person, by design. The system's job is to put the decision in front of the right person, with the evidence they need.
How the constraint is done technically
- A closed set of permitted outputs. The model answers in a fixed structure against a fixed framework. It cannot invent a category, and it has no authority to change the framework.
- Rules enforced in code. The things the automation must never do are enforced by code with tests, not by asking the model nicely in a prompt. A prompt can be argued with; a guard cannot.
- A way out. There is always a defined way to say this does not fit. An item that fits nothing is left unresolved and shown to a person, never forced into the nearest category.
- Voting. Where a judgement matters, the same item is judged several times, independently, and the agreement between the runs is the signal. Unanimous answers pass. Split answers are not averaged away: they are routed to a person and the dissent is recorded, because disagreement between runs is the cheapest reliable sign that an item is hard.
- Adversarial testing. Before anything is relied on, we try to break it: cases built to be hard or ambiguous, inputs that try to redirect the model, and a separate checking process whose only job is to find what the first one got wrong. What survives goes into the test set permanently.
- The smallest model that does the job. We start with the smallest, cheapest model and effort setting and move up only when an evaluation fails. Cost then follows evidence, and the rule is one a client can hold us to.
- Escalate by need. Human review starts with the items most likely to need judgement, such as split votes, low confidence, unusual or high consequence, so limited attention goes where it matters most.
Demonstrate: how we know it is good enough
Where a tool involves judgement, we test it against work produced by experts in that field, using a proper statistical framework, and we do not call it ready until it meets the threshold the warrant set.
- Build a golden set first. The experts produce it by hand, before the tool sees the material. It is the standard the automation is judged against, and it is kept separate from anything used to tune the tool.
- Measure agreement properly. Chance-corrected statistics, per category as well as overall, not raw percentage agreement. Our note on validating an LLM that codes open ends shows why raw agreement flatters a model when categories are unbalanced.
- Report every figure with a confidence interval. No performance number appears without one. A result of 90% from fifty items and 90% from two thousand are very different claims, and the interval is what tells them apart. We plan the size of the golden set in advance so that the interval will be narrow enough to act on.
- Allow a result to be inconclusive. Each test ends in pass, fail or inconclusive, with the baseline captured before the change is approved. Forcing every result into pass or fail manufactures confidence nobody earned.
- Compare with the human baseline. Experts do not agree with each other perfectly either. The target is at least as good as the manual process, and the manual process is measured the same way.
- Say what was and was not tested. If a tool has not been validated yet, we say so and describe the checks that run on each piece of work instead. Results are shared openly, including the ones that are unflattering.
- Test again when something changes. A new model version, a reworded instruction or a different kind of data is a new thing to test.
Document: an audit trail for everything
Every output comes with a full audit trail from raw data to finished result, supplied as standard, and it completes the warrant. It holds the inputs and their version, every transformation in order, every automated judgement (which rule or model, which version of the instructions, what it decided, how confident it was, how the votes fell), and every human decision (who, when, what they changed). Outputs link back so that any figure or quote can be traced to the rows or passages behind it.
The trail is append-only: entries are never edited after the fact, the evidence is attached at the moment each entry is written, and anything done by hand is marked as done by hand.
An audit trail is written for someone who was not there, months later, with one question about one output. That is a different document from a system log, and it has to be designed. We wrote up what it needs to contain.
The warrant: what you keep
The four steps build one thing, and it is the thing you are really buying. It is the evidence that an automated process is lawful, tested and defensible, in a form you can hand to an auditor, a regulator or a member of the public who asks.
| Step | What it adds to the warrant |
|---|---|
| Decide | The purpose and business case, the impact assessments, the lawful basis, the answer to the automated-decision question, the accountable owner, the risks accepted and the standard to be met. |
| Design | The specification: what it handles, which decisions stay with people, the rules it cannot break, and which model was chosen and why. |
| Demonstrate | The golden set, the results with their confidence intervals, and the verdict against the standard that was set at the start. |
| Document | The audit trail and the incident runbook, so the thing can be operated, challenged and explained months later. |
Governance runs through all four
Before work starts we agree, and write down:
- where data is processed and stored;
- who can access it, and how that access is limited;
- how long it is kept, and how it is deleted;
- whether any AI service is allowed to retain or learn from it (we choose services and settings so that it is not, and say which);
- whether personal details are removed before processing where they are not needed;
- which decisions need a named person's approval.
What this is not
Why a standard, not a one-off
When each automation gets its own bespoke controls, standards vary, each one has to be explained separately, and lessons learned on one are not applied to the next. Asking the same four questions of everything has three effects. Leaders get one story they can tell in a sentence. Checks are improved once and everything benefits. And there is one place to see what is running, what has been tested and what is not ready yet.
The trust sheet
Every automation we deliver gets a one-page trust sheet. It is written for the person who has to decide whether to rely on the thing, not the person who built it. The rule is simple: if something has not been validated yet, the sheet says so.
Template
Trust sheet: [automation name]
A worked example, for a tool we run ourselves: the trust sheet for AI Signal.
How we say what is ready
Each capability carries one of three statuses, and we use the same words in proposals and on trust sheets.
| Status | Meaning |
|---|---|
| Green | Tested against expert work, results available, safe to rely on within its stated limits. |
| Amber | Working and being validated. Used only with full human review or on low-consequence work, and we do not quote performance figures we have not measured. |
| Red | Not ready, or not something we would offer. We say so rather than stretch. |
How we talk about it
Claims should match the evidence and state their limits, so some phrases are off the table.
| We say | We avoid |
|---|---|
| Tested against expert work, with the result and what it does not cover | "Accurate", "100% reliable" |
| Every step is recorded and traceable | "Black box", "magic" |
| People make the decisions that matter | "AI does the analysis" |
| Faster on the routine steps | "Instant", "effortless" |
| "Not yet validated", when that is true | Saying nothing |