AI governance: control for systems that don't repeat

The discipline is not new

Running an interoperability protocol across many partner networks means living with a system you do not control end to end. Another company's implementation changes without notice, behaviour varies between partners, and when something goes wrong months later you have to prove what your side sent and received.

The answer in that world is well understood: conformance testing against a fixed suite, versioned change control, strict data boundaries, immutable audit logs, and periodic compliance audits against the specification rather than against what happens to work.

AI governance is that same discipline pointed at a probabilistic component. The vocabulary is newer; the controls are not. What genuinely changes is that your test suite can no longer assert equality — it has to measure a distribution and alert on movement.

The five controls

Every one of these has a direct analogue in protocol compliance work. None of them requires a specialist AI-governance product to implement.

Evaluation is the regression suite

You cannot ship changes safely to a system you cannot measure. The golden set is the foundational artefact of AI governance, and building it is the first thing to do — before prompt tuning, before model selection, before anything.

What it contains:

• 100–200 real questions drawn from actual usage, not invented.
• Expected sources for each — which documents should have been retrieved.
• Reference answers, or at least acceptance criteria.
• Adversarial cases — questions the system should refuse, ambiguous questions, questions with no answer in the corpus.

Because output varies between runs, gate on distributions rather than exact matches: run each case several times, track the mean and the spread, and alert when either moves. A metric that becomes more variable is degrading even if its average holds.

Change control

“What is in production?” must have a precise answer. For a RAG system that means at least five versioned components:

Provider-side model updates are the governance gap most teams have. If you point at a floating model alias, your system's behaviour can change without any deployment on your side, and your evaluation results become historical fiction. Pin to explicit model versions, treat a version bump as a change requiring an evaluation run, and keep the previous version available to roll back to.

Record the full component set with every response. When an answer is disputed weeks later, reconstructing which prompt, which model version and which index generation produced it is the difference between an explanation and a shrug.

Data boundaries

The context window is a data egress path. Anything placed in it has left your database and entered a model provider's inference pipeline, and in most enterprises that crossing needs to be deliberate.

Three rules that carry most of the weight:

• The model inherits the caller's permissions, not the service's. A retrieval service running with broad database access that does not filter per user has effectively granted every user that access. Enforce ACLs inside the search query — never as a post-filter, which is a known leak pattern.
• Exclude secrets at ingestion. Credentials in a wiki page will be embedded, indexed and eventually retrieved. Scan and strip on the way in, because once vectorised they are difficult to find again.
• Know where inference runs. Under India's DPDP Act and comparable regimes elsewhere, sending personal data to a model endpoint in another jurisdiction is a transfer with obligations attached. Regional deployment is an architectural decision, and it is far cheaper to make at design time than to retrofit.

The audit trail

The governing question is: can you reconstruct why the system said what it said? That requires recording more than the input and output.

request_id       req-8f2a91
timestamp        2026-08-11T09:47:10Z
caller           user-4471  (roles: support-l2)
question         "What is the reservation cancellation fee?"
retrieved        [doc-882#c14, doc-104#c3, doc-882#c15]   scores [0.81,0.77,0.74]
filters_applied  tenant=acme  acl=support-l2
prompt_version   v2026-07-18
model            gpt-4.1-2026-05  (pinned)
index_version    idx-2026-08-09
tokens           in 3,412 / out 218
response         "Cancellation is free while the booking is..."
citations        [1] doc-882#c14
escalated        false

The retrieved chunk IDs are the field teams most often omit and most often need. Without them, an incorrect answer cannot be diagnosed — you cannot tell whether retrieval failed or generation did, which is the first branch of every investigation.

Retain on the same basis as any other financial or customer-facing record, and treat the log as immutable. It is the artefact that answers a regulator, a customer complaint, or an internal dispute — the same role a raw protocol payload plays in a partner disagreement.

Human oversight

Automation without an escalation path is an unowned system. Three mechanisms, in increasing cost:

Sampled review is what surfaces the failures nobody reported. Users rarely report a confidently wrong answer — they act on it, or they quietly stop using the system. A steady sample read by someone who knows the domain is the cheapest early-warning system available.

The review output feeds the golden set. Every failure found becomes a test case, so the same failure cannot silently return. That loop — incident to test case to gate — is what makes governance compounding rather than ceremonial.

Production lessons

• Build the golden set before anything else. Without it every subsequent decision is unmeasured.
• Treat prompts as production code. Versioned, reviewed, tested, deployed.
• Pin model versions. A floating alias means uncontrolled change.
• Log retrieved chunk IDs. The most commonly omitted field and the first one an investigation needs.
• Enforce access control inside the query. Post-filtering is a leak.
• Strip secrets at ingestion. Once embedded they are hard to locate.
• Sample and review continuously. Silence is not evidence of correctness.
• Feed every incident back into the suite so failures cannot recur unnoticed.
• Decide data residency at design time. Retrofitting regional inference is expensive.