Provenance by construction: AI analysis your auditors can replay

There is a moment in every AI pilot at a financial institution where someone senior asks the question that decides the project's fate: "And when the regulator asks where that number came from?" Most pilots end there — not because the answers were wrong, but because nobody could prove they were right. "The model said so" is not an answer a compliance officer can sign, and everyone in the room knows it.

The usual response is to bolt an explanation feature onto the side: ask the model to describe its reasoning, store the description, call it an audit trail. We think that approach fails in a way that no amount of polish can fix, and this article is about the alternative we built instead — provenance as a construction constraint. In DataQube, the agent cannot produce a claim without the evidence chain existing, because assembling the chain is how claims get produced. The audit trail is not documentation of the analysis. It is the analysis.

What an AI audit trail actually has to contain

Start from what a reviewer needs, because that defines the data structure. When a supervisory team, an internal auditor, or a model-risk function examines an AI-produced figure, they ask five questions in a predictable order: what was the agent allowed to touch, what prior knowledge did it apply, what did it actually run, how was the number computed from what came back, and what checks were done before anyone repeated it.

Each of those maps to a stage in what we call the evidence chain, and every answer the agent produces carries all five:

• Scope — the workspace, the grants in force, and the governance gate the session passed before the first call.
• Recalled knowledge — which validated definitions and conventions were applied, and when each was last confirmed.
• Audited calls — every query, file read, and run, each with an audit event reference.
• Computation lineage — how retrieved rows became the figure on screen.
• Checks — what was reconciled against an independent source before the answer shipped.

None of this is prose. Each stage is structured data, emitted at the moment the work happens, with identifiers that resolve into the audit log. That distinction — structured events at execution time, not narrative afterwards — is what makes the rest of this article possible.

Anatomy of one auditable answer

Here is a real example of the shape, from our standing demonstration scenario: a bank's Group Risk team asks about commercial-real-estate concentration ahead of a supervisory response. One finding — "Office CRE is at 92% of its internal sub-limit" — assembled from four different systems, each call carrying its audit reference:

Replay run

Based on real customer conversations — all data fictional.

Walk the chain behind that finding and you find no narration, only records. The exposure side came from a read-only warehouse query — 18,204 rows, 213 milliseconds, audit event evt-4c19d2. The 12.5% aggregate limit was not remembered or assumed: it was read from §3.2 of the board-approved policy document, and the citation points at the section and quoted passage, not just a filename. The monthly figures were not derived fresh — the agent invoked the standing cre-monthly-exposure workflow, run #38, parameters pinned, which is why the number in this ad-hoc conversation is identical to the number in the distributed monthly pack. And before the finding was stated, the segment exposures were reconciled to the general ledger within ±0.1%.

Four systems, four audit references, one correlation id tying the whole run together. A reviewer does not have to trust the sentence; they can open any link in it.

Claims carry receipts, not footnotes

The load-bearing design decision sits one level below the UI: a claim in DataQube is a data structure whose fields are its evidence. Here is the same Office CRE finding, unwound — this is the panel that opens when anyone clicks "How was this derived?" on the finding above, exactly as it appears in the product:

Based on real customer conversations — all data fictional.

The order of operations is what matters. The agent does not write the claim and then attach supporting material — it cannot. The claim object is assembled from the receipts: the query result supplies the exposure, the document read supplies the limit, the workflow run supplies parity with the standing pack, the check supplies the reconciliation. Remove any receipt and there is no claim to state. We call this provenance by construction, in deliberate contrast to provenance by decoration — the same JSON generated after the fact by asking a model what it probably relied on.

The practical consequence shows up in the failure cases. When a source is unavailable — the policy document moved, the warehouse timed out — a decorated system produces a confident answer with a thinner explanation. A constructive system produces no figure, and says why. For a regulated team, the second behavior is the entire point.

The refusal is part of the record

An audit trail that only records successes is a marketing document. Real reviews spend most of their time on the negative space: who tried to access what and was refused, which limits fired, what the system declined to do.

Every authorization decision in DataQube lands in the same immutable log as the data access it governs — including, especially, denials. If an analyst's session attempts to reach a source outside their grant, the refusal is recorded with the rule that fired. If a sensitive action required a person's approval, the record shows who was asked, what exactly they approved, and when. The permission prompt and the human answer are events with the same standing as a query.

This is also why "explicit denials always win" is an architectural statement rather than a policy preference: a denial that can be argued around by an agent is not evidence of control, and evidence of control is what an auditor is actually there to collect.

What a reviewer can replay without system access

The chain would be of limited use if examining it required a DataQube login and a friendly admin. It exports.

The exported trace for a thread contains the five stages for every claim, the full audit-event list with timestamps and row counts, the document citations with their quoted passages, and the workflow run references with their pinned parameters. A reviewer working from the export alone can do three things that matter: verify that every figure has a complete chain with no gaps; re-run the cited queries against the source systems through their own access and compare row counts and results; and confirm that policy numbers trace to the governing documents rather than to the model's memory. Log lines never contain credentials, personal data, or raw result sets — the trail is evidence, not a second copy of your data — so the export can move through a review process the way any other audit artifact does.

In other words: the person who signs the review does not need to trust our software. They need to trust their own warehouse, their own document store, and arithmetic.

Why post-hoc explainability fails the audit

It is worth being precise about the rejected alternative, because it is the industry default. "Explainable AI" in most current products means asking the model, after it has answered, to produce an account of its reasoning. The account is fluent, plausible, and structurally unverifiable — it is a second model output about the first, with the same epistemic status and no independent check. When the explanation and the actual computation diverge, nothing detects the divergence. An auditor cannot distinguish a faithful explanation from a confabulated one, which means, for audit purposes, the explanation carries no information at all.

The constructive approach gives up something real in exchange: the agent is slower to answer than a system that skips bookkeeping, and it refuses in situations where a chat product would improvise. We consider both costs the product working as intended. An answer produced without its chain is not a discounted answer — in a regulated setting it is a liability with good grammar.

The question to ask any vendor

If you are evaluating AI analysis tools for a regulated environment, there is a one-question test that separates the two architectures: pick any number in the output and ask to see the query that produced it, the document behind any policy figure, and the record of what the system refused to do along the way. A provenance-by-construction system answers with links. A decoration system answers with prose.

Our answer to that test is on the security page, including the evidence chain rendered in full — and if you want to run the test against your own data, book a demo and bring your hardest number.