Data provenance
Data provenance is the recorded lineage of a result: the sources, queries, transformations, and decisions that produced it, retained in enough detail that someone else can verify the figure independently.
Provenance answers a question every reviewer eventually asks: where did this number come from? A complete answer names the source system, the query or file that produced the value, any transformation applied along the way, and the time the data was read. Without those, a figure is an assertion rather than a finding.
The distinction between provenance and logging is worth drawing. A log records that something happened. Provenance records enough to reproduce it. A log entry saying a report was generated at 14:02 does not let anyone re-derive the report; a provenance record naming the query, parameters, and dataset version does.
Provenance is most credible when it is a by-product of how work is done rather than documentation assembled afterwards. Reconstructed lineage depends on memory and good intentions; captured lineage does not.
What distinguishes it
- Names the specific source, query, and time behind each figure
- Retains enough detail for an independent party to reproduce the result
- Captured while the work happens rather than reconstructed afterwards
- Survives staff turnover, because it does not depend on anyone's recollection
In DataQube
DataQube attaches evidence as answers are produced. A figure in a thread carries the query that generated it, the source it ran against, and the permission decision that allowed the access — so a reviewer can replay the derivation without needing access to the original analyst or their session.
Common questions
How is data provenance different from an audit log?
An audit log records that an event occurred, usually for security review. Provenance records how a result was derived, in enough detail to reproduce it. The two overlap but serve different questions: an audit log answers who did what, provenance answers where this number came from.
Why does provenance matter for AI-generated analysis?
Because a language model can produce a confident, well-formatted figure with no underlying computation. Provenance is what distinguishes a retrieved and calculated result from a generated one, which is precisely the distinction a reviewer or regulator needs.
What does complete provenance need to include?
At minimum: the source system, the exact query or file read, any transformation applied, the time of access, and the permission under which it occurred. Anything less leaves a gap a reviewer has to fill with trust.
Related terms
AI data agent
An AI data agent is software that answers analytical questions by planning and executing its own steps — querying approved databases, running code, and citing what it used — rather than only generating text.
Deterministic workflow
A deterministic workflow is an ordered, parameterised sequence of steps that produces the same result when run again with the same inputs, making it suitable for recurring reporting and review processes.
Air-gapped AI
Air-gapped AI is an AI system that operates entirely inside a network with no external connectivity, meaning model inference, data access, and licence validation all function without any call leaving the perimeter.