AI auditability is the ability of anyone outside the original session to check what an AI agent was asked to build, what it produced, who accepted it and what tested it. For a team running coding agents, that takes four linked records, one each for the requirement, the agent’s output, the human acceptance and the test evidence.
Without those records, handing implementation to a coding agent can leave us with far more code than we ever wrote by hand and a smaller share we can account for. Adding logs later doesn’t recover that account, because the pieces that explain the code were never captured as linked records while the work happened. This guide covers the four records in detail, the workflow design choices that keep or lose them, and how auditability differs from explainability.
What Is AI Auditability?
AI auditability is the property of an AI-assisted workflow that lets us reconstruct what was asked for, what the agent produced, who accepted it and what verified it, even when we weren’t in the original session. In an agent workflow, the records that answer those questions form a chain from the requirement to the generated output to the review evidence, and the chain is complete only if each record is versioned and linked to the next. We can have detailed logs and still fail the test if the logs capture tool calls but nothing ties the diff back to an approved requirement.
High-risk AI systems must technically allow for the automatic recording of events (logs) over the lifetime of the system under Article 12(1) of the EU AI Act. Providers of high-risk systems must keep those logs for a period appropriate to the intended purpose and of at least six months under the same act. AI actors should also ensure traceability, including for datasets, processes and decisions made during the system lifecycle, under the Organisation for Economic Co-operation and Development (OECD) AI Principles. Both texts govern AI systems themselves: the logging and retention duties bind high-risk systems and their providers, and a coding-agent workflow used to build ordinary software has no such minimum requirement.
Neither text uses the word auditability. Auditability is the capacity of an AI system and its development processes to be independently evaluated for compliance whether or not the system itself is transparent or explainable, as Verma, Padh and Thelisson define it in a 2025 paper. AI used as a tool to write software isn’t addressed in the FDA draft guidance on AI-enabled device software functions from 7 January 2025. The European Union Aviation Safety Agency’s Notice of Proposed Amendment 2025-07 doesn’t address it either and leaves generative AI to a later extension of the agency’s framework, so the burden of proving what an agent built stays with us in the meantime.
Tool qualification offers one established model for a coding agent. Under DO-178C, a tool whose output becomes part of airborne software and could insert an error is a Criterion 1 tool, and DO-330 governs its qualification unless its output is independently verified. An agent that writes code fits that description by analogy, so without tool qualification its output needs the same independent verification. For our workflow, that means a named person checked the change against its requirement and recorded that they did.
What Makes an AI-Assisted Workflow Auditable?
Four records, each versioned and linked to the next, make an AI-assisted workflow auditable: the requirement the agent worked from, the output it produced, the acceptance a person recorded, and the verification that proved the output met the requirement. If one record is missing, AI traceability stops at that point, and the chain cannot answer the corresponding question about intent, output, responsibility or proof.
How Do We Govern the Requirement the Agent Uses?
An auditable input is an approved specification with a version, an owner and a record of who approved it. A specification-based framework can align humans and agents before code is generated. OpenSpec appears in the Thoughtworks Technology Radar as one example. Reviews should also cover the tests, invariants and boundaries that constrain what the agent generates, as Ken Mugrage at Thoughtworks argues. Neither source says who approves the specification or how that approval attaches to a version.
A general-purpose AI tool used on its own has no product context, no requirements governance and no audit trail back to an approved, versioned requirement. The specification it worked from is therefore whatever text happened to be in the context window. In Jama Connect®, a web-based requirements management and traceability platform for complex, regulated product development, the approved requirement can retain its version history and reviewer sign-off. The item the agent read is then the same item an auditor later sees.
The quality of that input is part of the record too. An ambiguous requirement becomes a finding in its own right because accuracy, consistency and unambiguousness are verifiable objectives for high-level requirements under DO-178C. Jama Intelligence scores a requirement 30 of the 40 International Council on Systems Engineering (INCOSE) rules and 6 Easy Approach to Requirements Syntax (EARS) patterns while the engineer is still writing. MilliporeSigma, which works in life science device development, was an early adopter of that authoring check. In an agent workflow, the authoring-time review has to finish before the agent turns an ambiguous requirement into code.
How Do We Version What the Agent Produces?
We version agent output at the commit, with the agent named as author and the session linked from the commit message, so the record shows what the agent produced and from which session. Commits from GitHub’s Copilot cloud agent are authored by Copilot with the task assigner as co-author, signed and shown as Verified. Each commit message includes an Agent-Logs-Url trailer linking to the session logs.
Session logs are one ingredient of auditability, but they “do not replace your own review and testing,” according to GitHub’s documentation. In Claude Code and Codex, exporting session telemetry is opt-in, and prompt text is redacted unless a team turns on prompt logging. The session record ties prompt to output, while we still have to create and retain the requirement-to-output link elsewhere. Without that link, a reviewer can replay the session and still be unable to establish why the commit exists.
Who Accepts Responsibility for AI-Generated Code?
The human who reviews the change against its requirement and submits it accepts responsibility, and our acceptance record names that person and the requirement they checked it against. AI agents must not add Signed-off-by tags under the Linux kernel’s coding-assistant policy, because only a human can certify the Developer Certificate of Origin (DCO). The human submitter reviews all AI-generated code and takes full responsibility for it, while contributions include an Assisted-by tag that discloses the AI assistance. More than 1,200 commits carried the tag by 7.2-rc4, and Greg Kroah-Hartman described it as a signal for closer review.
Vendor tools record acceptance as aggregate counts, such as Claude Code’s “accepted code lines” and Cursor’s totalAccepts, or as tool-action approvals. Aggregate counts and approvals don’t create a per-change record of who accepted responsibility against a named requirement.
A merge approval on its own is weak evidence as well. Larger AI-generated changesets are more likely to get a quick approval, a senior engineer observed in a discussion Mugrage reports. More developers actively distrust the accuracy of AI tools (46%) than trust it (33%) in the Stack Overflow 2025 Developer Survey, which drew more than 49,000 responses. We need an acceptance record that turns human-in-the-loop review into a structured field the merge gate can check, holding the reviewer, the requirement they reviewed against and the review they performed.
Which Tests Prove the Output?
Proof comes from tests written from the requirement before the agent writes any code, each named in our verification record against the requirement it covers. Tests written from generated code check what the code already does, and a model asked to write both the implementation and its tests tends to produce tests that pass against its own implementation. A pull request (PR) with test files can also look well verified to a reviewer scanning the file list, whatever the underlying verification strength, practitioners reported in one study. Test-driven development prevents the agent from writing a test that merely verifies its own broken behavior, which is why Mugrage recommends it.
A test written first becomes the executable spec, as Cecilia Geraldo at Thoughtworks put it. Session logs are not test records, so linking each commit to the test results that verified it is a step we have to design into the workflow ourselves. The standards already expect the trace to reach the tests. IEC 62304 expects software system test procedures to trace to the software requirements they verify, and DO-178C requires bidirectional traceability from requirements through code to tests.
How Our Workflow Design Choices Create or Destroy AI Auditability
Choices made before the agent runs determine whether the four records behind AI auditability are kept, and each choice below loses part of those records. We see the effect of those choices when an auditor asks us to reconstruct the change:
- Context kept in markdown files: Current workflows rely on markdown: almost all AI coding context engineering consists of markdown files containing prompts. Some tools also compact the conversation history periodically, so we can no longer version or replay the full context that shaped a change.
- Attribution by text marker: An empirical study of Claude Code PRs identified agent PRs only through the explicit string “Generated with Claude Code” across 567 PRs in 157 projects. The study’s binary classification, agent or not, may overlook hybrid workflows, so mixed human-and-agent changes lose the record of who wrote which part.
- Merging without a recorded reason: A third of the rejected PRs inspected by hand (117 of 353, or 33.1%) lacked any observable decision rationale in an analysis of 11,048 closed agentic PRs that sampled 717 for manual inspection. Closing a PR therefore did not always preserve why a reviewer rejected the agent’s work.
- Guardrails expressed as prompts: An agent deleted a production database during the July 2025 Replit incident, and the user who lost it concluded that a code freeze could not be enforced in the tool. A guardrail that exists only in a prompt is the unenforced kind, and it leaves nothing for an auditor to inspect.
The choices that create AI auditability are the opposite, and most of them are enforced in the tooling. Explicit approval of high-impact actions, action previews and rollback are recommended in the Open Worldwide Application Security Project (OWASP) AI Agent Security Cheat Sheet. We should also apply least-privilege tool policies per agent instance and keep a log that connects every agent action to its originating input, following OWASP Cornucopia AAI2. With that log in place, a reviewer can trace each agent action back to the input that triggered it without replaying the session.
How Does AI Auditability Differ From Explainability?
Explainability asks how a model reached a decision, while auditability asks whether the system and its development processes can be independently evaluated. For a coding agent, auditability rarely depends on the model explaining itself. The table places the four properties side by side, using definitions from the National Institute of Standards and Technology (NIST) and academic literature.
| Property | Question it answers | Defined by |
| Transparency | “what happened” in the system | NIST AI Risk Management Framework (AI RMF) 1.0 |
| Interpretability | “why a decision was made” and its meaning to the user | NIST AI RMF 1.0 |
| Explainability | “how” a decision was made in the system | NIST AI RMF 1.0 |
| Auditability | Whether the system and its development processes can be independently evaluated against standards | Verma, Padh and Thelisson, 2025 |
The NIST AI RMF treats the first three as distinct characteristics that support each other. Auditability can be preserved even for a process whose inner workings stay hidden. Cryptographic techniques can prove that the same process was applied in every case without revealing what the process is, as Joshua Kroll argues in his work on accountable algorithms. He calls that property procedural regularity.
If our record is complete, does anyone still need the model to explain itself? For an audit, it doesn’t: the auditor asks what the requirement said, who accepted the code and which tests passed, and those records exist whether or not anyone can explain why the model chose one implementation over another. In spec-driven AI coding, compliance documentation can exist as a byproduct of the development process, and the specifications are the system of record for audit evidence. Each generated behavior traces to an approved requirement and an accountable human decision, while the model generates but is never accountable.
How Jama Connect Supports AI Auditability
Every change that Jama Intelligence or the MCP Server makes in Jama Connect is recorded in the item’s version history as made by that tool, so an auditor or reviewer can confirm which content came from AI. External AI tools cannot provide that record. Jama Connect provides the foundation that is the Product Context Layer for engineering teams, connecting requirements, risks, tests, Systems Modeling Language (SysML) models, code repositories, simulations, defects, reviews, approvals, verification evidence and change history into a governed system of record. The Traceability Information Model (TIM) relates requirements to code, tests and verification results, which covers three of this article’s four records. Jama Connect Review Center holds the fourth, the formal human review and approval behind the acceptance record.
The Model Context Protocol (MCP) Server, which shipped on 4 May 2026 with Jama Connect 9.35 as an add-on for Cloud and Self-Hosted deployments, exposes that governed context to agents in tools such as Claude, Codex, Cursor or GitHub Copilot while enforcing existing Jama Connect permissions, lifecycle workflows and audit requirements. Jama Intelligence drafts test cases from the governed requirement, but the source link and test artifact are created only after an engineer reviews and accepts the suggestion, and the engineer can regenerate with additional prompting first. When the requirement changes, each link to a downstream test case becomes a suspect link for the tester to reassess, and milestone baselines preserve a point-in-time state of the record. Jama Intelligence is an add-on for Jama Connect Cloud.
Build AI Auditability Into Your Agent Workflow
The merge gate is where an agent workflow becomes auditable. A gate that captures the approved requirement identifier, agent-authored commit, named reviewer and passing test result can answer an auditor from the record. For agents working outside Jama Connect, the commit can include the requirement identifier that points back to the review and verification links held there. If we hand implementation to an agent today, the record we keep is the one we’ll be judged on when guidance arrives.
When agents already produce more code than we can trace back to a requirement, the first thing to check is what our current merge gate keeps with each change. A demo shows a requirement, its linked tests and their review status held together when we see it in action.
Frequently Asked Questions About AI Auditability
Does AI auditability require keeping every prompt?
AI auditability doesn’t require every prompt, because the durable record is the requirement identifier on the commit, which stays with the code after session logs age out of vendor retention windows. In Jama Connect, baseline control can preserve the approved requirement version associated with that identifier.
How does AI auditability apply under IEC 61508 or ISO 26262?
Both standards ask for evidence about a tool whose output becomes code. A tool whose output can contribute to executable code falls into class T3 under IEC 61508, which requires evidence that the tool conforms to its specification. Under ISO 26262-8, the tool confidence level comes from tool impact and error detection. Generated code that ships in the runtime system is software under both standards, so an agent’s code needs the same review, trace and release records as a person’s.
Who is accountable for AI-generated code in an audit?
We place accountability with the person who accepted the code, and under the OECD principles accountability also rests with AI actors “based on their roles.” Developers remain responsible for harms and should not blindly accept suggestions, according to the Open Source Security Foundation (OpenSSF) guide on AI code assistant instructions. A significant contribution taken from an AI tool without changes requires disclosure under Fedora’s policy. Apache Kafka contributors are asked to add a Co-Authored-By, Generated-by or similar commit trailer.
What is the difference between AI auditability and an AI audit trail?
An audit trail is the record, and auditability is the property that lets the record and its links answer an auditor’s question. We should be able to move from the requirement identifier to the accepted change and verification result without reconstructing the session. Jama Connect item version history records who changed what, when and why along that path.
