![\"State](\"https:\/\/www.jamasoftware.com\/media\/2026\/06\/ai-governance-at-scale-1.png\")
<\/p>\n<\/p>\n
Imagine a state agency\u00a0deploys\u00a0an AI tool to help draft procurement requirements. The tool is fast, the outputs look good, and program staff start using it across multiple projects.\u00a0<\/span>\u00a0<\/span><\/p>\n Six months later, the inspector general requests documentation showing how a specific contract requirement was developed, who approved it, and how it connected to the original policy directive.<\/span>\u00a0<\/span><\/p>\n No one can produce that record.<\/span>\u00a0<\/span><\/p>\n The AI tool did exactly what it was supposed to do, but the governance structure around it\u00a0didn’t.<\/span>\u00a0<\/span><\/p>\n This scenario plays out in variations across state government every day. AI adoption is accelerating, but the frameworks needed to make that adoption defensible, including under legislative oversight, audit scrutiny, procurement challenges, and public records requests,\u00a0haven’t\u00a0kept pace.<\/span>\u00a0<\/span><\/p>\n This article outlines\u00a0concrete\u00a0challenges of governing AI at scale in\u00a0government settings, explaining\u00a0why\u00a0they’re\u00a0especially difficult for state agencies, and offers practical guidance on where to focus first.<\/span>\u00a0<\/span><\/p>\n When an AI tool gets deployed,\u00a0multiple\u00a0teams often think someone else owns it:\u00a0<\/span>\u00a0<\/span><\/p>\n This distributed confusion\u00a0isn’t\u00a0unique to government, but\u00a0it’s\u00a0more consequential there. In the private sector, a gap in AI ownership creates operational risk.\u00a0<\/span>\u00a0<\/span><\/p>\n In state government, it can mean a program\u00a0operates\u00a0without accountable\u00a0review,\u00a0a use case expands beyond what was originally authorized, or no one flags that an AI output influenced a high-stakes decision without oversight.<\/span>\u00a0<\/span><\/p>\n Effective AI governance\u00a0requires\u00a0clearly defined ownership at every stage:\u00a0<\/span>\u00a0<\/span><\/p>\n Here\u2019s\u00a0what this looks like in practice.\u00a0A state agency might have a program manager, an IT lead, a compliance officer, and a vendor all involved in an AI implementation.\u00a0<\/span>\u00a0<\/span><\/p>\n Without explicitly assigning who owns governance decisions, each one defers to the others. The result is no one does.<\/span>\u00a0<\/span>\u00a0<\/span><\/p>\n This is one of the most underappreciated AI governance challenges, particularly in government programs:\u00a0AI tools\u00a0amplify\u00a0ambiguous input.<\/span>\u00a0<\/span><\/p>\n When a requirement is incomplete, untestable, or vague, running it through an AI workflow\u00a0produces an output that is\u00a0harder to trace back to what was authorized.\u00a0<\/span>\u00a0<\/span><\/p>\n The speed of AI makes this\u00a0worse, because\u00a0teams can move far down a development path before anyone notices the foundational requirement was flawed.<\/span>\u00a0<\/span><\/p>\n Strong governance requires that requirements meet basic quality standards before AI touches them:<\/span>\u00a0<\/span><\/p>\n \u00a0<\/span>Catching quality issues at the point of authoring is far less costly than catching them during an audit, a procurement challenge, or a legislative review.<\/span>\u00a0<\/span><\/p>\n Requirements traceability<\/span><\/a>\u00a0is a basic accountability standard in government programs.\u00a0It’s\u00a0also one of the first things to break down when AI is introduced without governance structure.<\/span>\u00a0<\/span><\/p>\n AI tools produce\u00a0outputs\u00a0quickly. When those outputs\u00a0aren’t\u00a0linked to the requirements that authorized them, the agency has no reliable record of:<\/span>\u00a0<\/span><\/p>\n This is\u00a0a governance architecture problem.\u00a0Traceability\u00a0needs\u00a0to be built into the workflow from the beginning. It is something to be\u00a0not assembled after the fact\u00a0when\u00a0an audit request arrives.<\/span>\u00a0<\/span><\/p>\n When\u00a0traceability is\u00a0part of normal work, agencies get a clear, continuous record.\u00a0<\/span>\u00a0<\/span><\/p>\n They know\u00a0how requirements evolved, what decisions were made, and how every deliverable connects to an approved specification.<\/span>\u00a0<\/span><\/p>\n Requirements\u00a0change,\u00a0that’s\u00a0normal in any government program.\u00a0<\/span>\u00a0<\/span><\/p>\n What’s\u00a0not normal, but increasingly common with AI-assisted workflows, is that downstream work\u00a0doesn’t\u00a0automatically reflect those changes, and no one knows it.<\/span>\u00a0<\/span><\/p>\n When a requirement is updated in a well-governed system,\u00a0there’s\u00a0a clear signal about what work is now out of alignment.\u00a0<\/span>\u00a0<\/span><\/p>\n When that structure\u00a0doesn’t\u00a0exist, teams continue\u00a0building on\u00a0a requirement\u00a0that’s\u00a0no longer current. They may deliver a product that\u00a0doesn’t\u00a0match what was authorized, without knowing it until a review surfaces the gap.<\/span>\u00a0<\/span><\/p>\n This is particularly\u00a0at\u00a0risk in AI workflows because outputs are produced faster and in higher volume. A single upstream requirement change can affect a large body of downstream work before anyone catches it.<\/span>\u00a0<\/span><\/p>\n Governance needs to include\u00a0<\/span>change visibility<\/span><\/a>,\u00a0a mechanism that surfaces\u00a0what’s\u00a0affected when a requirement changes, so program managers can make informed decisions rather than discover problems at the worst possible moment.<\/span>\u00a0<\/span><\/p>\n Most government programs still build compliance documentation at the end of a project cycle.\u00a0This was always imperfect, but with AI-assisted work, it becomes untenable.\u00a0AI tools can generate large volumes of work in\u00a0a short time. Reconstructing the decision trail for all of it is time-consuming, often incomplete, and\u00a0frequently\u00a0inaccurate.\u00a0<\/span>\u00a0<\/span><\/p>\n The solution is to shift documentation from a closing task to a byproduct of normal work. When links between requirements and outputs are created at the time work is done, and every action connected to a requirement is logged in a\u00a0<\/span>version-controlled record<\/span><\/a>, agencies\u00a0don’t\u00a0face a documentation gap. They have\u00a0an accurate, continuous audit trail.<\/span>\u00a0<\/span><\/p>\n Not every AI use case carries the same risk. An AI tool used internally to summarize meeting notes is\u00a0very different\u00a0from one used to help evaluate procurement bids, score program applications, or develop regulatory guidance.<\/span>\u00a0<\/span><\/p>\n One of the most common AI governance challenges is applying the same governance requirements to every use\u00a0case, or\u00a0applying no requirements at all. Both are failures.<\/span>\u00a0<\/span><\/p>\n When controls are too light for\u00a0high-stakes\u00a0uses, the agency is exposed. When controls are too heavy for\u00a0low-stakes\u00a0uses, teams work around them, and the governance process loses credibility.<\/span>\u00a0<\/span><\/p>\n Effective governance applies controls that match each use case’s actual risk profile, factoring in:<\/span>\u00a0<\/span><\/p>\n Matching controls to use-case risk is harder than applying a blanket policy, but it produces governance that people follow.<\/span>\u00a0<\/span><\/p>\n This challenge tends to grow as AI\u00a0usage\u00a0expands.\u00a0Tools proliferate, with some\u00a0approved and others adopted informally.\u00a0Without a clear view of where AI is being used,\u00a0it’s\u00a0nearly impossible\u00a0to\u00a0identify\u00a0higher-risk activity.\u00a0This visibility gap creates a compounding problem: the more AI is used, the harder it becomes to govern.<\/span>\u00a0<\/span><\/p>\n A practical approach is\u00a0maintaining\u00a0an active inventory,\u00a0covering approved\u00a0systems,\u00a0AI features embedded in existing tools, and known informal uses. This\u00a0becomes the foundation for prioritizing governance resources and\u00a0identifying\u00a0where controls need to be strengthened.<\/span>\u00a0<\/span><\/p>\n The challenges of governing AI at scale affect all types of organizations. But state\u00a0<\/span>government agencies<\/span><\/a>\u00a0face a specific combination of pressures that makes these challenges more acute.<\/span>\u00a0<\/span><\/p>\n Legislative oversight, inspector general reviews, procurement audits, and public records requests all require a clear, traceable record of what was decided, why, and how it connected to authorized requirements.\u00a0<\/span>\u00a0<\/span><\/p>\n In private organizations, gaps in that record can be managed internally. In government, they become public problems.<\/span>\u00a0<\/span><\/p>\n When an AI-assisted decision affects a contract award, a\u00a0program\u00a0eligibility determination, or a regulatory outcome, agencies need to explain how that decision was made.\u00a0<\/span>\u00a0<\/span><\/p>\n “The AI\u00a0tool\u00a0recommended it” is not an acceptable answer in any oversight context.\u00a0<\/span>\u00a0<\/span><\/p>\n The governance structure needs to produce an explanation that survives scrutiny.<\/span>\u00a0<\/span><\/p>\n In government programs, what was authorized matters as much as what was delivered. Requirements\u00a0serve the purpose of\u00a0internal planning. They are also\u00a0the\u00a0basis for contract terms, compliance reviews, and procurement challenges.\u00a0<\/span>\u00a0<\/span><\/p>\n Weak\u00a0requirements\u00a0quality and\u00a0<\/span>poor traceability<\/span><\/a>\u00a0open the door to\u00a0operational risk\u00a0and\u00a0legal exposure.<\/span>The Top Challenges of Governing AI at Scale for State Agencies<\/span>\u00a0<\/span><\/h2>\n
1. Unclear Ownership Across Teams<\/span><\/h3>\n
\n
\n
2. Weak Requirements Quality Before AI Is Applied<\/span><\/h3>\n
\n
3. Limited Traceability Between Outputs and Approved Requirements<\/span><\/h3>\n
\n
4. Invisible Downstream Impacts When Requirements Change<\/span><\/h3>\n
\u00a05. <\/span>After-the-Fact Documentation<\/span><\/h3>\n
\u00a06. <\/span>Controls That Don’t Match the Risk Level of Each Use Case<\/span><\/h3>\n
\n
7. Low Visibility into How AI Is Being Used<\/span>\u00a0<\/span><\/h3>\n
\nRELATED: <\/strong>Accelerate AI-Driven Development with Jama Connect MCP\u2122<\/a><\/span><\/span><\/h4>\n
\nWhy These\u00a0AI Governance\u00a0Challenges\u00a0Are Especially Critical for\u00a0State Agencies<\/span>\u00a0<\/span><\/h2>\n
Non-Negotiable\u00a0Accountability\u00a0Standards<\/span>\u00a0<\/span><\/h3>\n
Explainability\u00a0Isn’t\u00a0Optional<\/span>\u00a0<\/span><\/h3>\n
Requirements\u00a0Documentation\u00a0Carries\u00a0Legal\u00a0Weight<\/span>\u00a0<\/span><\/h3>\n