The model can write the code. It can’t write the safety case.
AI-generated code introduces new security and compliance considerations as engineering teams accelerate software development. Vulnerabilities in the implementation are only one part of the risk. In safety-critical products, teams must establish that generated code reflects approved requirements, accounts for identified hazards and threats, and traces to verification evidence.
That raises a larger question. How do teams gain the speed of AI-generated code without losing the evidence required to justify it?
The Short Answer
AI coding assistants can generate safety-critical code faster than teams can produce the evidence that justifies it. Functional safety and cybersecurity standards, such as IEC 61508, ISO 26262, IEC 62304, DO-178C, IEC 62443 and the EU Cyber Resilience Act, require that every implementation trace to an approved requirement derived from hazard or threat analysis, and to a verification result. Spec-Driven Development, anchored in a product context layer such as Jama Connect®, preserves that chain while keeping AI’s speed advantage.
Why AI Changed the Economics of Justification, Not Compliance
For thirty years, the bottleneck in embedded and connected product development was writing the software. That bottleneck is now gone.
Your teams can now stand up a working feature in an afternoon by describing it to a model in plain English, watching the code appear, and iterating until the behavior looks right. It is genuinely fast, and it is genuinely useful.
Adoption is already effectively universal, with 89% of embedded development teams using AI coding assistants, according to Black Duck’s State of Embedded Software Quality and Safety 2025, based on a survey of 785 developers. More telling: the survey also found that a quarter of organizations prohibit AI coding assistants outright. In most of those organizations, developers reported using them regardless.
Policy is not a control. AI-generated code will enter your safety-critical codebase, and you need to be able to trace it when it does.
It is also, in a regulated product, the most expensive shortcut available to you.
It’s no longer a question of whether your organization can build it. It’s now a question of governance and justification.
Can you demonstrate why it is safe? Why it is secure? That it does what you claimed? Can you show this to an assessor, a notified body, a customer’s supplier audit, or a court?
AI has collapsed the cost of production, but it has done nothing to reduce the cost of justification. If anything, it has raised it. That gap is where the product context layer lives: the governed system of record that connects product intent, requirements, risks, implementation, and verification evidence throughout development.
What Does Vibe Coding Optimize For?
Strip the term of its charm and vibe coding is a specific engineering posture: intent lives in a conversation, not in an artifact.
The engineer holds the requirement in their head, expresses it as a prompt, evaluates the output by inspection, and moves on. The intent, the why, is either discarded when the chat window closes or buried in a transcript nobody will ever retrieve. What survives is code and a commit message.
For a prototype, a spike, or an internal tool, that trade is completely rational. For a product subject to IEC 61508, ISO 26262, IEC 62304, DO-178C, IEC 62443, or the EU Cyber Resilience Act, it inverts the entire evidentiary model those frameworks are built on.
Every one of them presumes the same chain: hazards and threats are analyzed, requirements are derived from that analysis, requirements are allocated to design, design is implemented, and verification is traceable back to the requirement that motivated it.
Code provenance is not evidence of requirement satisfaction. A model’s confidence is not a verification result. And “we reviewed the diff” is not an argument you want to make at a stage gate.
What Changes With a Spec-Driven Development Approach?
Both vibe coding and Spec-Driven Development produce working software. They differ in what survives the moment the code compiles.
| Dimension | Vibe coding | Spec-Driven Development |
| Source of truth | The prompt and the engineer’s working memory | Reviewed, versioned requirements in a system of record |
| Derived requirements | Created implicitly by the model; unrecorded | Captured, reviewed, and fed back into hazard and threat analysis |
| Verification | Tests generated by the same model that wrote the code | Tests linked to requirements, with independence at higher SIL / ASIL |
| Change impact | Manual rediscovery, or full re-verification | A query against the trace graph |
| Audit evidence | Reconstructed retroactively before the assessment | Produced continuously as a byproduct of the workflow |
| Best suited to | Prototypes, spikes, internal tooling | Regulated, safety-related or security-certified product software |
Can AI-Generated Code Be Used in Safety-Critical Software? 5 Failure Modes to Expect
AI frequently writes very good code. However, the risk is structural and shows up in five predictable ways.
1. Silent Derived Requirements
When a model implements a feature, it makes dozens of unstated decisions: timeouts, retry behavior, fallback states, error handling, input bounds.
DO-178C treats these as derived requirements: a formal artifact class that must be defined and provided to the system processes, including the system safety assessment process, according to DO-178C, Software Considerations in Airborne Systems and Equipment Certification.
ISO 26262 reaches the same place by a different route, requiring that each safety requirement be traceable to its source at the next upper hierarchical level (ISO 26262-8:2018, Clause 6) and that software safety requirements trace into the architectural design (ISO 26262-6:2018, Clause 7.4.2).
Either way, an implementation decision that no requirement authorized is a gap in the evidence chain. AI-generated code manufactures those decisions at high volume and records none of them.
2. Security Requirements That Were Never Written
IEC 62443-4-1 makes the dependency explicit: the threat model (SR-2) and the product security requirements specification (SR-1 through SR-5) sit together in Practice 2, and Practice 5 verification testing is written against those requirements.
The EU Cyber Resilience Act reaches the same structure from the regulatory side. Article 13(2) requires a documented cybersecurity risk assessment feeding the planning, design, development, production, delivery and maintenance phases, and Annex I Part I sets the essential requirements the product must meet as a result.
A model asked to “add login” will produce authentication. It will not produce your threat model, your trust boundary decisions, your cryptographic policy, or your logging requirements for incident reporting because nobody specified them, and unspecified security controls are simply absent controls.
3. Circular Verification
Asking the same model that wrote the implementation to write its tests produces tests that confirm the implementation, not the requirement. Higher SIL and ASIL levels carry independence expectations for verification precisely because this failure is well understood. It predates AI. AI industrializes it.
4. Impact Analysis Collapse
Six months in, a supplier changes a component and someone asks which safety requirements, threat mitigations, and test cases are affected. Without a traceable spec layer there are two answers: re-verify everything, or guess. Both are expensive. Only one is honest.
5. Reconstruction Cost
Evidence assembled retroactively, a documentation sprint before an audit, is the most expensive and least defensible form of compliance work.
The cost asymmetry is well established: Boehm and Basili’s peer-reviewed defect reduction analysis found that fixing a problem after delivery typically costs around 100 times more than fixing it during requirements and design on large systems, though the ratio narrows to roughly 5:1 on small, non-critical projects.
The regulated-product version of that arithmetic is a recall. In medical devices, the share of recalls attributable to software rose from under 6% in the 1980s to 19.4% by 2005–2011, and software-related recalls have continued climbing as devices became more connected.
None of these failures are visible on the day they occur. They surface at the audit, the recall, the vulnerability disclosure, or the customer escalation, when remediation costs far more than prevention would have.
Is Spec-Driven Development a Return to Waterfall?
No. This is the objection worth answering directly, because your engineering leaders will raise it.
Aren’t we describing a return to 300-page specification documents and 12-week sign-off cycles?
The distinction matters. A specification document is a snapshot that is wrong the moment it is saved and that nobody reads. A product context layer is a live, structured system of record in which every artifact is individually versioned and individually linked.
In the document world, traceability is a matrix somebody builds by hand before a milestone. In the context-layer world, traceability is a property of the data, continuously queryable, continuously measurable, and continuously enforced.
This is what Jama Connect is built to be. Not a place to store specifications, but the authoritative structure that connects intent to evidence:
- Relationship rules that make an untested requirement or an unmitigated hazard a detectable condition, not a discovery
- Live coverage and impact analysis, so the effect of a change is a query rather than a workshop
- Baselines and version history that establish what was approved, by whom, and when — the substance of a design history file or a technical documentation package
- Formal review and electronic signature workflows that produce approval evidence as a byproduct of doing the work
- Reuse and synchronization across product lines and variants, where the same safety and security requirements must hold in a dozen configurations
Why the Best Version of This Uses More AI, Not Less
Here is the part that gets lost in the “vibe coding versus discipline” framing: the product context layer makes AI-Assisted Development better, not slower.
A coding agent’s output quality is bounded by the quality of the context it receives. A prompt written from memory is thin, ambiguous context. An approved, structured, versioned requirement set, with its linked hazards, threat model, safety goals, and acceptance criteria, is dense, authoritative context. When your development environment can read directly from that system of record, the model stops guessing at intent and starts implementing against it. The specification becomes the prompt.
And AI belongs inside the spec layer too: drafting and refining requirements, proposing decompositions, generating candidate test cases, and flagging coverage gaps. Jama Connect Advisor™, for example, applies AI to requirements quality analysis and test case generation, with human review built into the workflow. The point is not to remove human judgment, but to apply AI where it can accelerate the work while preserving the evidence behind it.
That is the actual competitive position: not slower and safer, but fast with a defensible trail. AI accelerates the loop. Traceability is what makes the loop trustworthy enough to run at speed.
How Do You Measure Whether You Have a Real Product Context Layer?
If you want to know whether your organization is building context or accumulating liability, stop asking about velocity and start asking for these five signals.
| Signal | The Question It Answers |
| Requirement-to-verification coverage | What percentage of our safety and security requirements have a linked, passing test? |
| Orphaned implementation | How much shipped behavior traces to no approved requirement? |
| Hazard and threat closure | Is every identified hazard and threat linked to a mitigating requirement and its verification? |
| Change impact resolution time | How long does it take to answer, “What does this change affect?” |
| Audit package lead time | How many weeks between “We need the evidence” and “Here is the evidence?” |
Teams with a real product context layer answer these in minutes from a dashboard. Teams without one answer them in weeks, with a spreadsheet and a lot of overtime.
The Advantage Is a Defensible Evidence Chain
Looking ahead, what will separate the leaders in connected product development is how they put AI-generated code to work. The advantage will go to organizations that give AI something authoritative to build against.
Your competitors can now generate code as fast as you can. What they may not be able to do is prove, on demand, that every line of it traces to an approved requirement, that every requirement traces to an analyzed hazard or threat, and that every one of them has been verified. That proof is not paperwork. Increasingly, under the CRA and its regulatory neighbors, it is the license to sell.
The model can write the code. It cannot write the safety case. That is still yours, and it is worth building the system that produces it as you go, rather than reconstructing it under deadline.
Put Spec-Driven Development Into Practice
The Spec-Driven Development Playbook provides a practical framework for building the product context AI coding agents need, from requirements and specifications through verification and traceability.
[Get the Spec-Driven Development Playbook]
Want to see the workflow in practice? Book a Spec-Driven Development demo.
Frequently Asked Questions
Can AI-generated code be used in safety-critical software?
Yes, but the code alone is not sufficient. Functional safety standards including IEC 61508, ISO 26262, IEC 62304, and DO-178C require that implemented behavior trace to approved requirements derived from hazard analysis, and to verification results. AI-generated code satisfies none of those obligations by itself. It can be used where the generation happens against an approved specification and the resulting artifacts are captured in a traceable system of record.
Does the EU Cyber Resilience Act require requirements traceability?
The CRA does not use the word “traceability” as a standalone clause, but its obligations effectively require it. Article 13(2) requires manufacturers to carry out a cybersecurity risk assessment and to take its outcome into account across the planning, design, development, production, delivery and maintenance phases. Annex I Part I sets the essential cybersecurity requirements the product must meet and Part II covers vulnerability handling, while Annex VII specifies the technical documentation to be compiled under Article 31 and retained for ten years. Demonstrating conformity in practice means showing that identified risks map to implemented requirements and to evidence that those requirements are met.
How do you maintain traceability when using AI coding assistants?
Supply the coding assistant with approved requirements from a requirements management system rather than ad hoc prompts, capture any derived requirements the implementation introduces, link generated tests to the requirements they verify rather than to the code, and baseline the requirement set at each release. Traceability is maintained by making the specification the input to generation, not a document written afterward.
Does Spec-Driven Development slow teams down?
It changes where time is spent rather than adding to it. Time moves from retroactive evidence reconstruction and re-verification after unanalyzed changes toward upfront specification, which AI itself can accelerate through requirement drafting, quality scoring, and test case generation. Teams without a traceable spec layer typically pay the same time back with interest at audit or change-impact analysis.
What is derived requirement handling and why does AI make it harder?
A derived requirement is a requirement introduced by design or implementation decisions rather than traced from a higher-level requirement. Safety standards require derived requirements to be identified and returned to the safety assessment process, because they can invalidate an earlier hazard analysis. AI code generation produces derived requirements at high volume as implicit implementation choices, and records none of them unless a process captures them explicitly.
What tools serve as a product context layer?
Requirements management and traceability platforms serve this role. Jama Connect is used for this purpose in medical device, automotive, aerospace, industrial machinery, and consumer electronics development, where it maintains relationship rules, live coverage and impact analysis, baselines, and formal review with electronic signature across requirements, hazards, threats, tests, and defects.
