AI for Systems Engineering: Benefits, Risks, and Adoption

Chapters

Chapter 15: AI for Systems Engineering: Benefits, Risks, and Adoption

Chapters

AI for Systems Engineering: Benefits, Risks, and Adoption

Systems engineers are being asked to adopt artificial intelligence (AI) tools whose behavior on their own artifacts has rarely been measured. Research on AI for systems engineering “remains empirically nascent,” according to a June 2026 review of the field, which cites a 2025 systematic review by Poulsen et al. that identified 284 papers at the AI and systems engineering intersection and selected 33 for in-depth review.

The measurements that do exist point one way. Large language models (LLMs) perform better on requirements, traceability, and model-generation tasks when the structured context they read from is clean, and generally worse when it is ambiguous. In systems engineering that context is governed requirements, architecture models, tests, and the relationships between them. Adopting AI responsibly means putting that context in front of the model and keeping every output auditable back to the input that produced it. This guide covers where AI output already holds up, the risks and certification positions regulated programs have to plan around, and a six-step adoption sequence that puts quality review ahead of any generation.

Where AI for Systems Engineering Already Delivers Benefits

AI for systems engineering is the use of LLMs and related machine learning tools to review, trace, test and model the artifacts a systems engineering team governs. Its most usable results so far are on requirements, because an engineer can check an AI flag or draft directly against the source text. Model generation and change impact analysis trail behind, with real but bounded gains.

Requirements Quality Review Against INCOSE and EARS Rules

LLMs can classify requirements against International Council on Systems Engineering (INCOSE) “good requirement” criteria with near expert-level agreement on routine checks, particularly for syntactic and structural attributes. The study behind that result compared the tested models with professional survey judgments. Its more useful finding is where the two disagreed, according to the preprint as posted to arXiv on 16 April 2026. Models and engineers usually agreed on which requirements needed revision but differed on the underlying cause, with the models flagging formal rule violations while the engineers prioritized ambiguity, missing context, and conceptual intent. Because of that disagreement, the authors propose AI as a “standardized pre-audit layer” that keeps expert authority over validation.

The INCOSE Guide to Writing Requirements V4 sets out 42 rules, and holding all of them in mind while drafting one requirement is difficult even for experienced authors. Automated checkers can score against INCOSE rules and Easy Approach to Requirements Syntax (EARS) patterns, and natural language processing can check those rules at least partially. Two properties, ambiguity and verifiability, deserve particular attention because formal rule checking cannot replace an engineer’s understanding of missing context and conceptual intent. A team piloting quality scoring should measure the false-positive rate on its own requirement set. The comparison cited here rested on 20 requirements rated by 21 practicing engineers, and those requirements were chosen where the models disagreed.

Traceability Suggestions, Change Impact, and Test Case Drafting

LLMs can suggest trace links and change impacts and draft test cases, but engineers must review each output before it enters the baseline. Combining Model-Based Systems Engineering (MBSE) with LLMs can produce suggested links for that review.

For change impact analysis, an LLM can propose possible impacts of requirement changes before implementation. GPT-4 has been tested on predicting software requirement change impacts, so the evidence so far comes from software projects rather than multidisciplinary systems. The output supports the analysis a systems engineer otherwise runs by walking the trace chain by hand. Each proposal still gets checked by an engineer against the chain the model can read.

Test case drafting is the most reviewable of the three because an engineer can judge each generated case against its requirement in minutes. The model needs a well-formed requirement, and engineers then read every draft and keep only the cases that hold up. An ambiguous requirement produces ambiguous test cases, so the quality review above comes first.

Model Generation for MBSE Remains a Skeleton-Building Aid

From natural language, LLMs can draft Systems Modeling Language (SysML) v2 skeletons, but they can’t yet be trusted to produce a semantically correct model. Available training material for SysML v2 remains limited. Direct prompting often produces structurally plausible outputs that fail syntactic validation, including hallucinated SysML v1 syntax and nonexistent v2 keywords, according to work by Cibrián et al. that a 2026 INCOSE symposium paper reviews.

The harder failure passes the parser, because a model can be syntactically valid while its mechanical, electrical, fluid, or signal connections are physically meaningless, and those domain errors can remain undetected by grammar checks alone. LLMs can produce early model prototypes for engineers to refine, but they don’t support autonomous end-to-end model generation. The failure mode has no equivalent in requirements text work, which is why MBSE teams should treat model generation as the last step in the adoption sequence.

How Ambiguity and Retrieval Affect AI Output Quality

How much ambiguity is left in the source requirements and whether the model reads approved records or a pasted copy decide the output quality a pilot will see. Both can be measured on a team’s own artifacts before the pilot starts.

Ambiguity in the Requirement Degrades Every Downstream Task

An ambiguous requirement degrades the trace links, the test cases, the impact analysis, and any generated code that reads from it, because each task inherits the same unresolved meaning. When a requirement contains unresolved inconsistencies or ambiguities, teams should not assume that an LLM can reconstruct the intended meaning during tracing. Trace-link performance was generally worse for requirements with those defects in an experiment that traced requirements to code.

Retrieval From Governed Sources Beats Generic Prompting

Retrieval-augmented generation (RAG) beats generic prompting because it gives the model approved program data and engineering guidance to work from. Retrieval reduces uncertainty, but hallucinations remain, so output still requires review.

RAG inputs for systems engineering can include the requirements baseline with linked architecture models and verification records. Requirements locked in documents are harder for AI tools to query. Traceability links between requirements, design elements, and verification activities have to be machine-readable as well, or the model ends up reconstructing them from prose. Programs should not assume that a base LLM can read program-specific, controlled specifications unless those specifications are deliberately provided through an approved workflow.

One governed source, holding one authoritative version of each requirement and a complete trace record, can reduce rework and review time before any AI reads it. Regent reported completing a design review in three weeks, against two months without Jama Connect. An LLM needs the same two properties from the context it reads.

A Standard Protocol for How Agents Read Engineering Data

One delivery mechanism for governed context is the Model Context Protocol (MCP). MCP gives software tools a common way to expose approved context to AI agents while allowing engineering teams to put access controls around that exchange. Requirements management tool vendors can expose requirements and traceability data through MCP servers. An agent working in a development environment or a chat client can then read an approved requirement without relying on a pasted copy. The protocol leaves requirement quality unchanged, and it turns what the agent reads into a data-governance decision, where systems engineering already has controls.

Read About Spec-Driven Development

The Risks Regulated Programs Have to Manage

Four risks come with putting AI output into a regulated evidence chain, and each needs a named control before the pilot starts. The controls exist to keep engineering judgment independent and every output traceable to its governed input.

Hallucinated Content in Requirements Work

Requirements workflows need testing for hallucinated outputs that conflict with governed engineering context or established facts, because precision and traceability in requirements feed directly into downstream system design. The hallucination rate depends on the workflow a team builds around the model. Refining prompts and workflows reduced major errors below previously reported human note-taking rates in a clinical note-summarization study. One domain’s results don’t transfer directly to engineering documentation, so teams need to measure performance by task and workflow on their own artifacts.

AI Output With No Auditable Provenance

An AI-generated artifact can enter the evidence chain with no record of what generated it, from which input, or who approved it. When AI behavior varies between runs, confidence scoring and hallucination filtering count only as partial controls for auditability. For regulated engineering, the audit trail should record that AI was used, what it proposed, which governed input it read and which qualified engineer approved the result.

Export-Controlled and Proprietary Data Leaving the Program

Public generative AI tools sit outside the program boundary until they’re explicitly approved for the relevant data. For Aerospace and Defense programs the exposure may be legal as well as commercial. Export-controlled technical data may be exposed through access even when no file is intentionally transferred, so access controls and service configuration need a review before anyone uses the tool. Uploading controlled technology to an AI service that has not been approved for it can create an export-compliance problem that is hard to remedy afterwards.

Automation Bias in the Review Step

Automation bias is the pull toward accepting a recommendation because a machine produced it, and a review step that approves every suggestion has become a rubber stamp. Reviewers must actively search for contradictions and independently assess each computer-generated recommendation. A human-in-the-loop control must stop a reviewer from incorporating a biased or incorrect recommendation without independent judgment. The AI acceptance rate, one of the expansion metrics below, shows when that control has started to fail.

What Certification and Regulatory Bodies Currently Accept

Safety and quality frameworks should not be treated as providing a general shortcut that turns AI-generated artifacts into standalone certification evidence. Guidance for generative AI is still developing, and the documents a systems engineer encounters differ by sector and by whether AI is inside the product or used as a development tool. Program teams should verify the current edition and status with the issuing body before making a certification decision.

Document or framework family Issuing body Why teams track it
AI Concept Paper, Issue 2 (March 2024) European Union Aviation Safety Agency (EASA) Aviation guidance for AI assurance and human oversight
Notice of Proposed Amendment 2025-07, Rulemaking Task 0742 EASA Aviation proposal on AI trustworthiness, consultation closed 10 March 2026
Roadmap for Artificial Intelligence Safety Assurance, Version I Federal Aviation Administration Federal aviation direction for AI safety assurance
DO-178C / ED-12C, DO-254 / ED-80 and DO-330 / ED-215 Radio Technical Commission for Aeronautics / European Organisation for Civil Aviation Equipment (EUROCAE) Airborne software assurance, airborne hardware assurance and development-tool qualification
Aerospace Recommended Practice (ARP) 6983 / ED-324 (draft) SAE International G-34 / EUROCAE WG-114 Emerging assurance guidance for aeronautical systems using machine learning, targeted for publication by 31 December 2026
International Organization for Standardization Publicly Available Specification (ISO/PAS) 8800:2024 ISO Technical Committee 22, Subcommittee 32 Road-vehicle safety and AI systems
ISO / International Electrotechnical Commission (IEC) 42001:2023 ISO / IEC AI management-system governance
Artificial Intelligence-Enabled Device Software Functions, draft guidance (January 2025) U.S. Food and Drug Administration Lifecycle and submission considerations for medical devices that use AI

AI as a Development Tool Under DO-178C and DO-330

A DO-178C program should distinguish between tool output that receives independent verification and tool output used to replace or eliminate a required process. An AI tool becomes more consequential when its output enters airborne software directly. Reliance on the output as verification evidence without an independent check also increases its consequence.

AI output that engineers verify independently of the tool can stay out of tool-qualification scope. RTCA publishes FAS topic paper FTP 1055 for informational purposes. It reads DO-178C to mean that a tool able to introduce an error needs no qualification when its outputs, such as requirements or source code, are verified independently of the tool. AI-generated code needs the same rigor as code an engineer writes by hand. Like any other code, it has to trace to an approved requirement, meet the project’s coding standard, be understood by the engineers who own it and pass deterministic verification. Engineers spend more of their time reviewing and editing, and they stay accountable for what ships.

Automotive and Medical Device Positions

Automotive and medical device teams should extend their existing safety and lifecycle processes to address AI-specific risks in development workflows. For automotive teams, ISO/PAS 8800:2024 extends ISO 26262 (road-vehicle functional safety) and ISO 21448 (safety of the intended functionality) to AI elements inside the vehicle, and it describes safety-related AI properties such as bias and robustness. Its ISO catalog abstract states that it gives no specific guidelines for software tools that use AI methods, so an AI tool in the development workflow is handled through the ISO 26262 process a team already runs. Medical device teams should evaluate AI inside the device against the IEC 62304 software lifecycle and ISO 14971 risk management. They also answer to the design and development requirements of ISO 13485 Clause 7.3, which 21 CFR 820.10(c) incorporates.

When AI is used as a development tool, teams keep applying the design and development, verification, validation, configuration management and traceability requirements that govern the underlying work. Across sectors, those expectations come down to three practices:

  • Advisory role: AI output feeds engineering decisions as advisory input.
  • Qualified review: A qualified human reviewer must evaluate, change, or reject every AI-generated recommendation before approval.
  • Artifact provenance: Provenance records document the model version, prompt, governed input, and approval decision for each artifact.

Placing model versions and prompts under configuration control lets a team reproduce the conditions under which an AI-assisted artifact was created. A team adopting AI is operating while sector-specific regulation is still evolving.

AI in Product Development

How to Sequence AI Adoption on a Systems Engineering Team

Adoption runs in six steps that start with putting requirements into structured, machine-readable form and end with generating new requirements and models. Training on how to read AI output and a written accountability policy run alongside the whole sequence, and four metrics decide when a pilot expands.

Pilot Sequence and Expansion Gates

Steps 1 to 3 are preconditions a team meets before the pilot’s first run, step 4 is the pilot, and steps 5 and 6 set the review control and how far generation goes:

  1. Structured data foundation: Requirements move out of documents into structured, machine-readable items, and their trace links to design elements and verification activities become machine-readable too. Historical requirements and regulatory documents form a curated corpus that gives AI tools retrieval context.
  2. Data access governance before any AI tool reads program data: Data access rules define which classes of data may go to which AI tools, with export-controlled programs handled first. Pre-deployment controls should prevent proprietary or controlled information from entering an unapproved service.
  3. Pre-AI measurements: Baseline measurements cover defect rates, traceability coverage, and cycle times before the pilot starts. Without those figures there is no way to tell whether AI is delivering measurable improvement or introducing new failure modes.
  4. Quality review before generation: The pilot begins with requirements quality scoring on one subsystem inside the existing authoring environment. Checking existing requirements carries lower risk than generating new ones.
  5. Mandatory human review from day one: Reviewers can see which artifacts AI produced and can change or reject each one before approval, from the first day of the pilot.
  6. Generation and modeling last: Expansion adds traceability suggestions after quality review, then test case drafting on a well-defined requirement set, then requirement and model generation. Governance controls require validation at each level before the next.

Metrics for Deciding Whether to Expand

Four metrics separate a pilot that produced evidence from one that produced anecdotes. Three of them compare against the step-3 baselines, and the acceptance rate starts counting with the pilot.

Metric What it measures What it tells you
Requirements defect rate Ambiguity and incompleteness flags per requirement set, before and after AI quality scoring Whether AI review changed authoring quality
Traceability coverage Percentage of requirements with verified downstream links to design elements and test cases Whether the structured foundation is complete enough for AI to read
Cycle time per requirement Elapsed time from initial elicitation to baselined requirement Whether review effort fell or moved elsewhere
AI acceptance rate Percentage of AI suggestions engineers accept without modification Model quality and reviewer trust, and a check on automation bias

Expansion from quality review into generation then depends on two operational gates, and a team checks both again for every new use case:

  • Source and record readiness: The governed-context and auditability controls established for the pilot must remain in force as the scope expands.
  • Review capacity: A qualified engineer must have enough time and context to review every generated artifact independently.

A failed gate identifies where governance or review capacity must improve before the scope grows. Both gates read from the same step-3 measurements, so a team can tell whether the source record or the reviewers’ time is the constraint before more generated artifacts reach the baseline.

How Jama Connect Supports AI for Systems Engineering

Jama Connect®, a web-based requirements management and traceability platform for complex, regulated product development, provides the Product Context Layer for engineering teams through the Traceability Information Model (TIM). The TIM defines how requirements, models, code, tests, simulations, and verification results relate to one another, which covers the relationships step 1 asks for, and it marks any requirement whose required downstream item does not exist yet. For step 2, the Jama Connect MCP Server, released in May 2026 with Jama Connect 9.35 as an add-on for both Cloud and Self-Hosted deployments, exposes Jama Connect’s structured requirements and traceability data to external AI agents and LLM-powered tools. It enforces the permissions, lifecycle workflows and audit requirements already set in Jama Connect. Approving which outside AI services may receive export-controlled data stays a program decision. Because the TIM defines those relationships, the traceability coverage baseline in step 3 has a defined model to measure against, and Live Trace Explorer™ reports that coverage as the percentage of linkage between artifact types. Getting requirements into that governed record pays off before any AI is involved, and Dexcom reported a 60% productivity boost for its systems engineering team with Jama Connect. For step 4, Jama Intelligence scores each requirement against 30 of the 40 INCOSE rules and six EARS patterns inside the authoring view, suggests revisions the engineer accepts, edits, or rejects, and reports scores the team can trend against the step-3 defect baseline.

Step 6 starts with traceability suggestions, which relationship discovery in Jama Intelligence provides. The engineer defines which relationships to look for, and it suggests trace links between a source item and related items in the product library. In keeping with step 5, nothing is linked until the engineer accepts a suggestion. Jama Intelligence is an add-on for Jama Connect Cloud. For test case drafting, the next stage of step 6, Jama Intelligence turns a requirement into draft test cases for the engineer to review, and the trace link to the source requirement is created only when the engineer accepts a case, with the option to regenerate with additional prompting if the first set doesn’t fit. Anything performed with AI inside Jama Connect is versioned and recorded as AI-generated, so the audit trail shows what was generated, when, and against which requirement. AI used outside Jama Connect may leave no equivalent record, which creates compliance risk when AI-generated artifacts are required as part of an audit.

Governed Specifications First, AI Generation Second

Governed specifications come first because both the quality-scoring pilot and every later generation step read from them. After the pilot, the next use case to widen into is the one whose source record and reviewer hours already hold up in the step-3 measurements.

If your team is choosing the subsystem for a quality-scoring pilot, Jama Intelligence supports this workflow by flagging ambiguous requirements while engineers write them. You can book a Jama Connect AI demo to watch it score a subsystem’s requirements and suggest the rewrites an engineer would accept or reject.

Frequently Asked Questions About AI for Systems Engineering

Does the European Union (EU) AI Act apply to AI used as an engineering tool?

Applicability depends on what the tool does, where it is used, its risk classification, and whether it is part of a regulated product or supports product development. Engineering teams should identify and document the obligations for their specific use case and keep watching the guidance for their sector. Aviation, medical device, machinery, and vehicle programs may also need to align the EU AI Act with the product-safety rules they already follow.

How does Design Assurance Level change what AI can do on a DO-178C program?

The higher the Design Assurance Level, the less an unverified AI output can be relied on. Teams on a DO-178C program can assess each proposed use by mapping it to the lifecycle objective it affects and asking whether an independent verification step will reliably detect a wrong output. Any tool output that is not independently verified can increase the tool-qualification burden, while advisory output that receives complete review and deterministic verification is easier to keep within existing development controls. Jama Connect can support this governed workflow by maintaining requirements and traceability within the engineering record.

Can AI replace systems engineers?

AI can’t replace systems engineers, although it can take on first-pass requirements work such as rule checks, draft test cases and suggested trace links. The judgment a certification record depends on stays with engineers, including what a requirement was meant to say, whether a trace is complete and who approves a change. Review becomes a bigger share of the job, so a team planning a pilot should budget reviewer hours as carefully as it budgets the tool.

Can an LLM generate a usable SysML v2 model from a requirements document?

A usable first draft is possible when the team has both a grammar validator and a domain expert assigned before the model enters a governed baseline. Retrieval from curated examples, validation and repair loops can improve generated SysML v2 models, though semantic accuracy still lags syntactic correctness. Without both checks, the output is a disposable prototype.

If I want to learn more about AI for systems engineering, what else should I read?

Beyond systems engineering, AI in requirements management surveys where AI fits across requirements work. The attributes a scoring pilot measures are set out in identifying and measuring requirements quality. The product context layer explains how governed requirements reach AI agents.

This article was authored by Mario Maldari and updated on October 30, 2026.

Book a Demo

See Jama Connect in Action!

Our Jama Connect experts are ready to guide you through a personalized demo, answer your questions, and show you how Jama Connect can help you identify risks, improve cross-team collaboration, and drive faster time to market.