Behavior-Driven Development Won’t Pass an Audit on Its Own

Published:

by

Software team reviewing code on computer screens

When we adopt a behavior-driven development (BDD) framework but write tests during or after coding, we can use the framework without practicing BDD. If we treat Cucumber only as a test runner, we overlook its role in agreeing on behavior before any code exists. We produce feature files that automate something but trace to nothing. On an International Organization for Standardization (ISO) 26262, DO-178C, or DO-254 program, untraced verification isn’t evidence an assessor can use. The same holds under International Electrotechnical Commission (IEC) 62304 and Automotive SPICE, where full BDD starts before coding and Gherkin stays subordinate to the authoritative requirement.

This guide covers where the practice has to start on a regulated program, what Gherkin has no clause for, and the four obligations that still sit beside a passing scenario.

What Is Behavior-Driven Development?

In BDD, business, development and testing agree on concrete examples of how a system should behave. We write those examples in plain language and automate them before production code exists. BDD evolved from test-driven development through early experiments with JBehave. Cucumber’s current guide describes BDD as a way of working that closes the gap between business and technical people, built on three practices: discovery, formulation and automation. The method is “outside-in”: we start with business outcomes and drill down into the features that will achieve them.

BDD begins with discovery. In a Three Amigos session, a product owner or analyst works with development and test representatives to examine examples of a feature until the group shares one understanding. Formulation turns the agreed examples into scenarios written in business terminology, often with testers helping before the product owner approves them. Automation wires each scenario to the code before implementation begins. Automation is also design work because it forces us to imagine the code we need. Following this order lets us use each activity as a readiness gate.

Only after we complete those three activities does the notation itself matter. Gherkin’s Given-When-Then is the format most teams recognize: Given sets the context, When names the trigger, Then states the expected outcome. Once we use that vocabulary, Given-When-Then exposes disagreements a prose requirement can hide, most often about which precondition actually applies. A scenario that runs against the real system is an executable specification: an acceptance test when the feature is done, a regression test afterward.

Why Gherkin Scenarios Are Not Your Requirements

Regulated programs keep authoritative shall-statements alongside Gherkin examples because a scenario illustrates behavior while a requirement generalizes it.

Easy Approach to Requirements Syntax (EARS), first published in 2009 by Alistair Mavin and colleagues, constrains natural language with a fixed clause order. The ruleset has six patterns today, one more than that paper set out. They cover every requirement EARS can express: ubiquitous (no keyword), state-driven (While), event-driven (When), optional feature (Where), unwanted behaviour (If … then), and complex, for any requirement combining more than one keyword. Fixing the clause order cuts ambiguity and sentence length, because the applicable condition and the triggering event both have to be stated alongside the response.

The International Council on Systems Engineering (INCOSE) Guide to Writing Requirements says individual requirements should meet the C3 Unambiguous and C5 Singular quality characteristics; C7 requires them to be verifiable. The guide defines further characteristics for a complete requirement set.

An EARS “While” precondition maps to Given; a “When” trigger with its shall-response maps to When plus Then; “If … then” becomes a When/Then scenario on the failure path. We can use that mapping to derive examples while keeping the shall-statement authoritative. Everything Gherkin has no clause for belongs in the shall-statement instead. That covers performance thresholds such as a 50 ms latency ceiling, hardware constraints such as mass or electrical-consumption limits, safety integrity level assignments, and explicit conformance to a named standard. A requirement such as “The mobile phone shall have a mass of less than XX grams” has no direct Given-When-Then equivalent.

Gherkin scenarios can capture concrete examples for non-functional requirements. A complete requirement set spans performance, constraints, standards, safety, and quality factors.

Build Audit Evidence Beyond BDD

A tagged, executed scenario discharges one audit obligation and leaves four standing. Under DO-178C, every test case derives from a requirement, so a Gherkin scenario linked to its requirement supports that one.

ASPICE 4.0 goes further and requires bidirectional traceability across the V-model, plus consistency between the linked artifacts. A scenario tagged with a requirement ID and executed with a recorded pass/fail result supports the trace half. It does not, on its own, support the consistency half.

ISO 26262 expects a documented chain from safety requirements through design and implementation to verified results; IEC 62304 expects lifecycle traceability through development and verification. Meeting IEC 62304 still requires approved requirements, bidirectional traceability, applicable independence and tool assessment, and retained execution results. U.S. Food and Drug Administration (FDA) design controls for medical devices run through ISO 13485. Under 21 CFR § 820.10(c) and Clause 7.3, the trace matrix can be a form of verification. The matrix linking user needs and design inputs to verification tests is central to design-control audits.

Four obligations beyond requirements-based testing catch BDD teams out, starting with the structural coverage objectives DO-178C defines:

Obligation Why BDD alone does not cover it
Structural coverage (DO-178C) DO-178C ties structural coverage to the assurance level, with modified condition/decision coverage (MC/DC) at Level A, decision coverage at Level B and statement coverage at Level C; scenarios run above the unit level and do not prove any of it on their own.
Independence (DO-178C Design Assurance Level (DAL) A and B) Some review objectives at higher assurance levels require independence; developers writing both scenarios and step definitions do not provide that independence.
Tool qualification A Cucumber or Gherkin runner used for verification credit without independent checking may require assessment under ISO 26262-8 or qualification under DO-330.
Hazard analysis Scenarios illustrate specified behavior and don’t substitute for hazard analysis.

When requirements remain in Word or spreadsheets and feature files in Git, rebuilding the requirements traceability matrix by hand before every assessment makes changes since the last baseline suspect. Keeping the shall-statement, its verifying test case and the recorded result together as the work happens is what removes that rebuild.

Four Failure Modes That Show Up in Multidisciplinary Teams

Product owners and systems engineers can interpret behavior differently from the engineers who develop and test it. The differing interpretations prevent the group from reaching a shared understanding before any technical problem in Cucumber. A 2023 literature review of BDD research separates the two. Its challenges are implementation cost and time, maintaining BDD specifications, and finding a common language. Its limitations are different: the difficulty of writing effective scenarios, the need for a cultural transition, and the absence of any standard to implement against. The same review records better communication, better tracking of requirements, and reuse of artifacts among the benefits. Treating Cucumber primarily as a testing tool leaves our understanding of the problem domain untested.

Four patterns recur on our hardware-software programs, and scenario review is where each one is cheapest to catch:

  • Imperative scenarios: Steps like “click button,” or in a hardware-software system a named internal signal state, tie the scenario to one implementation. Declarative scenarios survive an implementation change intact.
  • Feature-coupled step definitions: When our BDD projects grow quickly and change often, step maintenance becomes a recurring burden. Steps organized by domain concept are more likely to survive refactoring.
  • Nobody owns the behavior: On our large-scale projects, we can struggle with specification and ownership of behaviors. Put the same three roles from the discovery session in a room and split the rules from the examples. Ownership gets settled there, before anyone writes Gherkin.
  • Stale feature files: Feature files that no longer match implemented behavior are a compliance liability, which is why they should remain living documents under version control, with open uncertainties in the front matter. A review should therefore check the implemented behavior, requirement link, and unresolved uncertainty before treating a passing scenario as current evidence.

Two or more of these in the same feature file is the point at which rewriting the scenarios costs less than maintaining them.

Can AI Write the Scenarios for You?

Partly, and only from good requirements. AI can draft scenarios from Jira tickets and reduce the manual effort involved, but a generated set will not reliably cover the unwanted-behavior cases a safety program cannot skip, which is why the negative-path review below is a separate step rather than a spot check. The defensible near-term role for AI-assisted test case generation is therefore human-in-the-loop decision support tied to requirement quality, negative-path review, and traceable acceptance.

Human review is the control for confirming that an LLM-generated scenario reflects its source requirement, and anything generated needs a record of what it came from. EARS-structured, INCOSE-conformant requirements are the prerequisite for usable AI output. During review, we should compare the scenario’s conditions and trigger with the authoritative requirement, then verify that the expected result matches it. A separate check should determine whether unwanted behavior needs its own scenario. We should document a generated draft’s provenance and confirm that its expected result matches the source. Acceptance also requires clear negative-path coverage.

How Jama Connect® Supports Behavior-Driven Development

Jama Connect covers the requirements-to-test trace and the recorded evidence behind it. It does not produce structural coverage, independence, tool qualification or hazard-analysis evidence, and those four are exactly what has to sit beside a passing scenario. Jama Connect Test Management Center stores requirements-based test cases, plans, and runs with pass/fail recorded per step. The per-step record preserves the result of each scenario-derived test. Through its integrations with test automation tools, Test Management Center passes test data automatically between those tools and Jama Connect. The Traceability Information Model (TIM) defines how requirements, tests and results relate, and Jama Connect surfaces the requirements that have no linked test case. That is how we see which requirements have no scenario-derived acceptance test behind them. An upstream requirements change and impact management process flags linked test cases as suspect, showing teams which scenarios require review when a requirement changes.

These capabilities can improve more than trace completeness. SPAN reports that, with Jama Connect, it reduced system validation timelines by up to 25%.

Jama Connect provides the foundation that is the Product Context Layer for systems, test, and quality engineers. The Product Context Layer gives those teams a governed system of record that connects requirements, risks, tests, SysML models, code repositories, simulations, defects, reviews, approvals, verification evidence, and change history. At its center is the TIM, which defines how requirements, models, code, tests, simulations, and verification results relate to one another and preserves the requirement-to-scenario-derived-test-to-result chain.

Upstream of formulation, Jama Intelligence, an add-on for Jama Connect Cloud, scores requirement text against 30 of the 40 INCOSE rules, plus the six EARS syntax patterns, and drafts test cases from a requirement. When an engineer accepts a draft, Jama Connect creates the test case and its trace link back to the source requirement. If the first drafts don’t fit, the engineer can regenerate them with additional prompting. Jama Connect versions anything teams generate with AI inside it and documents it as AI-generated. The resulting record shows what was generated and which accepted requirement produced the artifact. An AI tool working outside Jama Connect leaves no such record, so an artifact it generated cannot be produced as evidence of what was generated, when, or against which requirement.

Starting Behavior-Driven Development on a Regulated Program

Teaching the product owner or systems engineer, developer, and tester to run discovery together comes before any runner license, so nobody approves two readings of the same EARS requirement. The order that holds up under assessment is EARS requirement first, rules-and-examples workshop second, scenario formulation third, automation last. If your scenarios are passing today without producing anything an assessor would accept, you can test that order against your own EARS statements by starting a free 30-day trial.

Frequently Asked Questions About Behavior-Driven Development

Is BDD the same as Specification by Example?

They are closely related names for the same underlying practice: examples agreed collaboratively and then automated to produce living, reliable documentation. We can practice Specification by Example without using Gherkin or a dedicated BDD runner.

What should we confirm before formulation?

Confirm that the requirement states when it applies and what the expected response is, and that any performance, safety, or hardware constraint has stayed in the shall-statement. During scenario review, check that the scenario preserves that meaning and that any unwanted behavior has a scenario of its own.

What should we confirm before claiming verification credit?

Confirm the requirement link, the scenario version, the execution result and the reviewer role. Then confirm the evidence BDD does not produce: structural coverage, tool qualification, independence, and hazard analysis. A passing scenario with none of that behind it is a test result, not verification credit.

Which BDD frameworks are still maintained in 2026?

As of early 2026, Reqnroll is the actively released .NET path for teams moving off SpecFlow, and Cucumber continues to ship releases for the Java Virtual Machine (JVM). Both projects are worth a maintenance check before you commit, because an abandoned runner leaves your verification evidence resting on software nobody is patching.

Can BDD be used for hardware or hardware-in-the-loop testing?

We can adapt BDD for cyber-physical systems, hardware/software co-design, airborne software, and circuit-level verification, including constrained scenarios intended to support formal hardware verification. We can also use keyword-based frameworks that support Given, When, Then, And, and But to bridge scenario language and embedded hardware-in-the-loop (HIL) automation.

You May Also Like

See Jama Connect in Action!

Our Jama Connect experts are ready to guide you through a personalized demo for requirements management for medical device development, answer your questions, and show you how
Jama Connect can help you identify risks, improve cross-team collaboration, and drive faster time to market through AI-native engineering management.