“On the Mars Climate Orbiter, the prime contractor used English units for the satellite thrusters while the operator used SI units for the model of the thrusters. Therefore, there was a mismatch between the space-based satellite and the ground-based model. Because the solar arrays were asymmetric, the thrusters had to fire often, thereby accumulating error between the satellite and the model. This caused the calculated orbit altitude at Mars to be wrong… Instead of orbiting, it entered the atmosphere and burned up. But we do not know for sure, because (to save money) tracking data were not collected and fed back to earth. Giving the units of measurement is a fundamental part of requirements development. And they did not state the measurement units correctly: so this was a requirements-development mistake. There was a mismatch between the space-based satellite and the ground-based model: This is a validation error. They did not do the tests that would have revealed this mistake, which is a verification error.”
A. Terry Bahill and Steven J. Henderson, “Requirements Development, Verification and Validation Exhibited in Famous Failures”
“When Boeing assembles a 747, all the sub-systems and components have been independently verified and validated. You wouldn’t run the engines for the first time on the test flight.”
EOLA Systems, “Basic Principles of Systems Engineering”
You wouldn’t excavate a quarry with a plastic beach shovel and bucket. Yet, many companies — mining collective intelligence in order to build products, versions and variants — are stuck using inadequate tools that make complex jobs more complicated. And for aerospace and defense, automotive, medical device and semiconductor teams that must demonstrate regulatory compliance, the work itself is complicated enough.
Being able to trace relationships between data types is, with the right tool, an efficient way to uncomplicate a very complex task. Tracing your verification and validation reduces the risk of error and failure in the following three ways:
- Connect test cases from problem statements to your requirements and design. If you can’t do this, you can’t be sure you haven’t overlooked something critical. Anything you miss at any stage can, and usually will, result in revisions that cost you time and money.
- Connect system requirements to business/stakeholder requirements. Same as above: Miss this connection and you’ll risk incurring unplanned expenses that can ultimately affect the launch date, stakeholders’ confidence and the bottom line — all three if the changes affect hardware.
- Improve decomposition. To make sure that components and sub-components all come together to make a useful, functional system, you need to relate the lower-level requirements to the higher-level requirements. Make a mistake here, and you’ll likely deal with delays as you scramble to put to put the pieces back together and make late-stage changes.
While disorderly verification and validation processes increase risks, it’s the weak requirements management tools engineering teams typically inherit that pose the greatest threats. And the larger and more complex the organizational structure, the greater the threat. Poor requirements management is one of the key reasons large companies struggle to establish business and engineering alignment and efficient collaboration.
Don’t take our word for it. Leading technology analysts have crunched the numbers to explain why embedded systems teams must adopt modern requirements management or get left behind. We’ve packaged their troubling data in a delightful infographic.