traceability Archives

Tag Archive for: traceability

Live Traceability™ for Airborne Systems Development

Airborne systems are incorporating more cutting-edge technology and becoming more complex with advanced embedded computing technologies, electric propulsion systems, sensors and data, and airframes. A major percentage of this complexity is handled at the software level. Any error in the avionics software of safety-critical airborne electronics could be catastrophic to the aircraft, its occupants, or persons on the ground.

Airborne systems development requires developers to adhere to the most rigorous safety standards in the world. These extremely rigorous, plan-driven, development processes are unimaginable to developers who have not done it before. Certification is expensive. Delays in certification due to lack of evidence of following mandated guidelines could spell the demise of a new startup. Mistakes in design and development could cost lives. Manual documentation or the use of legacy tools introduce risk. Achieving certification for safety-critical airborne software is costly and time consuming. Once certification is achieved, the deployed software cannot be modified without recertification.

Airborne certification bodies such as the Federal Aviation Agency (FAA) and European Aviation Safety Agency (EASA) recognize international standards as “acceptable means, but not the only means, for showing compliance with the applicable airworthiness regulations for the software aspects of airborne systems and equipment certification.” The most common standards that are followed for airborne systems as a means of compliance are: RTC DO-178C (also published in Europe as EUROCAE ED-12C), DO-254 (also published in Europe as EUROCAE ED-80), and SAE 4754A (also published in Europe as EUROCAE ED-79) standards for Airborne systems.

Click here to learn more about Live Traceability™ and get your free Traceability Diagnostic!

Systems Engineers Career Path

Most Systems Engineers we speak with have a common perspective on their role within their organization. Systems engineering as a concept is understood and supported by leadership. And, if systems engineering best practices are followed across all engineering disciplines, leadership also acknowledges the benefits the organization can realize. However, leadership will not force change on engineering disciplines (software, hardware, electrical, risk, verification, and validation) to remove barriers to systems engineering best practice adoption. This leaves most Systems Engineers in the challenging position of possessing responsibility for achieving systems engineering benefits without the authority to ensure engineers adopt best practices.

The good news is that there is a way out of this situation. The first step to elevating your career is to realize that managing the product engineering process through data is the key and that the Systems Engineering role is the natural role to lead this. For most organizations, the product engineering process is the only process in the company that is not managed through data. No one can go to a system and see the status of all product requirements and where development, integration, risk, and testing stand at any point in time. There is no way to manage by exception. There are no alerts that requirements are not covered, test cases are missed or that hardware made a change that now impacts software. Most organizations have come to accept this state of affairs and try to manage the process through endless meetings, emails, and Slack messages.

Until the product engineering process is manageable through data, Systems Engineers will be stuck in their current trap — endlessly trying to address issues after the fact, holding unwanted meetings, and the uphill battle of trying to persuade changes in behavior. Below is a 3-step approach that Systems Engineers we work with have used to solve this organizational challenge and have thereby elevated their careers. As with any process change, it is best to do it in stages. You can start with the following steps.

Baseline current process performance
Build the business case for change to gain support
Deliver quick wins

To Get Started With Your Free Traceability Diagnostic, CLICK HERE

Step Two | Build Business Case for Live Traceability™

Once you have established a baseline, it is now possible to build a business case for change that will resonate with leadership. Based on your baseline, the Traceability Diagnostic determines the probability of negative product outcomes (defects, delays, rework, cost overruns, etc.) and the magnitude of these events. This quantifies the risk of maintaining the status quo and doing nothing. In addition to the risk reduction potential of Live Traceability, the Diagnostic also calculates the engineering productivity gains from eliminating the need for time-consuming, manual, after-the-fact traceability efforts.

Step Three | Start with Quick Wins

Once you have secured support to move forward, it is common to be able to deliver some quick wins to the organization shortly after project kick-off. The typical place to start is the painful and time-consuming after-the-fact traceability efforts. For example, continuous syncing between requirements and software development task management in Jira or Azure DevOps can be set up quickly to automate traceability from requirements to user stories, eliminating a large source of risk and manual, after-the-fact traceability effort. Once quick wins are shown, organizational momentum increases even further and puts you on the success path to begin managing the product engineering process through data.

Clients of ours that have taken this approach have received significant recognition, been promoted into roles with greater leadership, and have increased their external visibility through speaking engagements. Live Traceability is a unique opportunity to elevate one’s career. Don’t miss the chance.

Live Requirements Traceability

Achieving Live Traceability™ of product requirements, as necessitated by industry standards, across siloed engineering teams and tools, is the #1 unsolved problem for most product development organizations. One of the main barriers is that each engineering discipline (systems, software, hardware, electrical, risk, verification and validation) has optimized its own process and tools. When looking at the end-to-end product development process siloed teams, tools, and data make it very challenging to trace development activity from initial requirement definition through development and testing.

As a result, requirements traceability becomes a time-consuming, error-prone, frustrating, and manual, after-the-fact process. The inability for the product development organization to continually trace ongoing development efforts and changes back to user and system requirements results in missed requirements, defects, rework, delays, audit letters, and cost overruns.

How to achieve Live Traceability™ without forcing change on engineering teams

So how does an organization achieve Live Traceability across a best-of-breed tool environment supporting disparate methodologies, terminologies, fields, and statuses? The answer is a 3-step approach:

Step One | Live Traceability Model

Define a Live Traceability model across the end-to-end product development process with relationship rules for the traceable data elements across best-of-breed tools. An automotive functional safety example is shown below. Here you can see the operational instantiation of functional safety standards requirements in a relationship model within Jama Connect®. All necessary traceable information is included with continuous syncs to best-of-breed tools within engineering teams to deliver Live Traceability.

Step Two | Adaptive Data Field Mapping

To achieve Live Traceability, integrations with best-of-breed tools (such as those shown in the example) are required. The typical integration approach standardizes field names and statuses to ensure consistency across the connected tool, but this does not achieve the dual objective of Live Traceability with no changes required in how each engineering team works. Alternately, the proven approach is to apply adaptive data field mapping to ensure no change to engineering teams’ fields and statuses which simultaneously ensures a consistent, process wide, Live Traceability model. This is achieved through robust mapping and normalization logic functionality to easily address the various approaches taken by each engineering team.

LEARN MORE

In this blog post, experts from Cadence, OpsHub, and Jama Software talk about enabling digital transformation in the hardware and semiconductor industries.

The relentless pace of innovation, rapidly changing markets, and increasing product complexity are creating intense pressures on companies in the semiconductor and hardware space. Some of the biggest challenges relate to scaling effectively and efficiently within the context of digital transformations.

Organizations in all sectors are looking to support faster release cycles and accelerate innovation. Siloed and legacy tool chains create a major hurdle in accomplishing these goals.

Watch the webinar or read the recording to learn more about:

Rich collaboration
Complete traceability
Full transparency among all stakeholders
Faster releases
Improved quality and productivity

Below is an abbreviated transcript and a recording of our webinar.

asd

Jama Connect: the Leading Platform for Requirements

Matt Graham: Thanks everybody for joining. So today, before we get into the agenda just to introduce the three products that there are three subject matter experts about. First of all, something near and dear to my heart, the Cadence vManager, verification management platform which is a scalable, reliable and very feature rich verification planning and management solution from Cadence. That sits on top of a number of our verification and provides a sort of roll up capability. And we’ll describe it in a little more detail in a couple of slides. On the OpsHub opposite side, we’ll be looking at the OpsHub integration manager that enables enterprises to integrate their best of breed tools together that are best suited for the various teams and their various roles and connect those two together for integration and collaboration. And then Jama Connect, which is the leading platform for requirements, risk and test management to help provide that sort of end-to-end compliance solution.

Our agenda today. First we’ll look at some of the challenges of the semiconductor and hardware development ecosystem. This is obviously a very fast paced, highly competitive type of environment and there’s a lot of specific challenges that the integration of the tools I just mentioned can help address and solve. We’ll look at how engineers in this space can scale effectively and efficiently utilizing some of these, the tools to address some of the ongoing transformations in that space. And then specific to semiconductor domain, bridging the gap in what has historically been a very siloed development process. And bringing together for efficiency, quality and reliability all of the various tools that I mentioned and giving it a really nice integrated development and verification environment. We’ll then have a specific use case and demo showing how the three tools work in concert and then look at some key takeaways. And as Marie mentioned, some Q and A.

Standards for Requirements such as ISO 26262

Specifically to the semiconductor and hardware ecosystem, there are a developing set of challenges. And of course they’ve always been challenges in this area. First pass design success is critical for hardware development. Just because the tooling costs are so great. We don’t want to have to respurn hardware. It’s not like just releasing more software. It is it requires expense. But that has been the way of hardware development for some time. In the last several years we’ve seen a need creeping into that environment for even stricter compliance, particularly around mission critical domain such as aero and defense, automotive especially as self-driving and autonomous vehicles come in. And adherence to standards like ISO 26262 presents another layer of requirements and need for management and collaboration on top of an already strict set of sort of design parameters.

As I mentioned, this development environment tends to be very siloed in its nature because it is so specialist. You have specialist designers, specialists verification engineers to test the designs, specialists post silicon, specialists layout engineers and so on and so forth. And all of those silos, well somewhat required of the specialty of each of those tasks tends to hinder collaboration, compromise quality and just impact efficiency and velocity overall. In an area where efficiency and quality is critical. We can’t have bugs in semiconductors going to automotive and we need to be able to turn those new cell phones, those new mobile devices as quickly as possible. So turnaround time is just getting compressed and the requirement for quality is increasing at the same time.

RELATED: A Guide to Understanding ISO Standards

All of that sort of siloed nature of the specialties as well as the need for velocity and quality really ends up in poor traceability of results in terms of compliance and quality issues creeping in. Especially when it comes to doing things like audits for ISO and other similar standards that are becoming the requirement across again aero and defense type applications, automotive type applications and even down into the sort of consumer device applications. And really traceability is a watch word now in the ecosystem of hardware and semiconductor development.

So how does the offering from Cadence, vManager fit into and help provide a solution to those challenges that I just mentioned? Well, for a number of years now, in fact, vManager has been around for about 15 years and in that entire time it’s had the key capability of the verification plan. And the verification plan really exists to provide traceability between what is being executed during the testing or verification of your semiconductor or hardware design. And what were the goals of that or the requirements of that testing or verification project. Things like testing interfaces, both internal and external to the semiconductor, testing compliance with standards like ethernet and USB, such as that, things like that. As well as the internal requirements of the device, it must route packets this fast. It must answer phone calls in this manner or whatever it might be.

And the verification plan in vManager really allows the user to enter those requirements and then connect them to the real results that are occurring. We ran these tests, these tests were associated with a given requirement. Those tests passed therefore the requirement is satisfied. And so the V plan becomes a very natural place. And in fact the appropriate place to connect the rest of the ecosystem via OpsHub, two tool requirements coming from Jama Connect so that we can have traceability across the software development, the hardware development, whatever. The mechanical development et cetera ecosystems. And the vManager and the verification plan is really where that hardware verification, that hardware and semiconductor development information enters that ecosystem through the conduit of the verification plan. So let’s look a little bit more on, well what exactly is in that verification plan that vManager provides.

Enabling Digital Transformation: Static Documents Cause Challenges

And the V plan is really what we call, what we refer to in our vManager sort of pitch if you will as an executable verification specification or an executable verification contract. And what that means is that there’s data incoming to that during the creation, the authoring of that verification plan. Not only through connectivity to tools like Jama but also from say static documents like standards specifications, ethernet that I mentioned before, USB those are standard protocols that have very lengthy standards documents and needs to be a way to import, kind of gather the data from that and put it in the verification plan. Another input to the verification plan is other verification plans. So if you think about a system on a chip that is not a single piece of intellectual property, it’s built up of many, many different pieces, a USB piece, a central processing piece, a memory management piece and so on.

And each of those pieces can have their own verification plans for the verification at that sort of lower block level as well as then can sort of conglomerating or aggregating those verification plans into a single sort of system on a chip verification plan. And the vManager, V plan allows that through sort of parameterization and instantiation and really flexible set of sort of reuse capabilities for verification lands. And then of course just engineers authoring their verification plan. Literally writing, typing in here’s a specific requirement et cetera. And then we have the component of mapping those requirements to items that exist in the actual testing environment. Things like we have a test, did it pass or fail? What requirement is that test related to? So there’s mapping the test to a particular requirement and then did that test pass or fail. Those of you familiar with hardware verification know that tests passing and failing is not the only statistic or metric that we track.

There’s other metrics and statistics such as code coverage, functional coverage, assertion coverage, software coverage, all tracking what scenarios and what stimulus were driven to the specific device under test. And what was the reaction of the device under test? And then what percentage of the device has been exercised during that test? It is all basically statistics gathering from the testing effort. All that data can be mapped into the verification plan, directed to the specific requirement or multiple requirements that it may satisfy. And of course, this gives us the ability to not only specify a requirement, but then capture whether that requirement was met. Was it satisfied? And this is the place where I’ll hand over to Jeremy now to talk about what those requirements in those higher level requirements or system level requirements in the general world and how they’re going to connect into this hardware verification, hardware development world.

Design Controls have been an FDA Quality System Regulation since 1997. Having worked on developing products in the regulated medical device industry for over 35 years, I have compiled a list of the five key takeaways for implementing design controls and achieving success in commercializing medical devices:

Design Controls not only help achieve regulatory compliance, they help develop better products
Design Inputs lay the foundation for product development, building a good foundation
Don’t underestimate the power of Bidirectional Traceability
Know the difference between Verification and Validation
Risk Management is a vital part of Design Controls

#1 – Design Controls not only help achieve regulatory compliance, they help develop better products

Many companies think that design controls are a burden to development organizations imposed by the FDA, and it’s the price to pay for playing in the medical device field. However, what is often overlooked is that design controls only define the basic minimum requirements necessary to develop a product that can…

…meet the needs of the user.
…be designed to be safe and effective.
…be reliably manufactured.
…be verified and validated.
…maintained and updated throughout the product lifecycle.

These are all things that any development organization should do to successfully deliver products to market. I like to say that if you are doing the right things in product development, compliance comes for free!

#2 Design Inputs lay the foundation for product development, building a good foundation

The FDA defines design inputs as the physical and performance requirements of a device

that are used as a basis for device design. To generate adequate design inputs, the foundation upon which product development is built, the user needs must first be well understood. These needs, ideally written with the voice of the user, must then be translated into design requirements. In contrast to the user needs, these design requirements should be written using the voice of the engineer, and as such should be measurable and testable. Furthermore, design requirements should be traceable to a specific user need, risk control, or standard that necessitates the existence of said design requirement.

Research has shown that, on average, companies that are successful at developing products spend about 25% of the product development time on the generation of user needs and the subsequent design requirements. The return on this investment of time and resources reduces the need for rework and redesign, and ultimately leads to higher customer satisfaction. Failing to make the investment ensures that design inputs are complete and correct is analogous to building a house on quicksand, where the flaws in the foundation can cause issues throughout the construction and subsequent (likely short) lifetime of the house. Issues with requirements will impact development, verification, validation, and user acceptance of the product, so spending the time to get requirements right will be well worth the effort.

RELATED: Three Ways to Proactively (vs Reactively) Incorporate Design Controls in Medical Device Product Development

#3 Don’t underestimate the power of Bidirectional Traceability

In an audit, the trace matrix should be valued as a friend! Having and maintaining bi-directional traceability throughout the product lifecycle provides a number of benefits:

Effecting project tracking
Thorough change impact analysis
Ease of making future changes
Re-use of elements of the design
More effective issue resolution

To derive these benefits, the relationship between the following entities should be established:

User Needs and Design Requirements
User Needs and Validation
Design Requirements and lower-level requirements
Design Requirements and Verification
Lower-level requirements and verification
Lower-level requirements and Design Outputs
Risk Controls and Design Requirements

In creating a trace matrix that has views to show all the bi-directional relationships of each of the elements described above can help answer most questions from an auditor. With this level of traceability, I can trace from a user need all the way through implementation and test.

#4 Know the difference between Verification and Validation

The terms “Verification” and “Validation” often get combined and abbreviated to V&V; however, these activities are vastly different.

Verification is confirmation by examination and provision of objective evidence that specified requirements have been fulfilled. It is design-centric and answers the question “Did I build the product right?” Verification also entails gathering objective evidence that the design behaves as intended through the use of observation (visual inspection), measurement (values and tolerances), testing (function) or analysis (reviews).

Validation is confirmation by examination and provision of objective evidence that the particular requirements for a specific intended use can be consistently fulfilled. Unlike Verification, this is a user-centric term, and answers the questions “Did I build the right product?” and “For whom is this the right product?” Validation entails gathering objective evidence that the design satisfies the user needs through the use of Usability Studies/Human Factors Studies, Clinical Evaluation/Clinical Studies, Customer Surveys, and through Analysis of Verification Data.

Knowing the difference between Verification and Validation is of quintessential importance for ensuring customer satisfaction and regulatory acceptance of the product.

#5 Risk Management is a vital part of Design Controls

The elements of design controls are Planning, Design Inputs, Design Outputs, Design Reviews, Design Verification, Design Validation, Design Changes and the Design History File. So, what happened to Risk Management? Risk is mentioned in Design Control Regulation (QSR 820.30) all of one time, under Design Validation. The statement simply reads “Design validation shall include software validation and risk analysis, where appropriate.”

Fortunately, the FDA Design Control Guidance elaborates on requirements for risk management. The guidance includes this paragraph:

Risk management begins with the development of the design input requirements. As the design evolves, new risks may become evident. To systematically identify and, when necessary, reduce these risks, the risk management process is integrated into the design

process. In this way, unacceptable risks can be identified and managed earlier in the

design process when changes are easier to make and less costly.

The takeaway from this is that although risk management is just cursively mentioned in the QSR Design Control regulation, the intent of the regulation is that Risk Management be practiced starting from the point where design inputs are known and practiced throughout the product life cycle. You cannot be compliant to the design control regulation without having an adequate risk management file.

Conclusion:

Design Control regulations have been around since 1997, but many manufacturers still have problems complying with design controls. Focusing on the best practices outlined above will derive the most benefit from implementing Design Controls, will lead to a more predictable development cycle, and ultimately result in higher-quality products that can be enhanced and maintained throughout their lifecycle.

Editor’s Note: This posts on lessons learned around medical device development during COVID 19 was originally published here by MedTech Intelligence and written by Josh Turpen, Jama Software’s Chief Product Officer.

In the fall, I wrote about how the medical technology industry has struggled to keep pace with other, similar industries. In the piece, I discuss how important it is for engineers designing those products to move gradually and carefully, even when under immense pressure, to reduce time-to-market. Now, a year on from the stay-at-home order issuance across the United States, it’s time to take stock on what temporary measures need to be made permanent to grow as an industry.

As we move forward into a post-pandemic world, it is important that companies are explicit with the lessons that they have learned from this past year. Executive staff, rightfully so, have been focused on keeping things going. Now the focus should shift to “how do we exit the pandemic in a better place than when we entered?” This is where it becomes important to create an open dialogue about what was successful and what could have been done better. This will assist in making those temporary adjustments a permanent fixture in medical device production.

Before we look towards a post-pandemic world, though, we need to evaluate where things went wrong and how to better address them moving forward.

What Have We Learned?

If COVID-19 taught us anything it is that we need to be more efficient at speed-to-market when creating products, especially when it comes to medical product production. Additionally, gone are the days of person-to-person-only collaboration. Organizations now have the capabilities for a hybrid environment consisting of remote and in-person teams.

The complexities of product development within health and life sciences should not be a surprise. What is more alarming is that, as the complexity of medical devices increases, we still have many engineering teams that are relying on decades-old technology such as Word documents and spreadsheets to manage requirements, risk assessment and testing. These legacy tools have a place in most of the professional world, however, they are not adequate for development teams who need to achieve alignment with massive amounts of data, regulations and standards to ensure device safety and quality.

As these device management teams face immense pressure to innovate while collaborating across software, hardware and quality teams, it is essential that their work is tracked and seamless to meet the increasing pace of market demand. That’s why we have seen traceability evolve to account for the complex, ever-changing nature of requirements, test and risk management. For a growing company to be successful, everything must be able to work simultaneously, at scale and across teams. Legacy tools do not provide the agile capabilities that modern traceability does.

It hasn’t been easy for engineering teams to adjust to their fully remote workplace. Even organizations that offered a hybrid working model previously are struggling to ensure their teams are aligned to meet delivery dates and project deadlines. The organizations that will have a distinct advantage over others are those focused on collaboration and context within their teams. These teams will be best set up to quickly build high-quality products, further ensuring better patient outcomes.

RELATED: How to Executive a Successful Design Review When Building Medical Devices

Where We Go from Here

As mentioned previously, the best course of action following this difficult year is to ensure company leaders are shifting their focus towards figuring out how to leave their companies in a better place following the pandemic, versus where they were when they entered. Based on my experiences as the chief product officer for a leading requirements, risk, and test management platform, I have noticed a few key ways that executives in the medtech industry can better prepare themselves moving forward.

1. Adaptability

In general, companies that have adaptability embedded in their DNA have already handled the peaks and valleys of the pandemic far better than those that remained rigid in their approach. Looking ahead, it will be immensely important to evaluate your process assumptions and determine how resilient you are to change.

Without a malleable business model, a company will constantly be scrambling any time it hits a road bump. However, with the right digital tools at your disposal, your company will be able to adapt quickly and effortlessly, allowing your employees and customers to remain calm in times of crisis.

2. Alignment

While companies in all industries adapt their business models to prepare for the new normal, innovative tech companies are transforming the devices and systems they build, and the technology and process they use to build them. Newer technologies in the medical field, like robotics nanotechnology and wearable health tech devices, bring added complexities for medical device companies. There is an additional risk for patients and consumers, which makes having the right product development solution in place even more important.

Medical device companies that embrace a proactive approach to quality will ultimately find fewer issues with their products, improve customer satisfaction, and stay competitive for the foreseeable future. To do this quickly and efficiently, though, teams must be aligned throughout the entire product development lifecycle. By leveraging an integrated platform for requirements management, teams can stay interconnected and deliver high-quality products that improve patient outcomes.

RELATED: Your Guide to Selecting a Medical Device Development Platform

3. Preparation

Benjamin Franklin once said, “by failing to prepare, you are preparing to fail.” As we look ahead into 2021, I believe it is unlikely we will see large regulatory changes in the medtech industry. However, over the next decade, a great increase in regulations for medical device development is definitely looming. So what can a company do in the meantime? Tighten up your risk management practices, before it’s too late.

The fact is, the medtech industry will always grow at a rapid pace, and regulations will follow. To avoid being left behind by events such as, finding regulatory issues in late-stage device development, and then having to implement confronting costly and time-consuming rework, your teams must align and future-proof the entire development lifecycle.

Future-proofing is the process of using digital tools to capture knowledge and ease accessibility for future employees, independent of the product development lifecycle stage. Employing future-proofing strategies helps company leaders and decision-makers ensure symbiosis throughout the entire process. Future-proofing is key in the age of digital transformation, as it helps address common concerns about collaborative environments, team efficiencies, and product integrations. A software that is up to this task will help you do this in three ways:

Comprehensively enables collaboration by giving users a single source of truth to track decisions, questions, and problems
Increase team efficiency by capturing knowledge within that single source—often without even realizing it—through feedback and team communication.
Seamlessly integrates digital tools that track development and with other information gathering and tracking solutions, knowledge is captured at multiple levels, streamlining future projects.

4. Harmony

Finally, to be successful, there will need to be harmonization on the development methodology across different devices, reducing the need to work off of different documents. Each product in development has its own particular set of customers, stakeholders and internal team members associated with it. Therefore, it is important that these individuals can be accurately connected to the items for which they are responsible. Enter traceability.

Traceability is all about relationships. To make informed choices, product development professionals need tools that allow them to see changes in real-time, within the team’s structure, and throughout the system where their product exists. Modern traceability makes it possible to manage and respond to change with confidence in a systematic and auditable way. When done correctly, traceability can be used as a key tool to allow for harmonious decision-making. Without it, accountability is incomplete and past decisions can’t easily be seen, learned from or built upon.

Overall, COVID-19 has been industry-defining as companies were made to quickly shift how their teams collaborated, now forced to have a remote workforce. As we see the light at the end of the tunnel it’s time to look towards the future of medical device development. Let’s take all that we’ve learned from this past year and use it to ensure that we’re putting out high-quality products, quickly and accurately.

Requirements Traceability – What are you missing?

For systems engineers, business analysts, and product owners, requirements traceability (the ability to trace requirements to downstream development, test, verification, validation, and risk activities) is an unquestioned good and an unquestioned need. Traceability is required to demonstrate compliance to relevant standards in industries such as medical device, automotive, semiconductor, aerospace, and financial services. In addition to compliance requirements, traceability is quite helpful to assess the impact of change required on all relevant requirements and related downstream activities. But the largest potential value is missed by many organizations.

After-the fact use cases

The two main use cases for traceability noted above are both reactive to requests: the need to demonstrate standards compliance to third parties and the need to analyze the impact of a change request. Both use cases view requirements as static and passive. Requirements are documented and links created to downstream artifacts in software, hardware, and electrical development test and risk assessment — which are stored in a system. The system then waits until a request comes in from outside to document that a process has been followed or to identify which elements are impacted by a change. Both use cases reflect an after-the-fact mindset that limits the value that can be achieved from the effort put into establishing traceability.

Analogies to other business-critical functions

To view something we think we know well in a new light, it is often helpful to place it in a different context and look at it from a fresh perspective. So, let’s give it a try and compare traceability in the product development process to traceability in the new customer acquisition process.

For engineers, these processes may at first appear to be too disparate for an apt analogy, but at a fundamental level, they are quite similar. Both start with a documented value (requirement vs. opportunity) that must transition through multiple phases and involve many other functions to reach the desired outcome (release vs. win) — and all sorts of things can go wrong along the way leading to delays, costs incurred, and failure.

The aspect I want to focus on is how these two processes are managed. In the sales process, the element of value (the opportunity) is living — actively measured, analyzed, and tracked for exceptions on a daily basis. The key process metrics are defined with ranges for acceptable performance. Current process performance is automatically calculated based on the movement through stages of all opportunities. Alerts are raised for opportunities stuck in a process stage too long (relative to average dwell times) and predictions are made about future period performance based on the opportunities in the system.

RELATED POST: Requirements Management – Living NOT Static

In contrast, for most product development organizations, requirement traceability is static and in after-the-fact analysis and not living — in the way opportunities are traced in sales as described above. To make requirement traceability living (like a sales opportunity) traceability would require software-enabled best practices in the following areas:

Process metrics must be defined
Actual performance against process metrics must be captured
Once standard metric performance is defined, current performance must be compared to the standard
Exceptions need to be defined
Alerts need to be set up to notify when exceptions occur
Learnings need to be incorporated into process improvement

Once these steps are taken to improve requirements traceability, then requirements become living, not static, and performance improvement is possible: the risk of negative product outcomes is reduced, and process performance is improved; the product development process can then be data-driven, measured, and managed like all other business-critical processes. Without measurement, there is no way to benchmark performance against other organizations. There is no way to learn, no way to know, and no way to improve.

Jama Connect’s Requirements Management Enables Live Traceability™ Across Your Development Process

Bridge engineering siloes across development, test, and risk activities. Provide end-to-end compliance, risk mitigation, and process improvement with our intuitive, award-winning requirements management platform. Learn more!

Better to stay in the dark?

There may be some that would prefer to keep the product development process shrouded in mystery and avoid data-driven analysis, measurement, and process performance improvement. There used to be chief revenue officers (CROs) that felt the same way. It is extremely hard to find any current CROs with that mindset. Those that can lead through data and drive performance improvements gain the support of leadership and elevate their careers. The same opportunity is now on the table for product leadership to turn requirements traceability into living requirements in order to reduce the risk of negative product outcomes and improve the performance of the end-to-end product development process.

To learn more on the topic of requirements traceability, we’ve curated some of our best resources for you here.

SEE MORE RESOURCES

Editor’s Note: This post about how traceability improves collaboration and decision making was originally published here on DevOps.com on September 23rd, 2020, and was written by Josh Turpen, Jama Software’s Chief Product Officer.

Jama Connect® creates Live Traceability™ through siloed development, test, and risk activities. Provide end-to-end compliance, risk mitigation, and process improvement with our intuitive, award-winning platform.

Minimize the risk of delays, defects, cost overruns, and the manual effort created by fragmented development processes and legacy solutions. Learn more!

New software is being developed at an incredible pace to help make our lives easier. This doesn’t change the fact that humans are still held accountable for product development decisions, whether these are made with or without advanced analysis tools. To make informed choices, product development professionals need tools that allow them to see comprehensible information in real-time as change is happening, both within the team’s structure and throughout the system in which their product exists.

Modern traceability makes it possible to both manage and respond to change in a systematic, auditable and confidence-enhancing way. Below, we will discuss three ways traceability has evolved to support key decision-makers in a number of industries.

Modern traceability captures when you make a decision

Decisions often have varying levels of durability. Sometimes, when you make a decision, you know then and there it’s final. Other times, you make what seems to be a minor choice and you end up dealing with the repercussions for years to come. With this level of uncertainty, it is essential to have mechanisms that allow you to see when decisions are made. As I discussed in a previous article, “5 Ways Traceability is Changing to Bolster the Remote Workforce,” this process can be compared to a map. By leading a team through every step of their processes, modern traceability helps product developers reach their goals without any surprises.

Often, products are expected to be maintained for years. This is significantly more challenging when you can’t properly track where a ruling originated or a change was made. While the team may move quickly in the development process, the record should always live on to provide future context where it’s needed.

Modern traceability provides people the context of what they’re working on as they go, not after the fact

Rather than rely on sharing often lengthy and disparate documents or running time-consuming general meetings, traceability allows teams to streamline their collaboration. Mapping out work items, including owners and contributors, gives people a reason to care and to trace those items carefully. It helps everyone know why they’re there, what they are discussing and how to address it.

Many smaller companies are fortunate to get by using Word documents and other legacy tools for their traceability measures. However, as these companies grow, so do the complexities. That’s why traceability has been evolving to account for the multi-dimensional nature of requirement, test and risk management. For a company that is seeing major growth to be truly successful, all related variables must work together continuously, at scale and across teams. Legacy tools simply do not provide the agile capabilities that modern traceability does.

Traceability captures and tracks past decisions and allows users to access them

Traceability is all about relationships. Each product in development has its own particular set of customers, stakeholders and internal team members associated with it. Therefore, traceability is only possible if these individuals can be accurately connected to the items for which they are responsible.

Knowing who made a decision and what information they accessed is equally as important as the information itself. If you can’t quickly piece that together, your traceability is incomplete. The responsible thing to do is to ease the process by keeping useful records. It doesn’t need to be forced behavior if it’s captured along the way as a byproduct of doing your job.

Overall, accountability is incomplete and past decisions can’t easily be seen, learned from or built upon without robust, modern traceability tools. It’s much harder to legitimately hold someone accountable when they’re working in the dark. However, when done correctly, traceability can be used as a key tool to support genuine liability and allow for a streamlined process of complex decision-making.

Download our eBook to learn how optimize product development with strategic team collaboration.

DOWNLOAD NOW

Traceability is the ability to track upstream and downstream relationships between requirements and other artifacts, ranging from test cases to higher-level system or subsystem requirements. Through end-to-end traceability, teams can see if a product’s development process is currently on track, as well as view any and all of the history and context associated with it.

That’s the ideal, at least.

Where Traceability Can Come Up Short – And How to Fix It

Traditionally, traceability has been performed using document-based workflows involving applications such as Microsoft Word and Excel. A team member creates a traceability matrix in a text document or spreadsheet and updates it manually throughout the product development lifecycle. Unfortunately, this approach has distinct limitations and is prone to error.

When teams trace requirements within discrete and static documents, they often create extra work for themselves, while also running the risk of missing out on critical updates or making mistakes.

Let’s say someone updates a matrix in Excel with the latest statuses of test cases related to a medical device that’s in development. It might seem like everything is proceeding smoothly on the surface, but multiple problems could be lurking underneath:

Human Error

Since everything is done by hand, this individual must regularly revisit the matrix to keep it in sync and up to date with product development activities across the whole organization. The complexity of the traceability matrix in question – with its numerous tables showing how requirements and test cases connect to one another – makes such work inherently complicated and error-prone.

Email-Centric Collaboration

Meanwhile, these updates to the matrix are primarily communicated via email. Between the normal flow of messages into everyone’s inboxes, people being out of the office, and other complications -like poor version control – and mishandling of incorporating everyone’s comments, it is likely that errors will occur and misunderstanding will necessitate rework down the line.

No Built-In Compliance

Finally, even if everything goes according to plan, there’s no guarantee that the traceability workflows in use will account for all relevant requirements and risks. For instance, in the case of a medical device, a matrix created within Excel won’t come with frameworks aligned with industry standards like ISO 14971, making it more difficult to ensure coordinated traceability and ensure successful proof of compliance.

Fortunately, these types of issues don’t have to hold your teams back and expose your product development processes to undue risk. End-to-end traceability is possible by upgrading from document-oriented workflows to a comprehensive requirements management platform that enables real-time collaboration, bidirectional traceability, and integrated risk management.

Let’s look at four central benefits of implementing end-to-end traceability with a solution like Jama Connect®.

1. Holistic, Actionable Visibility into Requirements and Stakeholders

Traceability is ultimately about relationships, not just between requirements and other artifacts, but between all of those items and the people responsible for managing them. With end-to-end traceability in place, teams can see:

How requirements trace forward to their implementations within work products, along with how those products trace back to original requirements and designs.
How all requirements were tested – i.e., the test cases linked to them, whether the tests passed or failed, and any associated defects identified along the way.
Who was involved in the development of specific requirements and test runs, so that they can be notified right away about necessary next steps or actions required.

This comprehensive visibility creates a system of action – one that teams can use to not only trace the life of each artifact but to initiate the appropriate activities to sustain product development and ensure coverage. Is a medical device safe to use? Does it comply with applicable standards? Can the processes used for building it stand up under an audit? A full-fledged traceability solution provides clear answers to these questions and others.

2. Better Impact Analysis of Changes

Truly informed impact analysis isn’t possible without end-to-end traceability. That’s because such analysis is rooted in being able to see how specific changes to an artifact will affect other items connected to it. Knowing those impacts requires traceability.

In traditional document-based workflows, it can be difficult to know how different items are affected by change. But in a platform with end-to-end traceability, much of this work is automated and becomes a byproduct of your daily work. The solution will instantly flag downstream links as “suspect” so that teams can attend to them as needed.

For example, teams can see if an altered requirement has test cases downstream and what share of them have passed – all in real time. This setup saves immense time and effort compared to manual processes.

3. Easier Identification of Gaps in Test Coverage

Speaking of tests, a requirement is typically only considered “covered” if it has corresponding test cases against it, as well as test engineers assigned to it. But too often, gaps in coverage only become apparent after the fact, when a product issue reveals how a key flaw was overlooked during development.

In fields with rapid change and innovation, such as medical device development and automotive manufacturing, any unidentified coverage gaps are risky to end users and costly to remediate. Improved tracking of test coverage in a platform with end-to-end traceability helps eliminate these blind spots and ensure quality.

More specifically, this level of traceability within a requirements management solution helps test engineers and project managers visualize where gaps exist and whether tests have been approved, completed, rejected, or drafted. As a result, product development becomes less risky overall.

4. Simplified, More Accurate Audit Passage

“Show your work” is a familiar adage to anyone who’s completed a mathematics assignment, and it is crucial advice when tracing requirements, too. Passing an audit will require presenting specific information about those requirements, in formats acceptable to reviewers and regulators.

A platform like Jama Connect simplifies this process by letting teams show clear supporting evidence of comprehensive traceability. It provides export templates like trace reports to provide this evidence, simplifying regulatory submissions and the audit process.

Jama Connect’s Requirements Management Enables Live Traceability™ Across Your Development Process.

Download our eBook to learn how to optimize product development with strategic team collaboration.

DOWNLOAD NOW

Impact analysis is the assessment of the implications of changes, in the specific context of product development. It is integral to requirements management, as it offers insights into dependencies and gaps in coverage, which help inform decisions about the product’s lifecycle.

Let’s say an organization is developing a medical device and must change some of its requirements along the way – something that happens all the time. By conducting an impact analysis of this requirement and its dependencies, the product development team could accomplish all of the following:

Determine the ripple effects of the change: How will it affect higher-level requirements upstream and links downstream? Are there are any conflicts to resolve? Proper impact analysis answers these questions and in turn reduces the risk of unexpected consequences requiring costly remediation later on in the lifecycle.
Connect people, not just requirements: The impact analysis process should go beyond flagging at-risk upstream and downstream items. It should also reveal who is affected by changes, plus notify those team members about the necessary next steps. This is where live traceability and real-time collaboration in one platform really matter.
Assess what is required going forward: Impact analysis is ultimately about gaining visibility into the future – almost like a crystal ball – and acting accordingly. In addition to showing potential risks to existing requirements, it can also identify new requirements and test cases that may be needed for keeping the project on track.

Teams may perform impact analysis using multiple tools. A dedicated requirements management platform, with integrated risk management and end-to-end traceability, is ideal for fully mapping out the effects of changes and identifying subsequent action items, along with the personnel responsible for them. When manual tools are used, data gets kept in silos which can create misalignment, poor visibility, and difficulty to assess the impact of change.

Replacing those manual workflows, which often revolve around software like Microsoft Word and Excel, is crucial for optimizing impact analysis. That’s because informed analysis requires accurate traceability, something made much easier with a proactive and automated platform that serves as a single source of truth at every stage of product development. In this way, requirements management software addresses the biggest roadblocks to effective impact analysis.

Overcoming the Obstacles en Route to Better Impact Analysis

From the rapid pace of innovation in industries like medical device development and automotive manufacturing, to the need to coordinate increasingly distributed and remote engineering teams, there is no shortage of possible challenges when building a complex, modern product. When it comes to impact analysis in particular, the three primary issues include:

1. Manual Traceability

Impact analysis isn’t possible without effective traceability. But with so many evolving requirements to keep track of, accurately tracing everything can be difficult without truly scalable and automated tools.

To see why, consider a hypothetical case in which a critical requirement needs an immediate change. The time constraints mean all upstream and downstream effects must be quickly accounted for, with requirements and other artifacts (like test cases) properly traced forward and/or backward as needed.

But doing so is difficult when working with nothing more than a traditional traceability matrix housed in Excel or Word. The amount of manual work required could result in something being missed due to human error and a lack of comprehensive traceability within the matrix itself. The team might not realize they’ve overlooked missing coverage until it’s too late.

Solution: Live traceability provides straightforward navigation of upstream and downstream relationships, along with automatic identification of risky links. It also updates items in real time so that team members are always looking at the most accurate assessment of test coverage.

2. Inefficient Modes of Collaboration

Email-oriented collaboration only compounds the above problem. When lengthy, complex documents – which might not even be up to date – are emailed between teams, confusion, and delay are almost inevitable.

Simply staying current with any changes to project requirements can mean searching through your crowded inbox and trying to reconcile various attachments. Is “RequirementsDoc_v2FINAL” really the final version, or is there something more recent circulating under a different name?

In order to effectively conduct impact analysis, everyone tied to the requirements needs to be notified in real-time when changes are made and provide their input. Centralizing this information creates a more systematic way for change management and enables accountability within the organization on how decisions are made past and present.

Solution: Instead of email, use a real-time platform with instant notifications and the ability to pull in internal and external collaborators as needed. On such a platform, everyone is looking at the same data, which helps expedite review and approvals.

3. Rework and Opportunity Costs

Impact analysis is supposed to reduce risk by letting you see how specific changes will play out and empowering you to take any corrective action as early as possible. But when impact analysis is built upon manual processes and limited traceability, it can do the opposite and actually make projects riskier.

For example, extensive rework may be required to ensure that all requirements are met. This work represents a major opportunity cost, as the time sunk into correcting process-related problems could have gone into more strategic initiatives.

However, teams can avoid getting bogged down in rework by upgrading their impact analysis and traceability tools. Leaving static documents behind for an automated platform with real-time collaboration built-in is both reliable and helps to ensure product quality.

Solution: Invest in a platform that enables proactive, rather than reactive, requirements management. Features like live traceability that connect requirements to tests make it easier to handle changes as they happen and avoid costlier actions later on.

Fueling the Engine for Superior Impact Analysis

A modern requirements management platform enables streamlined impact analysis while bringing teams closer together to work in real-time, even if they’re physically distributed. As a result, analyzing upstream and downstream relationships becomes more practical, as does the overall development of high-quality products within budget and on time.

More specifically, the right solution will be an engine for better impact analysis, propelling key advantages throughout product development, including:

Automatic identification of suspect links: If a requirement is modified, the platform will automatically flag its downstream links as “suspect” so that team members can review them before proceeding with development.
Easier relationship navigation: Users can efficiently navigate upstream and downstream relationships. A visual schematic lets team members save time in finding missing coverage and make sure that they’re not overlooking anything.
Enhanced collaboration: Team members get notified about relevant changes and can be pulled in right away to respond, on a shared platform offering a single source of truth. This real-time setup is much more efficient than relying on email alone.

Want to learn more about how Jama Connect can improve your impact analysis? Set up a free trial today to get started.

GET STARTED