Tag Archive for: Product Development & Management Page 10
Tag Archive for: Product Development & Management
The Jama Software® Discovery Center: Learn the Value of Jama Connect® for Complex Development
Welcome to the Jama Software® Discovery Center — a dynamic resource designed to guide you on your journey with Jama Connect®. Whether you’re just beginning to explore modern requirements management or you’re an experienced user looking to optimize your processes, the Discovery Center provides everything you need— right at your fingertips. Learn about Jama Connect and grow your knowledge at your own pace with this comprehensive resource.
Explore the Four Key Areas of the Discovery Center
The Discovery Center is organized into four main areas, each designed to meet you wherever you are on your journey with Jama Connect or simply better understanding how to optimize complex product, systems, and software development.
Here’s an overview of what you can expect:
Discover – Just starting out? Begin your journey in the Discover section, where you’ll find a wealth of resources to help you grasp the fundamentals of modern requirements management. This area is equipped with a comprehensive buyer’s guide, best practices for requirements management (RM), and insights on how centralizing your RM can mitigate risks in product development. Whether you’re evaluating RM tools or seeking to refine your current processes, this is the ideal starting point.
Explore – Curious about how Jama Connect can address your development challenges? The Explore section is for you. Here, you can delve into how Jama Connect® can cater to your specific product, systems, or software development needs. Access curated resources, including customer stories, a complimentary 30-day trial of Jama Connect, and our Get Started video series. This section is designed to facilitate informed decision-making by demonstrating why industry-leading organizations worldwide, choose Jama Connect.
Align & Launch – Ready to implement Jama Connect or better understand what that might look like? The Align & Launch area serves as your go-to resource for successful installation and adoption of Jama Connect within your organization. This section provides key resources to support implementation planning, including FAQs, installation tips, and an in-depth examination of the platform’s features and functionality. It encompasses everything you need to ensure a seamless transition and effective rollout.
Optimize – Already using Jama Connect and seeking to maximize its potential? The Optimize section is tailored for users who want to enhance their environment and ensure long-term success. Here, you’ll find comprehensive information on the REST API, optimization videos, workshops, tutorials, and other tips and tricks to help you fully harness the power of Jama Connect’s robust capabilities.
The Jama Software Discovery Center is more than just a resource collection — it’s a comprehensive guide that empowers you to take control of your knowledge journey with Jama Connect. Whether you’re discovering, exploring, aligning, launching, or optimizing, the Discovery Center is here to support your success every step of the way.
Note: This article was drafted with the aid of AI. Additional content, edits for accuracy, and industry expertise by Decoteau Wilkerson, Kenzie Jonsson, Karrie Sundbom.
In this blog, we’ll recap our recent webinar, “Achieving Success in Energy Storage Development: Tips & Best Practices” – Click HERE to watch it in its entirety.
Achieving Success in Energy Storage Development: Tips & Best Practices
Are you prioritizing the safety and success of your energy storage systems (ESS) development?
Teams developing ESS must prioritize product safety and effectively navigate the certification process. By learning best practices and gaining insights into Underwriter Laboratories (UL) standards, they can enhance safety and ensure compliance with industry regulations.
You’ll gain a thorough understanding of these topics and more:
Proven best practices for energy storage system development
Key tips for achieving certification to UL standards
How Jama Connect®‘s pre-defined framework supports successful energy storage systems development and ensures compliance
Don’t miss this opportunity to gain valuable insights and learn how a modern requirements management solution can streamline your energy storage development efforts.
Below is an abbreviated transcript of our webinar.
Steven Meadows: Welcome to today’s webinar on achieving success, as well as energy storage development tips and best practices. So for today’s webinar, we’re going to dive into some essential topics that are going to be very important for successful and safe energy storage system development and successful certification. So our agenda really is going to cover a few areas, including key tips for achieving certification with UL standards and development best practices, Jama Software’s perspective on development challenges, and the effective use of Jama Connect for managing the development of energy storage systems, as well as a sneak peek into our new energy storage development framework.
Now, before we get started, I’d like to briefly introduce myself and my background. My name is Steven Meadows and I’m a principal solutions lead here at Jama Software. With a pretty robust background in requirements management, I bring around about 10 years of experience in implementing software and working with hardware and software teams across a broad spectrum of industries, helping important market game changers really succeed in their development efforts. Now, throughout my career, I’ve had the privilege of working closely with many, many incredibly innovative and life-changing organizations, helping them navigate the intricate landscape of engineering. So Jama Software, my focus has been on empowering teams to achieve their project goals efficiently and with precision. Whether it’s harnessing the full capabilities of Jama connect the platform or strategizing for complex project scenarios and engineering scenarios. My passion really lies in delivering tangible results that drive innovation and enhance operational excellence. Today I’m excited to be joined by Chris Flueckiger who will be talking about energy storage development best practices and certification tips. He’s a bit of a guru in the space. Would you like to introduce yourself, Chris?
Christopher Flueckiger: Sure. Thank you. It’s nice to meet everybody. I’ve spent about the first half of my career working with the design phase of electrical equipment and then the past almost 30 years in the certification of renewable energy systems including battery energy storage systems. So I’ve been working with large companies. I work with all the certifiers currently and represent companies as they look towards certification, all the way from the concept and design of a product through the final certification in marketing and installation. So I look forward to talking with everybody.
Flueckiger: This is going to be interesting. In a half hour, we’re going to try to cover the complexity of certification, but to do so in a way that’ll provide efficient moving from the development of a product to the marketplace. And this is an image of that process when we talk about certification. It starts with a list of documentation, knowledge, and an understanding of what applies to your product. And to go back just one step, it’s understanding what your standards and codes are that are driving that certification of your device. You’ll see the blue text here that represents work that you can do ahead of time to ease the process of certification when you actually present your product to the certifier. And this blue information here is a collection of documentation and evidence, if you will, that your product indeed complies with whatever standards might apply.
When we’re talking about energy storage systems, we’re looking primarily at UL9540 and UL9540A. And the code that drives that is going to be the NFPA 855. And so all these little blue boxes are critical as we prepare for our submission for certification. I will say that not having this information as you enter into certification results in significant time delays, and resource costs as far as samples needed, retesting, redesign, etc. And so what we’re going to do is go through some of these critical points here and discuss how we can make it easier, what we can do to prepare better so that we can be as efficient as possible. Just an example of how efficiency can help you, a typical certification of an energy storage system is going to take about … Well, it could be anywhere from 14 to 16 weeks. We’ve had some certifications that have taken more than a year to complete when they haven’t been prepared properly or they’ve had to be redesigned or test results have come back that have shown a lack of compliance with the UL9540 standard. So it’s critical that we move forward in a smart, organized, and efficient manner.
Talking about the beginning of this whole process is a knowledge of all of your codes and standards that apply to your energy storage system. It’s more than just a marketing scheme. It’s a legal requirement that devices are certified or proven safe to the authorities having jurisdictions or AHJs that approve those installations in one of the many, many, approximately 3000 different jurisdictions across the United States. Those codes are called out by the building codes. From there it calls out specific codes for electrical and fire safety. And those codes are adopted by local jurisdictions giving them some teeth. They’re legislated into effect, which gives them the legal basis as a requirement in order to install an energy storage system within a particular jurisdiction. And that legal basis is important. When we talk about certification, often companies see it as a necessary step, maybe even an obstacle that they have to achieve just to get their product to market, but it has more meaning than that.
It basically establishes a bar of entry for electrical devices that are suitable and acceptable for installation in specific jurisdictions. Now in the United States, we fortunately have the NFPA 70 or National Electrical Code that drives those requirements for safe electrical equipment, and that follows through along with the fire codes. So there is a legal basis for doing this. And yes, it can be time-consuming and it can be costly to go for certification, but it’s a necessary step in order to demonstrate that our products meet the industry standards for a level of safety. We do that basically as a consensus agreement within the industry on the requirements that are established both in the codes and in the standards. So it’s not simply a checkoff, it is an integral part of the safety of our electrical systems in the United States.
Meadows: Have you worked with teams in the past, maybe missed the upfront research around codes and standard requirements, and any examples of some of the impacts of that?
Flueckiger: Oh, absolutely. And oftentimes, as I say, they’re looked at as a necessary evil and bypassed, if you will, during the design phases. And then they come back later with those requirements and try to implement them at the last minute. And that often requires redesign of electrical circuits, repackaging of the product, et cetera. So it can be rather detrimental and costly to wait until the end. Good question.
So when we look at typical requirements that are included in the standards and the codes, we’ll be referencing UL9540 quite often here. They’re broken down into three major categories. We have your general construction requirements, and this is your design, right? These are the components that you’re putting together into a package that provides whatever functionality you are advertising or you’re stating that your energy storage system will provide. For example, you may have an input voltage of 480 volts. You might have an output voltage of 120 volts, whatever the case might be. You may have a certain amount of energy storage and, a number of batteries in your system that operates in a certain way to provide either backup power or to even maybe clean up a microgrid someplace in an industry, for example. But all of those general requirements for ratings rely on construction. Everything from the wire nuts holding two wires together or the connectors, all the way to the batteries and through the whole system with cooling systems potentially, and other systems that make up a part of the whole system that you’re looking to market.
And so those general construction requirements are basically material type requirements, component requirements, and mechanical requirements, and all of them are specified or delineated in UL9540. From the electrical side, we have a little bit more to deal with as well. For example, there are different circuits in battery energy storage systems. We may have an AC connection, we may have a DC set of batteries, we may have an inverter that is making that DC energy usable to the outside world, etc. All of those different circuits within the system have to be isolated from each other so that we don’t have somebody touching an antenna on a communications device and getting 480 volts at that antenna. We want to make sure they’re safe and that they’re separated. And one of the common things that’s missed along the way is that isolation. Spacing is a word that’s commonly used in electrical to define how we develop that isolation using air, for example, as an insulator.
We also might use materials that we talked about in the general construction requirements that have certain dielectric strengths to them so that we don’t have arc over within electrical circuits going from a high voltage, for example, to low voltage. So that’s an important part that has a huge impact on whether or not design is adequate, the design of your circuit boards, your interconnection of devices, etc. Whether or not they’re adequate to meet the requirements of the standards.
And then that follows by those individual components that we’re interconnecting. Whether it’s the batteries, the interconnection to a battery management system or a charging system to an inverter, to a fan, to a cooling system, or an HVAC system depending on the size of your system. But those components that make up the structure have to be relied upon in order to function safely. Not only to function safely but to produce the desired output that’s required of your energy storage system, whether it’s AC or DC, whether your input is AC or DC. All those components work together and are interconnected to produce the output and the functionality of your system.
Jama Connect Features in Five: Jama Connect Interchange – What Sets Us Apart
Learn how you can supercharge your systems development process! In this blog series, we’re pulling back the curtains to give you a look at a few of the powerful features in Jama Connect… in about five minutes.
In this Features in Five Integration Series video, Mario Maldari, Director of Solutions Architect at Jama Software – will demonstrate the Jama Connect to Jira integration via Jama Connect Interchange™.
VIDEO TRANSCRIPT
Mario Maldari: Hello. My name is Mario Maldari and I’m the Director of solution architecture here at Jama Software. Today we’ll be discussing the Jama Connect to Jira integration via our Jama Connect Interchange. We’ll be focusing on some specific advantages and differentiators that our integration provides for our clients. These advantages provide added value to our integration and allow our clients to better meet the needs of their process and ultimately speed up their development time with improved quality.
Let’s start in Jama Connect’s Live Trace Explorer™ and analyze gaps in coverage between system requirements and user stories. We can easily drill down into the trace view and identify gaps within our coverage. Once the gaps are identified, we can create new user stories to satisfy the coverage and improve our traceability score. Let’s go ahead and do that.
You’ll now see the gap in coverage for this particular item has been satisfied and is no longer being flagged by the tool. One unique aspect of our Jira integration is that a user can configure the synchronization interval. It can range from 15 seconds to up to 24 hours. This built-in flexibility allows for the integration to be customized to meet the needs of many different process flows and organizational needs.
Maldari: Navigating back to the user story that we just created in Jama Connect, you will now see an integration URL has been populated to the corresponding user story that was just created in Jira. In this case, the synchronization was quick, 15 seconds. Any subsequent update made to the item in Jira will synchronize and reflect back in Jama Connect with the same frequency. This way the two tools are always kept in sync and there is essentially no lag time.
Another major differentiation that our integration provides is the ability to create conditional rules for the integration. This allows users to be very specific about conditions for the creation and synchronization. They do not have to worry about the burden and overhead of sinking their entire project. They can be very specific regarding the conditions.
As an example, imagine a process that dictates that a new user story in Jama Connect will only get created in Jira when the Jira status in Jama Connect is set to do. This provides the requirements manager a level of process control where they can review the user stories and determine which ones and when they should be created in Jira.
In the Jama Connect Interchange configuration, if I navigate to the creation rule tab, I can see that there is a rule for user story creation. It’s configured to be triggered when the Jira status is set to do. I can add any number of rules and conditions based on my particular process flow. This allows for customization at a lower level and does not require the entire project to be synchronized.
Now let’s see how this works in Jama Connect. Utilizing Jama Connect’s List View, I can see which stories do not currently have a Jira status field set. I can review these and determine that I would like them to be created and synchronized to my Jira project. I can utilize the bulk edit feature and set the status for multiple items at a time.
Maldari: Because of the conditional rule I have set in my Jama Connect Interchange settings, all three of these user stories will be created in Jira and participate in synchronization. You will see that the Jira URL has now been populated and the corresponding item has been created in Jira. This is just an example, but you can imagine how this flexibility can be applied across your projects and specific to your attributes.
Another important distinction to note is how relationships in Jama Connect and Jira are maintained through the integration. This allows for a more holistic, contextual view of how requirements and user stories relate outside the direct linkage of the integration. For example, as a developer in Jira, I may want to have a reference to how the requirements relate to upstream and downstream links. By choosing a relationship type and performing a field mapping, I’m able to get this reference to display as web links in Jira. As a developer in Jira, this allows me to get a sense for the overall context of the traceability.
In this example, a developer will see a user story come into their queue, and they will have a web link reference to the upstream and downstream requirements in Jama Connect. This provides them with additional context while working on developing the user story. They are even able to navigate directly to those items in Jama Connect for further detail.
One of our core philosophies at Jama Software is making our tools easy to use and therefore easy to adopt and maintain. Our integration to Jira is no exception. The user interface is point and click and the field mapping has intelligence built in that does most of the work for you. It’s also worth noting that we do not require you to be a Jama Connect administrator to set up, configure, and utilize our integration. Jama Connect Interchange also provides detailed event logging, which allows IT teams to monitor events and current status of the configured integrations.
As you can see, the Jama Connect to Jira integration offers teams a robust set of features with configuration options to meet various organizational needs. This flexibility and ease of use combined with specific well-thought-out functionality sets us apart from other requirement tools in Jira integrations. To find out more about our Jama Connect to Jira integration through our Jama Connect Interchange, please reach out to your customer success manager or visit our website today at jamasoftware.com
Jama Software is always looking for news that would benefit and inform our industry partners. As such, we’ve curated a series of customer and industry spotlight articles that we found insightful. In this blog post, we share an article, sourced from MedTech Dive, titled “FDA Seeks Feedback On Health Equity for Medical Devices” – originally published on August 6, 2024, and written by Nick Paul Taylor.
FDA Seeks Feedback On Health Equity for Medical Devices
The agency plans to develop a framework for when devices should be evaluated in diverse populations to support marketing authorization.
Dive Brief:
The Food and Drug Administration is seeking feedback on health equity for medical devices to inform a potential regulatory approach to the topic, the agency said Monday. The paper is open for comment until Oct. 4.
In the discussion paper, the FDA shared its thinking on how sponsors can select study populations that adequately reflect the intended use for a particular medical device.
The discussion paper is part of a broader focus on health equity, one of the Center for Devices and Radiological Health’s strategic priorities, that also includes advice on diversity action plans.
The FDA framed the discussion paper as a response to the “urgent public health need for innovative technologies that help to reduce barriers to achieving health equity.” Seeking to address that need, the CDRH has committed to developing “a framework for when a device should be evaluated in diverse populations to support marketing authorization” as part of its strategic focus on health equity.
Running clinical trials that generate results consistent with how a device will perform in the real world is a way to improve health equity. Acknowledging that generating clinical data “can be complex,” the agency said it has focused its discussion paper on “a few important considerations that may be relevant for FDA’s evaluation of clinical evidence.”
The paper outlines factors that may help sponsors and investigators develop study objectives. To inform early trial design, the FDA recommends asking how the burden of disease and how the prognosis of a disease varies across a device’s intended use population. Sponsors can also ask how a device may introduce, exacerbate or mitigate differences in outcomes across the study population.
Another section of the document describes considerations related to the FDA’s evaluation of safety and effectiveness. The FDA reviews marketing authorization filings to determine whether there is reasonable assurance the device will be safe and effective in the intended population. As such, the agency is looking at whether clinical data “are generalizable to, and representative of, the intended use population.”
The FDA said it “considers it important to understand a sponsor’s rationale regarding the relevance of the provided clinical data to the intended use population.” The rationale could cover the processes used to select and enroll study populations, the agency said, or help the FDA understand difficulties a sponsor encountered when trying to obtain data from certain populations.
In the final section of the paper, the FDA describes example scenarios intended to prompt feedback on its assessment of the benefits and risks of devices. The section features a table that shows how the FDA may respond depending on whether the evidence suggests there are differences in patient populations and health outcomes, as well as whether the sponsor included specific populations in the clinical study.
In some cases, clinical trial sponsors may choose to design studies with “enriched” populations, meaning they intentionally enroll people from groups where differences are expected. The table suggests sponsors that fail to enrich study populations to reflect the intended use may face difficulties at the FDA if the available information suggests differences in patient populations.
The FDA said the absence of enriched populations may “introduce uncertainty” on the applicability of the data to the intended use population.
In this blog, we recap our webinar, “Application of Systems Engineering in Healthcare”. Click HERE to watch the entire webinar.
When it comes to healthcare, the first to market usually gains 80% of the market share, making development speed one of the most crucial aspects of success – or failure. That’s why many organizations are looking at systems engineering as a way of connecting needs to solutions.
In this webinar, Chris Unger of PracticalSE LLC and Vincent Balgos Director, Medical Device Solutions at Jama Software® have partnered up for an engaging webinar on the application of systems engineering in healthcare.
We invite you to join in as we delve into transformative role systems engineering is playing in the healthcare industry.
What to Expect:
1. The Power of Simplicity:
Discover how focusing on the basics, while maintaining world-class performance levels, can yield astonishing returns. We’ll show you how simplicity can be a game-changer in the complex world of healthcare systems engineering.
2. Market-Driven vs. Contract-Driven:
Intrigued by the difference between market-driven and contract-driven industries? We’ll explore how systems engineering varies in these two landscapes. Learn why “Market Driven” industries emphasize competitive value creation and use cases more than traditional requirements, and how this shift can redefine your approach in healthcare.
3. Striking the Perfect Balance:
Explore the ideal state of systems engineering in healthcare, often a harmonious blend of Agile, Lean Startup, and more traditional systems approaches. Uncover strategies to adapt, innovate, and succeed in this dynamic field.
Don’t miss this opportunity to gain a comprehensive understanding of how systems engineering can revolutionize healthcare. Whether you’re a seasoned professional or just beginning your journey in healthcare systems, this webinar promises valuable takeaways for all.
Below is an abbreviated transcript of our webinar.
Application of Systems Engineering in Healthcare
Chris Unger: We’re going to talk today about systems engineering in the medical industry, particularly medical device development. So the medical device industry faces several challenges. There’s clearly constant time pressure in developing and launching safe and effective products. We’ve got to be faster than the competition with better products. And as you can see from the statistics, this is a challenge. Part of the challenges in delivering things on time is the shifting regulatory landscape. I’m sure everyone’s aware of MDR. There’s software for medical devices. The FDA is going to think about redoing design controls next year. When we were at GE Healthcare, there were like 8,000 regulations we had to monitor. So it’s a very challenging and shifting regulatory landscape. Not only do you have to be compliant with regulations, but you have to ensure your device is safe. And so quality issues, safety and just keeping performance are key elements of delivering on time and that’s getting more and more expensive as you can see here, billions of dollars of financial risk of getting this wrong.
So to make all that harder, there’s a constant increase in complexity. When I first started, there were typical software development teams were 20 to 40 people. It’s now hundreds of people and lots of interactions. So additional things like AI, machine learning, or new technologies, really have to manage a lot of complexity inside of your devices. The organizational structure is getting more and more complex. There’s a heavy focus on acquisition, so you’re integrating new teams, new cultures, and geographically distributed development teams. So that makes it all challenging. So we’re going to talk about how systems engineering can help address some of these particular challenges.
Unger: As I mentioned, a key differentiator is getting to market faster. So the success of a program in a market-driven environment is basically profits. The first mover tends to collect the lion’s share of the profits. We typically have many customers. You don’t have a single customer marketing and product management tells you roughly what they think the competition will be and what differentiates versus in a contract-driven environment, success is satisfying the contract. So within GE Healthcare, the avionics and oil and gas businesses typically had a single customer. We would produce a floating city block to British Petroleum or Shell, go to the North Sea or the Caribbean and you had a contract and you delivered to that contract versus an engine, an aircraft engine, or a medical device, we deliver to the marketplace. We decide the timing, we decide the features.
So the stakeholders and market-driven are internal to the business and you can negotiate budget and time. If you get a really, really cool feature, you can take an extra month or quarter to develop it, versus in a contract-driven, it’s really fixed. So the challenges of market-driven and contract-driven are different. Contract-driven requirements are a key commitment. You’ve got to negotiate a formal design control versus within a market-driven environment they’re critical. You have to deliver validated requirements, but they’re definitely an internal business tool that helps communication across all the business functions.
So what’s the value of systems engineering in a market-driven industry? We basically turn the ambiguous needs that we get from product marketing or product management and turn them into clear and feasible solutions to be implemented by the hardware and software teams. The key value we produce is that those implementations seamlessly integrate into the customer’s workflow and work systems. So they work really well from day one, they reliably meet their needs. They work really well after five years and not just meet their needs, they delight the customer. We really want to deliver something that the customer enjoys using. So we have to make it work day one, we need to make it work day 50. We need to make it work for every single customer. So you have to deal with all the known variability of hardware and process. Every installation and every service event has to produce a uniform, high-quality, high-performing product.
So with those constraints, we want to optimize the business value. So when we have multiple options, marketing will tell us the customer value of these options. The implementation teams will tell us the delivery and product cost of those functions. The role of systems engineering is to make trade-offs between those and really optimize the business impact based on the cost of implementation. So we want to make sure the work done by those implementation teams is tied to the maximum market impact. And associated with that is managing technical risks. If you go down a path and it turns out to be infeasible, while it might’ve been nice if it worked, you just wasted a lot of that work. So that cost has to be scaled by risk.
In doing all those first four bullets, our key value is making sure design decisions are identified and closed predictably, and that the team acts with one voice. So decisions are framed, the options are agreed to, the decision criteria are agreed to and the final decision is closed and stays closed as stakeholders change. So once you have a frozen design, do you want to make sure that actually integrates easily and when you have integration or quality problems, they’re found early and resolved early. When you have time to react, it’s a lot easier to adjust your design in the first half of your program. It’s really hard when you find severe quality issues with a month left before shipping.
Unger: And so really winning products happen when systems thinkers are effective. So clearly there’s going to be a need for some systems engineering process thinkers, but they’re system thinkers across the entire program. And so we want to make sure that everybody’s involved in systems and that the creativity of the entire program is maximized. So getting specific to GE Healthcare, what is systems engineering at GE Healthcare? Well, we have the essential customer-perceived performance. So a lot of our programs are imaging, so we have the image quality. Still, we also have things like maternal-infant care where we deliver the right humidity and temperature around the neonate. In delivering that essential performance, you’ve got to make sure it’s safe and you’ve got to make sure you have regulatory compliance. And I mentioned we really want products that are easy to use and delight the customer. So usability is a critical part of systems engineering. In doing that, we make sure we define the right implementation requirements and the right reliability strategy, and that it can be installed and serviced properly.
So with that being the overall goal, how do we organize? Well, there are a lot of things that are common across all of our product teams. We do have common program milestones. We do have a common systems’ lifecycle. It’s basically the V-cycle with iteration and agile built in. What differs is that different product teams at GE Healthcare have different levels of safety hazards, so FDA risk. We go from anesthesia where you can easily kill somebody down to ultrasound, where it’s non-ionizing equipment, that’s the light handheld probes. You can’t pinch or crush anybody to even service software that has zero patient impact. There are also almost no risks for anybody and we respond to that by adjusting the process rigor so that the higher-risk safety risk modalities have higher process rigor.
Additionally, things vary across the world or we have different locations with different cultures and different sizes of organizations. We have many systems engineers across the company, but the SE team sizes vary from less than 10. In fact, we had some sites with maybe 10 engineers and the systems engineer was half a person to teams that had over a hundred systems engineers. The scale of the programs we work on is less than 10 engineers and months-long programs to many hundred as engineers applied to a program that might last three years and were based on technology developed over the prior decade. And you want some systems engineering thinking even during that basic research decade.
Unger: The organization goes from product centralized, it’s like the SE GM for that hundred engineering group where they all reported to a dotted or solid line, to decentralized in where that team of 10 with one or half a systems engineer, there the manager was a general engineering manager and did not have a lot of systems engineering experience. So I joke that if there is a way of organizing systems engineering, we have one of those within our group somewhere.
But how did we think about tailoring? And so this is a page I put together that was generalized that you might be able to use. Obviously, as I mentioned, higher technical risks including safety risks. One way of measuring that is how many risks there are in your hazard analysis. For things that are a higher risk we looked for a higher level of functional excellence, more process documentation, more process compliance, and higher rigor of the technical design reviews, and maybe more independent reviewers. Team experience. This is subjective to measure, but Joel Goldstein did a very nice study, from Carnegie Mellon, that the value of systems engineering increases with program complexity, but it decreases with a more experienced team if you have a small team that is experiencing the technology and the application, they can get by with less process rigor and while systems engineering excellence delivers some value, it delivers less value.
In this blog, we recap a section of our recent eBook, “Buyer’s Guide: Selecting a Requirements Management and Traceability Solution for Oil & Gas” – Click HERE to read the whole thing.
Buyer’s Guide: Selecting a Requirements Management and Traceability Solution for Oil & Gas
Use a Single Platform to Manage the Complexity of Standards Compliance and Product Development
Increasing complexity due to enhanced global and local regulatory scrutiny and rapidly evolving technological advancements causing cost and scheduled overruns is a top challenge for oil & gas companies in two key areas:
Standards & Practices Compliance
Oil and gas companies face numerous challenges in demonstrating compliance with the increasingly complex standards from local, state, and national regulatory agencies, as well as adhering to industry best practices and corporate guidelines. The first reason that problems arise is because these companies often attempt to manage critical processes using Word, Excel, or PDF document-based technology. While this manual approach may be adequate for simple projects, it fails as complexity and scale increase. The second reason is that companies continue to rely on legacy document management software such as SharePoint for tracing, storing, sharing, and retrieving standards documents, despite data siloes, lack of interoperability, constant changes, security threats, and limited collaboration and analysis.
As a result, companies have difficulty:
Reporting compliance to standards
Directly tracing standards to engineering requirements
Managing updates and implementing changes across concurrent projects
Collaborating effectively at a granular level around standards
Creating an audit trail around sign-off and implementation of standardsMitigating error-prone assessment of the impact of making changes to standards
Exploration and Delivery Product Development
Oil & gas companies that develop products such as undersea robots that support exploration and delivery face their own set of challenges. First, the use of document-based technology or siloed point software
solutions fails to keep up with the increasingly complex hardware and software systems and subsystems that must perform flawlessly together. Second, reliance on manual tools that are not optimized for managing a
complex development process often results in inefficient teamwork and late detection of defects. As a result, companies find themselves:
Relying on inefficient meetings or email communications involving internal, partner, and supplier teams to discuss, review, and approve product requirements and tests
Missing opportunities to detect defects and other issues early in the development process when it is typically easier and less costly to ensure product quality and performance
Incurring contractual penalties or losing revenue due to delayed availability or delivery of products
BOTTOM LINE
The increasing complexity and reliance on outdated tools make it difficult for oil & gas companies to efficiently comply with and show compliance to all relevant standards and/or develop products that meet internal or customer needs.
This Buyer’s Guide incorporates insights from Jama Software®’s more than 15 years of experience partnering with forward-thinking development teams and industry experts. We’ve designed a platform that helps oil & gas companies efficiently manage standards compliance and/or product development. This allows them to:
Take advantage of modern digital solutions
Manage standards with more confidence and efficiency
Manage exceptions throughout development to help create better-quality products
Collaborate in a single source of truth with a complete audit trail of activities
Avoid unnecessary costs and delays
Use these insights to better understand the challenges you’re up against and thoughtfully consider potential solutions. Plus, learn how to get the buy-in you need to undertake the kind of transformation necessary to succeed with complex projects.
Making the Case for Change
Jama Connect® helps oil & gas organizations to manage standards compliance and/or product development complexity by replacing documents and legacy tools with a single digital platform that provides a single source of
truth which is easily accessible by all stakeholders at any time. When standards and/or product requirements, architecture, V&V, and safety analyses are managed in a centralized platform, users benefit from a straightforward process and the business impact and value of the platform becomes clear across the organization. That makes getting executive buy-in easier.
If your company is not considering the importance of transitioning to a more modern, streamlined compliance and/or development process, time is not on your side. Failing to act quickly can leave your organization even further behind. But in order to see the value of the positive impact a system can have, stakeholders in an organization have to appreciate the challenges first.
This is where you come in. You can help quantify the problem within your organization and provide data to help make the case for change. Go through the exercises in the next section using data from your organization to identify your current situation and the size of the potential opportunity.
IDENTIFYING THE BUSINESS VALUE
Tools to Assess Compliance and Development Pain Points
Throughout the past decade of working with oil & gas among other organizations managing complex compliance and/or development projects, four common systems development pain points continuously arise for those who
have yet to transform their process.
We’ll provide context around the problems and share equations with examples to help you uncover the savings from a modern standards compliance and development solution. Remember to adjust the variables according to your company’s metrics to get a more precise estimate, and rethink how your team functions.
Improving any one of these four aspects of your process produces real savings. While the calculations on the following pages aren’t cumulative, they impact one another and can add up to significant value for your
organization.
This is the potential of using a modern digital platform. If realized, it can radically change your business and be the competitive edge you need in today’s market.
THE FOUR COMMON COMPLIANCE AND/OR DEVELOPMENT PAIN POINTS
Proactive and Live Traceability™ in Jama Connect® vs. Retroactive and Lagging Traceability in Excel
Incremental increases in product development complexity can lead to an exponential increase in the effort required from Engineering and Research and Development teams to keep up in a document-based (Word/Excel) environment.
PROBLEM
Our medical device & life sciences team has thousands of conversations per year with people interested in product and systems development improvements.
When we look at customer data over the two years prior to adopting Jama Connect, 83% of these organizations had their traceability maintained in an Excel-based matrix.
In this environment, all of the traceability components are maintained in a separate system (in other documents or tools). What we found was that this led to traceability being disconnected from the actual design. And in these cases, this delta was maintained and updated manually.
Updating the Excel-based traceability matrix to reflect changes or new artifacts is always a manual process.
Because this process is not automated, it takes a significant effort and is also highly error-prone. On a small scale, it can be manageable. But once a change is made, managing it effectively in this way becomes a significantly greater problem. We found that these events can exponentially increase both the level of effort needed to maintain traceability and the risk of a negative outcome or occurrence. We’ll provide some examples of this shortly.
As organizations make incremental improvements and changes, or grow and scale the company further, the manual, Excel-based process can become a major bottleneck. Because the level of effort associated with changing this workflow can seem too big of a task to complete, process improvements are de-prioritized. We see an exponential scale difference between an increase in complexity and the difficulty in managing traceability in Excel manually – tightening this bottleneck. Even a slight increase in complexity can lead to a high-severity issue/business impact/time waste.
As an example, let’s take a simpler medical device (a single-use catheter). Here are a few examples of how you could have the traceability schema established:
1-2 levels of requirements traceability (e.g., User Need > Product Requirement > Verification, or User Need > Validation)
Few items per level at each level in the hierarchy (e.g., 5 User Needs, 10 Product Requirements, and 1-2 Verifications/Validations)
As described in the example below, this means for simple products like a single-use catheter, there are roughly 225-440 possible trace relationships:
Let’s now imagine that we want to make a seemingly simple change and additional functionality to the device. We want to connect the catheter to a mobile application, so it can monitor its usage and analyze it for diagnostics purposes.
As this device is now part of a “system” with two major sub-system components, we break down the “product requirements” into two separate levels: “system” and “sub-system” requirements.
The complexity associated with managing requirements and maintaining traceability increases exponentially. For example, we’ve outlined a common example of a Class II system, where you can see that a 4x – 10x increase in the number of User Needs translates to a 15x – 24x increase in the total number of trace relationships that need to be managed.
An increase of 4x-10x in the number of User Needs translates to a 15-24x increase in the total number of trace relationships that need to be managed.
Here are a few common real-world scenarios where we see this complexity change occurring; A couple of examples of other complexity increases for a medical device organization:
A medical device with a newly added component and functionality (software, electrical, mechanical, hardware, etc.)
A research use application (RUO) that is reclassified as an IVD – becoming a diagnostic device
A medical device startup going from an R&D phase into having a product out in the market
A medical device being implemented into a larger system
Expanding into different markets and adhering to new regulations (US FDA, EU MDR, etc.)
Introducing product families, product lines, and variations to more effectively reuse existing components
The more this process/tool improvement is delayed, the higher the risk to the business.
We see this happening to medical device companies both big and small. Here are a few of our customers describing the problem.
“With Word and Excel, if something is changed and a link is broken, that document is gone and it’s literally floating around somewhere in the cloud without linkage to anything. This makes it very scary, especially from a quality or regulatory perspective. Our Word and Excel process evolved with the organization and therefore it was put together layer by layer, making it really hard to have the full depth of knowledge about how the quality system works.” – Rene Wenmekers, Director of Quality & Regulatory, Microsure
“We work in a highly regulated environment, and Microsure’s product has hundreds of requirements on system, subsystem unit, and component levels. And from a regulatory documentation standpoint, information is scattered.” – Robin Brounds, Software Team Lead, Microsure
“When we make changes in medical device development, they need to be reported to the notified body. And when that change hits the level of ‘significant change,’ the whole documentation set needs to be provided to the notified body to be reassessed on safety and efficacy. Every time a requirement changed, it needed to be updated across the whole documentation path. This was not sustainable using Word and Excel, and it was risky.” – Rene Wenmekers, Director of Quality & Regulatory, Microsure
Convergent Dental
While using Word and Excel, Convergent Dental found themselves tracking across multiple documents, all with their own trace matrix tables relating to different requirements. The fallout from this process is that even a single word or letter change in a low-level subsystem requirement led to updating corresponding requirements documents in their trace matrix tables. So, a single letter turns into not one change but potentially six changes across five different documents.
“We have a small team with a large amount of features and updates to perform on an ongoing basis. We all work really hard here, and there’s no option to be dead weight. Getting rid of that wasted time in Word and Excel and getting our test engineers back to work is the ultimate goal.” – Craig Woodmansee, Electrical Systems Engineer, Convergent Dental
NEGATIVE BUSINESS IMPACTS – of not changing
Wasted Time and Inefficient Processes
In a complex setting (working with a complex product, highly regulated environment, high-risk product, cross-functional teams working together, lots of different product variations, target customers/markets, etc.), this can be a person or a team’s full-time effort to keep up to date.
Time is being spent on people trying to find the right and most up-to-date documents
Sitting through review and alignment meetings with all stakeholders
Increased Risk of Negative Outcomes
This process relies on people constantly monitoring and updating each change. If a change goes unnoticed or people forget to update traceability, this gap is difficult to notice. If traceability gaps are noticed later during the product development lifecycle, there is a significant increase in the risk of one of the following negative events happening:
Releasing a faulty & untested product, quality compromise, product callbacks
Forcing organizations into late-stage changes that are costly to implement
Regulatory issues and audit findings (non-conformities, FDA warning letters, etc.)
Product not meeting the original requirements and customer/stakeholder needs
The expected outcome – backfilling documentation and traceability at the very end of the project.
Real outcome – Many issues/gaps went under the radar, leading to project delays, missed deadlines, or regulatory/quality issues:
Decreased Organization Maturity, Disconnected and Siloed Teams
Enforce the defined process – In a document-based environment, it’s close to impossible to monitor and enforce the defined process
Impact on employee tenure – Engineering and R&D are forced into manual documentation instead of actual design & development
Impact on talent acquisition – High-quality talent is more attracted to companies with proper tooling and processes in place
Communication and Transparency – Audit trails and change logs are often lost, hard to keep people accountable for changes
The solution to this problem is having integrated risk management with Live Traceability™ in Jama Connect®. Jama Connect will be the overarching system across all product development initiatives, bringing together all disciplines, making it significantly easier to visualize complex traceability hierarchies, replacing the manual effort needed to keep the documentation up to date, etc.
Jama Connect® brings comprehensive and detailed insights into your complex product, systems, and software development processes – automating the measurement of requirements traceability and coverage across disciplines and your organization’s toolchain.
This level of visibility helps eliminate rework due to out-of-date information; and the biggest fear for engineering leadership – that the greatest risks to a project are unseen until it is too late.
Understanding UN155 and Its Impact on Cybersecurity Management
In the ever-evolving landscape of cybersecurity, staying ahead of emerging threats and regulations is crucial for organizations worldwide. One such regulatory framework making waves in the cybersecurity community is UN155. This post aims to shed light on UN155 and its significance in cybersecurity management.
What is UN155?
UN155 is a regulatory framework established by the United Nations to enhance cybersecurity practices across various sectors. The framework sets forth comprehensive guidelines and standards for organizations to protect their information systems, data, and infrastructure from cyber threats. It emphasizes a proactive approach to cybersecurity, encouraging organizations to implement robust security measures and continuously monitor and adapt to the evolving threat landscape.
UN155 encompasses several critical components designed to strengthen cybersecurity management:
Risk Assessment and Management: Organizations are required to conduct regular risk assessments to identify potential vulnerabilities and threats. This involves evaluating the likelihood and impact of various cyber risks and implementing appropriate mitigation strategies.
Incident Response and Reporting: UN155 mandates the establishment of incident response plans to swiftly address and mitigate cybersecurity incidents. Organizations must also report significant incidents to relevant authorities, ensuring transparency and accountability.
Data Protection and Privacy: Protecting sensitive data is a cornerstone of UN155. Organizations must implement stringent data protection measures, including encryption, access controls, and data minimization, to safeguard personal and sensitive information.
Continuous Monitoring and Improvement: UN155 emphasizes the importance of continuous monitoring and improvement of cybersecurity practices. Organizations are encouraged to regularly review and update their security measures in response to new threats and vulnerabilities.
Training and Awareness: Educating employees about cybersecurity risks and best practices is crucial. UN155 requires organizations to conduct regular training and awareness programs to ensure that staff members are equipped to recognize and respond to cyber threats.
The implementation of UN155 has significant implications for cybersecurity management:
Enhanced Security Posture: By adhering to the guidelines set forth by UN155, organizations can significantly enhance their security posture. Proactive risk assessments, robust incident response plans, and continuous monitoring contribute to a more resilient cybersecurity framework.
Regulatory Compliance: Compliance with UN155 is not just a best practice; it is often a legal requirement. Organizations that fail to comply with the framework may face legal penalties, reputational damage, and financial losses.
Improved Incident Response: With established incident response plans, organizations can respond more effectively to cybersecurity incidents. This minimizes the impact of breaches and ensures a quicker recovery, reducing downtime and financial losses.
Increased Stakeholder Confidence: Demonstrating compliance with UN155 can enhance stakeholder confidence. Clients, partners, and investors are more likely to trust organizations that prioritize cybersecurity and adhere to recognized standards.
Global Harmonization: UN155 promotes a standardized approach to cybersecurity, fostering global harmonization of security practices. This is particularly important for multinational organizations operating in diverse regulatory environments.
UN155 represents a significant step forward in the global effort to enhance cybersecurity management. By adopting the framework’s guidelines and principles, organizations can bolster their defenses against cyber threats, ensure regulatory compliance, and build trust with stakeholders. As the cybersecurity landscape continues to evolve, frameworks like UN155 play a pivotal role in shaping a secure and resilient digital future.
Note: This article was drafted with the aid of AI. Additional content, edits for accuracy, and industry expertise by McKenzie Jonsson and Matt Mickle.
Leveraging Artificial Intelligence in Requirements Management
Artificial Intelligence (AI) is everywhere – from healthcare, to finance, to customer service – the integration of AI into various business processes has become a game-changer. One area where AI’s impact is particularly profound is in requirements management, a key process in product, systems, and software development.
Effective requirements management is crucial for the success of any project, ensuring that the final deliverables align with stakeholders’ needs and expectations. By leveraging AI, organizations can enhance their requirements management processes, leading to improved project outcomes, increased efficiency, and reduced risks.
Understanding Requirements Management
Requirements management involves the process of gathering, analyzing, documenting, and managing the needs and requirements of stakeholders throughout the project lifecycle. It ensures that all project requirements are clearly defined, tracked, and verified, minimizing the chances of misunderstandings and errors. Traditional requirements management, while effective, can be time-consuming and prone to human error. This is where AI steps in, offering innovative solutions to streamline and enhance the process.
Automated Requirements Elicitation: AI-powered tools can automatically gather requirements from various sources, such as emails, documents, and meeting notes. Natural Language Processing (NLP) algorithms can analyze text data to extract relevant information, saving time and ensuring that no critical requirements are overlooked.
Intelligent Document Analysis: AI can analyze and interpret complex documents, identifying key requirements and relationships between them. This capability allows for more accurate and comprehensive requirements documentation, reducing the risk of misinterpretation and gaps.
Enhanced Traceability and Impact Analysis: AI can improve traceability by automatically linking requirements to their corresponding design elements, test cases, and implementation tasks. In the event of a change in requirements, AI can quickly assess the impact on the project, providing valuable insights for decision-making.
Predictive Analytics and Risk Management: AI can analyze historical project data to identify patterns and predict potential risks and challenges. By leveraging predictive analytics, organizations can proactively address issues before they escalate, ensuring smoother project execution.
Real-time Collaboration and Feedback: AI-powered collaboration tools facilitate real-time communication and feedback among stakeholders, ensuring that everyone is on the same page. These tools can also provide intelligent suggestions and recommendations, enhancing the quality of discussions and decisions.
Continuous Learning and Improvement: AI systems can continuously learn from past projects and refine their algorithms, leading to ongoing improvements in requirements management processes. This continuous learning capability ensures that organizations stay ahead of the curve and adapt to evolving project needs.
Case Study: AI in Action
Consider a software development company implementing AI in their requirements management process. By using an AI-driven tool, the company automated the elicitation of requirements from client emails and documents. The tool identified and categorized requirements, linking them to specific project tasks and design elements. This automation reduced the time spent on manual documentation by 50%, allowing the team to focus on higher-value activities.
Moreover, the AI tool provided real-time impact analysis when changes were requested by the client. It quickly identified affected areas, allowing the team to make informed decisions and adjust project plans accordingly. The result was a more agile and responsive project management process, leading to increased client satisfaction and successful project delivery.
Conclusion
The integration of Artificial Intelligence in requirements management represents a significant advancement in how organizations handle complex projects. By automating routine tasks, enhancing traceability, and providing predictive insights, AI empowers teams to deliver high-quality projects on time and within budget. As AI technology continues to evolve, its impact on requirements management will only grow, offering even more innovative solutions to meet the challenges of tomorrow’s projects.
Organizations that embrace AI in their requirements management processes will not only improve their project outcomes but also gain a competitive edge in an increasingly dynamic and demanding business environment.
Note: This article was drafted with the aid of AI. Additional content, edits for accuracy, and industry expertise by McKenzie Jonsson and Mario Maldari.
Unleash the Full Potential of Your Requirements with Jama Connect Advisor™
Successful product delivery starts with precise, well-written requirements that align design, software, and hardware teams with clear project goals. Jama Connect Advisor™ is a cutting-edge tool powered by natural language processing, guiding system engineers and product developers to create effective requirement specifications based on INCOSE and EARS standards.
Teams are using Jama Connect Advisor to:
Improve the quality and usability of requirements
Save time authoring, reviewing, and updating requirement statements
Continuously enhance team requirement authoring skills with regular use
Deliver programs and projects on time and on budget with long-term success
How to Overcome Development Challenges: Collaboration & Alignment in Complex Product, Systems, and Software Development
In the dynamic world of product and software development, effective collaboration is crucial for success. However, teams often face significant challenges that can hinder progress and innovation. Understanding these challenges and finding robust solutions is essential for seamless and productive teamwork. One such solution is Jama Connect®, a powerful platform designed to enhance collaboration and streamline development processes. In this blog post, we will explore the common challenges in collaboration and how Jama Connect can help overcome them.
Common Collaboration Challenges in Product and Software Development
1: Communication Gaps
Problem: Miscommunication or lack of communication can lead to misunderstandings, errors, and delays. With team members often working remotely or across different time zones, keeping everyone on the same page can be difficult.
Solution: Jama Connect provides a centralized communication platform where all team members can access the latest information, updates, and discussions. This helps ensure everyone is aligned and informed, reducing the risk of miscommunication.
2: Fragmented Documentation
Problem: Keeping track of documentation and ensuring it is up-to-date can be challenging, especially when it is spread across multiple tools and platforms. This fragmentation can lead to confusion and inconsistencies.
Solution: Jama Connect offers a unified repository for all project documentation. Teams can create, store, and manage documents in one place, making it easier to maintain consistency and quickly find the necessary information.
“Jama Connect has a very powerful user interface, traceability, coverage, revision history, and review collaboration. The verification and validation plan runs are great for program tests traceability.” – Ander Solorzano, Principal Systems Engineer, Astrobotic
3: Requirement Mismanagement
Problem: Managing requirements effectively is critical in product development, but it can be challenging to track changes and ensure all requirements are met. Mismanagement can result in overlooked or unmet requirements, leading to product issues and customer dissatisfaction.
Solution: Jama Connect’s requirements management capabilities allow teams to define, track, and manage requirements throughout the development lifecycle. This ensures that all requirements are documented, tracked, and verified, reducing the risk of oversight.
Problem: Limited visibility into the project’s progress and status can hinder decision-making and delay issue resolution. Team members and stakeholders need a clear view of the project’s health to make informed decisions.
Solution: Jama Connect provides real-time visibility into project status, progress, and potential roadblocks. Dashboards and reporting tools offer insights into key metrics, helping teams identify issues early and take corrective action.
5: Inefficient Review and Approval Processes
Problem: Traditional review and approval processes can be time-consuming and prone to bottlenecks. Delays in getting approvals can slow down the entire development process.
Solution: Jama Connect streamlines review and approval workflows, allowing teams to collaborate efficiently. Automated notifications and reminders ensure that reviews and approvals are completed promptly, keeping the project on track.
“Prior to selecting Jama Connect, we had experience with other platforms, we had multiple trials and settled on Jama Connect because of how innovative it is. Especially in areas of Collaboration, Ease of Use, and the ability to configure and integrate with other lifecycle tools.” – Sr Principal Business Analyst and Enterprise Architect, Biotechnology Company
How Jama Connect® Facilitates Effective Collaboration
1: Centralized Platform
Jama Connect serves as a single source of truth for all project-related information, enabling seamless collaboration. Team members can easily access and share information, ensuring everyone is working with the latest data.
2: Real-Time Collaboration
With real-time collaboration features, Jama Connect allows team members to work together on documents, requirements, and tasks simultaneously. This reduces delays and fosters a more collaborative environment.
3: Traceability
Jama Connect ensures end-to-end traceability, allowing teams to trace requirements, design, implementation, and testing back to the original source. This traceability helps maintain alignment and accountability throughout the development process.
“Jama Connect is a centralized platform for managing requirements, fostering collaboration, and ensuring alignment across global teams.” – Sekhar Ghandikota, Senior Engineer, Ford Motor
The platform offers customizable workflows that can be tailored to fit the specific needs of the team. This flexibility ensures that teams can implement processes that work best for them, improving efficiency and collaboration.
5: Integration with Other Tools
Jama Connect integrates seamlessly with other popular development tools, such as Jira, Slack, and GitHub. This integration helps teams maintain a connected ecosystem, reducing the need to switch between different tools and platforms.
“Jama Connect offers robust collaboration features, traceability and API endpoints. Throughout the development process, efficiency is gained.” – Sekhar Ghandikota, Senior Engineer, Ford Motor
Conclusion
Collaboration is the cornerstone of successful product and software development, but it comes with its own set of challenges. Jama Connect addresses these challenges by providing a robust platform that enhances communication, documentation, requirement management, visibility, and workflows. By leveraging Jama Connect, teams can overcome collaboration hurdles, streamline their development processes, and ultimately deliver high-quality products.
Are you ready to transform your collaboration and take your product development to the next level? Explore Jama Connect today and see how it can help your team achieve its goals.
Note: This article was drafted with the aid of AI. Additional content, edits for accuracy, and industry expertise by Kenzie Jonsson.
![[Webinar Recap] Application of Systems Engineering in Healthcare](https://www.jamasoftware.com/media/2023/10/Application-of-Systems-Engineering-in-Healthcare-1.png "[Webinar Recap] Application of Systems Engineering in Healthcare")
In this environment, all of the traceability components are maintained in a separate system (in other documents or tools). What we found was that this led to traceability being disconnected from the actual design. And in these cases, this delta was maintained and updated manually.
