Responsible AI in Regulated Industries Explained

Image of a woman acting as the host of this webinar about responsible AI.

In this blog, we recap our recent webinar, “Empowering Complex Development with Responsible AI”

[Webinar Recap] Empowering Complex Development with Responsible AI

AI is reshaping the way teams manage complexity in product development — but in regulated industries like aerospace, defense, automotive, and medical devices, responsible AI adoption is critical.

In this webinar, our expert Katie Huckett will explore how AI is transforming modern development processes, the ethical considerations of AI adoption, and the latest AI-driven innovations in Jama Connect^®.

What You’ll Learn:

How AI is streamlining requirements management with automation and predictive insights
Best practices for responsible AI adoption in compliance-heavy industries
Jama Connect’s AI-powered features that enhance requirements quality, traceability, and risk mitigation
Our Amazon Web Services (AWS) partnership and how it enables secure, scalable AI-driven workflows

Below is an abbreviated transcript of our webinar.

Katie Huckett: Welcome everyone. Thank you for joining us today for this exciting webinar. Empowering Complex Development with Responsible AI. In today’s rapidly evolving world, industries like aerospace, defense, automotive, medical devices and financial services are facing unprecedented challenges. As products and systems become increasingly complex, ensuring regulatory compliance requirement clarity, and test coverage has never been more critical. Artificial intelligence is transforming the way we approach these challenges, offering powerful tools to automate tedious processes, enhance decision-making and improve requirement quality. But with AI’s potential comes a responsibility to ensure fairness, transparency, security, and compliance.

By the end of this session, you’ll gain a deeper understanding of how AI can accelerate development while maintaining accuracy, compliance, and ethical integrity. I’ll be your speaker today. My name is Katie Huckett. I bring over 15 years of product management experience and enterprise solutions. I’m one of Jama Software’s senior product managers, where I play a key role in bringing Jama Connect Advisor™ to life, as well as our AI strategy and roadmap for the future evolution of our AI offerings. In this webinar, we’ll explore the role of AI and complex product development in regulated industries. What is responsible AI and how is it used? Our partnership with Amazon Web Services, including our commitment to AI governance and security, ensuring AI aligns with industry standards and best practices.

Jama Connect’s AI-driven innovations leveraging AWS AI tools and discuss how AI enhances traceability, compliance verification, and validation while keeping human expertise in control. We’ll have a brief question and answer portion before we conclude today. Let’s dive in. Starting with Jama Software and our role in the product development ecosystem. Our vision and our purpose is to make sure that innovators can efficiently achieve success, and as you’ll see from today’s discussion, that’s really at the core of what drove our introduction for Jama Connect Advisor. From a broader solution standpoint, Jama is the number one requirements management provider in the marketplace.

We help teams with requirements management and product development through live traceability that also spans not only requirements, but the validation and verification components on the test side, risk management, and other key data that drives those processes forward. The value that we hope these innovative organizations or customers derive is really focused around things like cycle time reduction, helping speed time to market, enabling through live traceability, the ability to gain visibility and control over the organization’s product development processes, and really drive a tremendous amount of value and ultimately ensure compliance and risk management.

RELATED: Jama Connect Advisor™ Datasheet

Huckett: As far as organizations that we work with, we span medical device, automotive, industrial machinery, software. This is just a sampling of customers that we have the pleasure to partner with. We have over 800 customers globally. These organizations span from smaller startup companies to large global enterprises. So with that very short intro to Jama Software, I would now like to review some of the complexity and challenges that we see today in product development and of course to introduce you to Jama Connect Advisor. Today’s systems have become much more complex and the emergence of the system of systems architecture has become the dominant approach for devices in all sectors, whether it’s aerospace, automotive, medical, and even consumer products.

The system of systems is actually a collection of independent subsystems that are integrated into larger systems and deliver the unique capabilities required by users. The challenges that is difficult to predict accurate, predictable models of all emergent behaviors. So Google’s systems of systems performance is difficult to design. That leads to testing and verification verifying upgrades to existing systems of systems is difficult and expensive as well, which is hard to scale. These are some of the factors that led us to think about how we can help. In today’s landscape, complex product and software development and manufacturing require organizations to balance innovation, compliance, and efficiency.

Industries today face increasing regulatory scrutiny, rising product complexity, and pressure to accelerate time to market. Let’s take a look at the role of AI in modern requirements’ management. Modern products are no longer purely mechanical. They integrate hardware, software, AI, and cybersecurity. For an example, a self-driving car must integrate advanced driver assistance systems, cloud connectivity, AI-powered decision-making, and functional safety. AI-driven requirement validation ensures that specifications are complete, testable, and free from ambiguities, preventing integration failures later. There’s also regulatory pressures and compliance challenges.

Industries such as aerospace, medical device, automotive, again, must comply with strict safety and cybersecurity standards. AI can map requirements to relevant regulations, ensuring compliance is automated and continuously monitored. There’s also a push for faster development cycles. Traditional product life cycles are shrinking due to market competition and innovative demands. AI-powered predictive impact analysis helps developers understand the effects of changes instantly reducing rework and speeding up time to market. There’s also a huge burden of manual processes. Many organizations still rely on spreadsheets, disconnected documents, and siloed teams to manage requirements.

AI-powered natural language processing can automatically detect inconsistencies, duplications, and incomplete requirements, improving efficiency. As well as a large need for better communication, collaboration, and visibility. So as teams become more distributed, cross-functional collaboration is more difficult. AI-powered requirement linking and automated traceability ensure that all stakeholders have real-time insights into requirement changes. AI is transforming requirements management by automating manual error-prone processes such as automatic requirement classification. AI can analyze text and categorize requirements by priority, risk, or compliance relevance.

RELATED: The Essential Guide to Requirements Management and Traceability: Leveraging Artificial Intelligence in Modern Product and Systems Development

Huckett: There’s also duplicate and conflict detection. AI can identify duplicate or conflicting requirements, reducing errors. There’s also a need for enhancing accuracy and requirements traceability, so AI can help ensure that every requirement is properly linked to tests, regulations, and design components, reducing compliance risk. We also have improved decision-making through predictive analytics. AI enables teams to make data-driven decisions faster and with more confidence. As product complexity increases, traditional requirement management cannot scale effectively. AI-driven tools and Jama Connect will automate compliance, enhance traceability, and improve decision-making, helping industries stay ahead of regulatory and market demands.

AI is not replacing engineers, it is augmenting their capabilities, enabling teams to develop safer, more compliant, and more innovative products faster. The rapid adoption of AI-driven tools in highly regulated industries brings immense opportunities for efficiency, automation, and innovation. However, AI also introduces ethical regulatory and governance challenges that must be addressed to ensure fairness, compliance, and trustworthiness in AI-driven decision-making. This section will explore the principles of responsible AI, regulatory hurdles, best practices for governance, and real-world examples of AI successes and failures.

As AI systems increasingly influence safety-critical industries, organizations must ensure their AI solutions adhere to ethical AI practices to prevent bias, misinformation, and harm. Responsible AI encompasses the following core principles. Fairness, which is your AI models must be free from bias to ensure equitable outcomes. Transparency, AI decision-making should be explainable and understandable. Accountability, organizations must take responsibility for AI-driven decisions. And of course, privacy and security, AI systems must protect sensitive data and prevent misuse.

AI is used in mission-critical applications such as aerospace and defense, where we have autonomous drones and AI assistance surveillance. In automotive, we have AI-powered advanced driver assistance systems, and in medical devices we have AI-driven diagnostics and robotic surgery. If AI models are not carefully designed, tested, and governed, unintended biases, errors, or security vulnerabilities could lead to catastrophic consequences. Not to mention the complexity of navigating regulations in a variety of industries. So for example, in aerospace and defense, AI and avionics and defense systems must comply with several standards.

Medical devices, we have the FDA Good Machine Learning Practices sets guidelines for AI-driven medical software. And in the automotive industry, AI in autonomous vehicles must meet ISO 21448 and ISO 26262. AI relies on large data sets, often containing sensitive information. Organizations must ensure compliance with regulations such as GDPR, which protects EU citizens data from misuse. There’s HIPAA, which governs healthcare AI solutions in the US and the California Consumer Privacy Act, which regulates AI handling consumer data. To build trustworthy and responsible AI organizations need robust governance frameworks that ensure AI models remain fair, explainable, and compliant over time.

Organizations should implement structured AI governance frameworks such as NIST AI Risk Management Framework, which provides a structured approach for assessing AI risks. In the way of the ISO/IEC 42001, which is an AI management systems standards, which establishes best practices for AI governance. We also have the IEEE Ethically Aligned Design, which focuses on human-centric AI development. For example, a medical device manufacturer developing an AI-powered diagnostic tool can use the NIST AI Risk Management Framework to ensure the model’s explainability, fairness, and reliability.

RELATED: Navigating AI Safety with ISO 8800: Requirements Management Best Practices

Huckett: AI models degrade over time as real-world conditions change, so organizations must continuously monitor AI performance for bias and drift, gather feedback from domain experts, users, and regulatory bodies, and implement AI auditing mechanisms to detect unintended outcomes. Let’s talk about a few real-world examples. So we have here a couple of case studies of successful AI adoption. So AI-assisted radiology tools have improved early cancer detection by 30% leading to better patient outcomes. One of the key factors in the success was the AI models were trained on large diverse datasets and continuously validated by human radiologists.

Another example, AI-driven predictive maintenance in commercial aircraft has reduced downtime by 25%, saving airlines millions in operational costs. One of their key factors for success was AI predictions were cross-validated with human engineering teams before implementation. And here we have a few real-world examples that are really cautionary tales in AI adoption. In 2018, an AI-driven hiring system was found to be biased against female candidates because it had been trained primarily on resumes for male applicants. So the lesson learned here is that AI models inherit bias from historical data emphasizing the need for bias audits and fairness checks.

And several self-driving car crashes occurred due to AI, misidentifying obstacles, pedestrians, or unexpected road conditions. So their lesson learned is that AI models require continuous real-world testing and human oversight to handle edge cases effectively. By proactively managing AI risks, organizations can unlock AI’s full potential while ensuring safety, fairness, and compliance in their industries. At Jama Software, we are committed to delivering responsible, scalable, and secure AI solutions to help our customers manage complexity in highly regulated industries. By partnering with AWS, we ensure that AI and Jama Connect is secure, responsible, and purpose-built for the industries we serve.

Enhancing efficiency without compromising compliance. AWS is at the forefront of AI and machine learning innovation, offering scalable, secure, and cutting-edge AI solutions that power businesses across many different industries. With industry-leading AI services, AWS enables organizations to automate complex tasks, extract insights from data and enhance decision-making with state-of-the-art machine learning models. From natural language processing to generative AI and predictive analytics, AWS provides flexible enterprise-grade AI tools that drive efficiency, improve accuracy, and accelerate product development, all while ensuring security, compliance, and responsible AI governance.

By leveraging AWS AI, companies can turn vast amounts of data into actionable intelligence, unlocking new possibilities for innovation and transforming the way they work. AWS’s AI and machine learning solutions are designed to scale effortlessly with business needs, supporting everything from small AI experiments to large enterprise applications. AWS AI services comply with industry-specific security standards, including HIPAA for healthcare applications, GDPR and CCPA for data privacy, ISO 27001, SOC 2, and FedRAMP for cloud security and governance. Amazon Web Services is deeply committed to the responsible development and deployment of artificial intelligence.

To watch the entire webinar, visit:

Empowering Complex Development with Responsible AI

Author
Recent Posts

Katie Huckett

Senior Product Manager at Jama Software

One of the things I love about product management and customer experience is the craft of it. Finally, a discipline that allows me to draw from every aspect of my eclectically diverse skill sets and interests.

With 14+ years of Enterprise SaaS experience, I have come across many complex feature requests and use cases. Never one to back down from a good challenge, I enjoy pushing the limits of what is possible while ensuring the big picture aligns with the overall product vision and strategy.

Experienced in:
• Developing product vision, strategy, and goals
• Develop and validate personas
• Creating and maintaining product roadmap
• Gathering product requirements (from user research, market research, user feedback, internal stakeholders, etc.)
• Creating and maintaining product backlog including prioritization
• Leading cross-functional teams across multiple office locations
• Presenting to stakeholders and obtaining buy-in
• Technical, analytical, and strategic thinking
• Managing product lifecycle from concept to end-of-life
• Product marketing to both customers and end-users

Specialties: User Experience, Customer Experience, User Research, Market Research, User Stories, Graphic Design, Data Analysis, Enterprise SaaS, User Champion, Product Lifecycle, Product Strategy, Goal Creation and Tracking, Technical Communication, Value Proposition, Product Marketing, Product Positioning, Training, Product Demonstration, QA Testing, Client Engagement, Business Analysis, Go-to-Market Strategy, Customer Segmentation Analysis, Sales Engineering, Customer Success, Agile Software Development, SCRUM, Value Selling

Tools and Languages: PhotoShop, InDesign, Illustrator, CRM, CMS, Aha!, Trello, InVision, Camtasia, Google Analytics, Zendesk, Sketchbook, SharePoint, Eloqua, Jira, Slack, Scrivner, HTML (some), Markdown