Requirements & Requirements Management

How to Plan for Large Language Model (LLM) Adoption Within Your Engineering Organization

By |

Large Language Model (LLM) Image

How to Plan for Large Language Model (LLM) Adoption Within Your Engineering Organization

The initial free and unprotected access to ChatGPT (the most well-known Large Language Model) has led some individual engineers to try out the tool by using company owned trade secrets and intellectual property (IP) as prompts. The predictable result has been IP leakage with numerous high-profile examples, including at Samsung. As a result, many companies, including Apple, have banned internal use of the technology outright. Additional risks are just starting to be understood given the lack of consent provided by the actual owners of content that was used to train the LLMs. This leaves the concept of ownership of LLM output, and the ability to protect intellectual property that includes LLM output in question and legal experts are advising caution. Clearly, it will take some time for legal frameworks and precedents to be established for the use of LLMs in product development and for enterprise-class integrations to be developed to LLMs that at properly allow for company level standards and governance of IP. Numerous lawsuits, such as Clarkson v OpenAI, are now underway alleging all of the data to train the LLMs was obtained without consent or renumeration and violates copyright law.

RELATED: Best Practices Guide to Requirements & Requirements Management

Clearly, it will take some time for legal frameworks and precedents to be established for the use of LLMs in product development and for enterprise-class integrations to be developed to LLMs that properly allow for company level standards and governance of IP. Given the risks and unresolved legal questions LLMs pose for product development, how should an engineering organization plan an adoption path to achieve potential benefits from intelligent assistance for engineering tasks?

The guidance we provide our clients is to focus on the following three areas, ranked in order of greatest benefit from intelligent assistance:

  1. Improve requirements quality – Poorly specified requirements account for up to 64% of defects and are the costliest ones to correct. The International Council on Systems Engineering (INCOSE) and the Easy Approach to Requirements Syntax (EARS) have established best practices for requirements specification and unfortunately, LLMs are trained on publicly available requirements content that is rife with all the most common errors addressed by INCOSE and EARS. The best intelligent assistant to improve requirement quality is a Natural Language Processing (NLP) approach that analyzes requirements against INCOSE and EARS best practices and recommends improvements – which is exactly what Jama Connect Advisor™ does. Jama Connect Advisor protects all IP and engineers learn how to write better requirements through intelligent guidance.
  2. Manage by exception – The engineering function is one of the last in the enterprise to be managed through data. The engineering process is often fragmented across teams and tools which leads to late identification of cross-discipline issues that result in defects, delays, cost overruns, and recalls. Jama Connect® intelligently solves this problem through Live Traceability™ which automatically syncs data across best-of-breed tools and tracks engineering progress against the chosen development model (e.g., V-model) to identify issues as early as possible and thereby reduce the risk of defects, delays, cost overruns, and recalls.
  3. Increase engineer productivity – The biggest drains on engineering productivity are most commonly integration meetings and rework. Jama Connect’s Live Traceability intelligently alerts teams to impactful change from other engineering disciplines. Live Traceability eliminates the need for time-consuming and mind-numbing integration meetings and is proven to reduce rework based on our groundbreaking benchmarking study. Further productivity gains can be achieved by leveraging LLMs for requirement decomposition and we intend to be one of the first to market with an enterprise-class solution that protects IP and enables company standards.

RELATED: Best Practices Guide for Writing Requirements

To get started with intelligent assistance, learn how best to improve requirements quality across your engineering organization with the NLP application of EARS and INCOSE best practices.


Marc Osofsky
Latest posts by Marc Osofsky (see all)