Reviews

Ensure Product Quality with These Review Process Best Practices

By |

Review Process Best Practices

Introduction

Reviews play a key role in successful product and systems development, helping to ensure the new project meets stakeholder, market, and compliance requirements. Peer and approval review processes enable organizations to both iterate and innovate quickly while providing a dedicated process to apply appropriate rigor for final reviews. In addition, integrating item workflow with approval reviews can eliminate manual processes and reduce human error.

In this blog post, we examine a generic approach to reviews and review coverage, independent of the application used. In the second half of this post, we’ll look into the way Jama Connect® can be used to support the review processes described in the first half.

Defining the Scope of a Review

The reviews discussed in this post are in reference to informal (aka ‘peer’) reviews, leading up to formal (aka ‘approval’) reviews.

For medical device manufacturers, the first part can also be applied to their existing document management (quality) system, where their formal review and sign-off are recorded for quality bodies, like the FDA (Food and Drug Administration).

Reviews Play a Key Role in Successful Product Development.

Reviews are an essential part of any product quality process, comparable to testing the product. Document reviews take place in the engineering phases of a product lifecycle when there is no product (parts) that can be subjected to tests yet.

Like testing, a review by itself can never cover 100% of all issues that are in the item under review. And as with testing, a review process is defined at multiple levels, each with a different focus, attention, or goal to ensure the highest degree of coverage that can be achieved.

Review Coverage

There are different ways to ensure your reviews get enough overlapping coverage to catch the issues:

  1. The reviewers you invite
  2. The focus you give each reviewer
  3. The goal you set per review

Related: Leveraging Peer and Approval Workflows to Optimize Your Peer and Approval Process

Who Should Be Included In A Review:

The documents you write, you do not write for your own benefit. Documents created form a means for knowledge transfer from the author to the next person in the product lifecycle. The information within the document should therefore be understandable for its intended audience (readers) and users (appliers).

Since the documents written are related to the level these documents describe in the system’s engineering V-model, it can quickly be understood which process roles need to be invited as reviewers:

Process roles depicted in the diagram above:

At any one level within the applied information architecture, there are at least two next levels that need to be invited, to make sure these following levels understand the information provided through those documents: the next level downstream (decomposition and detailing) and the same level across testing.

It is also important to know if the author has understood the documentation that was provided by the process role in the product engineering phase before theirs (i.e., the upstream level).

Inviting colleagues in the same field, or at the same product level as other product lines (i.e., fellow Subject Matter Experts, or SMEs), is a good way to ensure those reviewers understand and evaluate the documents against company standards to ensure a consistent quality of deliverables. It also ensures that specific product knowledge is spread throughout the company when involving fellow SMEs outside their own project.

Because of their differences in process roles, each reviewer will naturally focus on the information that is important for them to understand in order to be addressed in accordance with the process they manage and maintain.

For example:

The System Requirements Specification (SRS) should be reviewed by the person(s) responsible for writing:

  1. The stakeholder/customer requirements (upstream; correct interpretation and coverage of that level’s ‘asks’)
  2. The Subsystem Requirements Specification (downstream; is it understandable, unambiguous, and specific enough to be able to ‘answer’ the ‘ask’ that the SRS has for them)
  3. The System Acceptance Test Plan (across; is it understandable, unambiguous, and specific enough to be able to be tested)
  4. SMEs on topics described and/or referenced in your document (quality, sanity, and completeness)

General Guideline: There should be at minimum three, but preferably four reviewers in any review.

Tips for Conducting a Successful Review:

  • When parts of the system will be developed and provided by a third party (e.g., subcontractor), include that subcontractor.
  • When reviewing the product needs, or tests to validate these, of a specific customer, include that customer (or meaningful representatives).
  • Although there does not appear to be any relation in the various test levels, it is still interesting to invite testers from those other levels, as they provide different insights, they’ve applied for similar tests defined at their level.

Assign Focus Areas to Each Reviewer

Even though reviewers get invited depending on their process role, related to the document under review, it is also important to assign focus areas to each reviewer to ensure not all reviewers comment on the same spelling error, which usually is only a minor inconvenience that takes away the focus on the important issues.

Simply mentioning what their expected contribution is will already achieve such a focus, i.e.:

  • A reviewer invited because of their upstream relation to your document should be assigned to look at the correct interpretation and coverage of their provided input.
  • A reviewer invited because of their downstream relation to your document should be assigned to check if they understand your document and if it is unambiguous and specific enough for them to further decompose and detail.
  • A reviewer invited because of their ‘across’ relation to your document should be assigned to check if they understand your document and if it is unambiguous and specific enough for them to define test cases and/or test approaches.
  • A reviewer is invited because they’re an SME to ensure the quality, sanity, and completeness of your document.
  • Finally, only assign one of them to also check for grammar and spelling errors. This (simple) assignment will ensure all other reviewers won’t remark on them, as someone else is already tasked with that and it keeps focus on their own assigned areas.

Informal Reviews Leading up to Formal Reviews

Not every review carries the same weight. Not every review has a formal context and thus doesn’t require the involvement of authorized (senior) colleagues, or managers, to formally sign off on the document.

Reviewers tasked with formal sign-off of your document, usually have yet another focus than reviewers tasked with evaluating the quality and completeness of its content. Combining these two types of reviewers will ensure either role will question their contribution to the review while the other role is addressing their found issue. Therefore, it’s advisable to have a two-level approach to reviewing your documents:

  1. Evaluate the content regarding any product, service, or inconsistencies.
  2. Evaluate the content regarding any business contextual (i.e., legal, contractual) aspects.

The review process and approach as described above have the quality and completeness of the content in mind, to allow the formal approval reviews to be nothing more than an administrative necessity; The subsequent approval of a document becomes a hammer piece.

“Review Center is facilitating communication. It has ensured a shared view of the world and agreement from all stakeholders. There are no surprises anymore. Jama Connect enables us to review documents and make decisions easily with everyone coming to a shared conclusion.” Craig Grocott Head of Systems Engineering, Teledyne e2v

Related Customer Story: With Jama Connect®, TELEDYNE e2v Improves Communication and Reduces Risk

Review Process in Jama Connect

The review process and approach described above are independent of any application supporting your review process and/or approach.

Jama Connect supports the above-described review process and approach. It can even provide more focus.

Divide and Conquer

Atomic Nature of Jama Connect Items

All Items, Components, Sets, and Folders are atomic.

Although Jama Connect doesn’t really have the concept of ‘documents’, most customers use their ‘Sets of’ and ‘Folders’ to represent the content and respectively (chapter/paragraph) structuring of their documents. As with documents where chapters and paragraphs are used to group and structure your information related to specific topics, level/priority of information, Jama Connect uses Folders, and a folder structure in a similar way within a Set.

Because everything in your Jama Connect project’s structure is atomic, you can select a Set and generate a document, or start a review of that Set, and all child elements underneath.

Utilizing This Atomic Nature of Items

The same ability is there when you select a Folder within that Set, allowing you to only select chapters and/or paragraphs on specific topics from your total document to your subject matter experts (SMEs), without them having to go through the entire document. Each specific topic can be sent for review to each (group of) SME(s) to get the most out of finding issues and correcting the (technical) content of each part of your document.

Once all reviews on all specific topics are concluded, you can move to formally approve the entirety of that document.

Related: Review ROI Calculator – Improve Review Process

Rolling Reviews

A “rolling” review is a review that changes the content of requirements that are included in each of the review’s revisions. Using this methodology, the review is much smaller in scope and can typically be completed faster.

Rolling reviews are standard Engagement Workbook nowadays. This mechanism actually binds a number of the review approaches discussed above into one:

  • Divide and conquer
  • Peer reviews leading up to approval reviews

It’s centered around the fact that all Items are atomic, each Item Type has the same status workflow, and the use of Filters to define the content of your review, where each new version you create of your review, will re-collect all Items that comply to the filter you’ve set up and baselines them.

It allows you to review each Item, or a subset of Items, separately, and collect all Items that have reached a status that indicates these Items are mature enough (content is of the quality, sanity, and completeness your organization strives for). Those Items then allow you to organize an approval review for the entire ‘document’.

Jama Connect Review Process

When initiating a review in Jama Connect, the steps included support this generic review process:

  1. Include the linked Items – upstream, downstream, and across – so your review invitees can evaluate the traceability.
  2. When using the ‘Rolling Review’-approach, select the corresponding filter.
  3. Invite at least three, but preferably four, colleagues in accordance with the contributions you can expect from their engineering role.
  4. Rewrite the standard invitation text of the email to assign focus areas to the invitees.

Related: Best Practices for Jama Connect Review Center

Additional Review Activities

Collaborative Review Meetings

Jama Connect allows organizations to run reviews online, which enables reviewers to determine when and where to spend their time participating in that review. However, much is to be learned from review meetings, where comments of a reviewer spark new insights and subsequent questions from another reviewer.

If the Moderator organizes a Review Meeting, a collective get-together, to discuss the review results and its found issues with all review participants, focus on the issues found, avoid discussions that take longer than a few back-and-forths, as the goal of the meeting is to try to process as many issues as possible in the (short) time available.

These discussions are important, so write down their topics and allow time to go into these discussions later; Ensure the meeting timeframe of the review session has a section for the actual review and a section for discussions.

Simply accept all grammar and spelling errors and ask the author to correct them after the session.

One thing to consider:
Abbreviations, terms, and definitions and how they’re used throughout your document do matter and should not be considered grammar or spelling errors!

Preparing For a Review Session

Insist everybody comes into the review session prepared, i.e., they’ve read the review comments of the other reviewers, made notes, and have their response ready. If they’re not prepared, participants may only read and evaluate reviewers’ comments and then respond to a comment much later, while the rest of the reviewers are already addressing the next issue.

Being prepared means the meeting can have short, to-the-point, and decisive discussions during the session, while still allowing you to process as many issues as possible.

In Conclusion

Defining the steps for approaching reviews and review coverage will help teams bring the scope of the review process into a more precise focus. By using an iterative and collaborative approach for reviewing requirements and other artifacts in real-time, organizations can improve stakeholder alignment, reduce review cycles, and ease the path to compliance.

RELATED


Danny Beerens
Latest posts by Danny Beerens (see all)