Tag Archive for: karl wiegers

Tag Archive for: karl wiegers

This image shows a bullseye target, graph, and money to show that change impact analysis is beneficial and that best practices help make goals.

Best Practices for Change Impact Analysis

Impact analysis is a key aspect of responsible requirements management. It provides an accurate understanding of the implications of a proposed change, which helps the teams make informed business decisions about which proposals to approve.

The analysis examines the proposed change to identify components that might have to be created, modified, or discarded and to estimate the effort associated with implementing the change.

Skipping impact analysis doesn’t change the size of the task. It just turns the size into a surprise.  In product development surprises are rarely good news. Before a developer says, “Sure, no problem” in response to a change request, he or she should spend a little time on impact analysis.

RELATED: A Guide to Good Systems Engineering Best Practices: The Basics and Beyond

Impact Analysis Procedure

Impact analysis has three aspects:

  1. Understand the possible implications of making the change. Change often produces a large ripple effect. Stuffing too much functionality into a product can reduce its performance to unacceptable levels.
  2. Identify all the files, models, and documents that might have to be modified if the team incorporates the requested change.
  3. Identify the tasks required to implement the change, and estimate the effort needed to complete those tasks.

Traceability data that links the affected requirement to other downstream deliverables helps greatly with impact analysis. On complex projects with thousands of artifacts, to manually determine what and who is affected by a change is time-consuming and error-prone. Alternatively, you could adopt a product development solution like Jama Connect, which includes built-in functionality for end-to-end traceability and impact analysis, and automatically highlights the items and people that are impacted when a change occurs.

RELATED: When Evaluating Product Development Software Tools, Not All Cloud is Equal

Whichever route you take, understanding the impact enables teams to quickly and accurately respond to change requests. The team can be responsive while maintaining control over the scope and customer expectations.

Lastly, impact analysis is essential on projects where quality and safety are an issue such as in healthcare, automotive, and aerospace projects. In these situations, it’s critical to understand the specific set of requirements and features that need to be retested after a change is implemented.

Steps in a typical impact analysis process look like this:

  1. Identify the sequence in which the tasks must be performed and how they can be interleaved with currently planned tasks.
  2. Determine whether the change is on the project’s critical path. If a task on the critical path slips, the project’s completion date will slip. Every change consumes resources, but if you can plan a change to avoid affecting tasks that are currently on the critical path, the change won’t cause the entire project to slip.
  3. Estimate the impact of the proposed change on the project’s schedule and cost.
  4. Evaluate the change’s priority by estimating the relative benefit, penalty, cost, and technical risk compared to other discretionary requirements.
  5. Report the impact analysis results to all stakeholders so that they can use the information to help them decide whether to approve or reject the change request.

In most cases, this procedure shouldn’t take more than a couple of hours to complete. This may seem like a lot of time to a busy developer, but it’s a small investment in making sure the project wisely invests its limited resources. If you can adequately assess the impact of a change without such a systematic evaluation, go right ahead; just make sure you aren’t stepping into quicksand.

Money Down the Drain

What can happen if you don’t take the time to perform impact analysis before diving into implementing a significant change request?

Imagine two developers on your team estimate that it will take four weeks to add an enhancement to one of your product lines. The customer approves the estimate, and the developers set to work. After two months, the enhancement is only about half done and the customer loses patience: “If I’d known how long this was really going to take and how much it was going to cost, I wouldn’t have approved it. Let’s forget the whole thing.”

In the rush to gain approval and begin implementation, the developers didn’t do enough impact analysis to develop a reliable estimate that would let the customer make an appropriate business decision. Consequently, you waste several hundred hours of work that could have been avoided by spending a few hours on an up-front impact analysis.

Learn how a requirements management solution eliminates many of the budget-draining headaches of product development in Karl Wiegers’ paper, “Getting the Most from a Requirements Management Tool.”

RELATED: A Guide to Good Systems Engineering Best Practices: The Basics and Beyond

A colorful link of chains showing the elements that make up the roles of traceability in change impact analysis; identifies effects, reduced rework, strengthens compliance, and enhances collaboration.

Editor’s Note: This section was added to this blog post on 4/08/2025. It was written with assistance from AI and reviewed and updated by McKenzie Jonsson.

The Importance of Traceability in Change Impact Analysis

Effective change impact analysis is not just about identifying affected requirements, tests, and risks—it’s about doing so with confidence, speed, and accuracy. This is where traceability becomes critical.

Traceability is the ability to track and document the lineage and history of requirements throughout the development process. It ensures that every requirement, test case, design element, and risk assessment is connected in a structured way, providing full visibility into all dependencies in the full process. When a change in one element is introduced, whether in response to evolving customer needs, regulatory updates, or system modifications, traceability enables teams to easily assess the impact of that change on all related artifacts.

How Traceability Enhances Change Impact Analysis

The truth is, the two go hand in in, really. Traceability supports change impact analysis in many ways, including:

  1. Identifies Downstream and Upstream Effects
    Changes in one part of a system can have cascading effects on related requirements, tests, and risk assessments. With end-to-end traceability, teams can immediately see what is impacted and take proactive steps to mitigate risks.
  2. Reduces Rework and Late-Stage Surprises
    Without traceability, overlooked dependencies often result in late-stage failures, expensive rework, or compliance gaps. A traceability-enabled approach to change impact analysis allows teams to plan changes with greater precision, minimizing unexpected disruptions.
  3. Strengthens Compliance and Audit Readiness
    Many industries, from medical devices to automotive and aerospace, require stringent documentation of changes and their impact. Traceability ensures that organizations can quickly generate reports that demonstrate compliance with regulatory requirements.
  4. Enhances Collaboration Across Teams
    By maintaining a single source of truth, traceability tools ensure that stakeholders across engineering, quality, and compliance teams stay aligned. This reduces miscommunication and streamlines decision-making.

RELATED: Traceable Agile™ – Speed AND Quality Are Possible for Software Factories in Safety-critical Industries

Leveraging Traceability Tools for Smarter Change Management

Manual impact analysis is time consuming and error prone. Modern requirements management platforms like Jama Connect® provide real-time traceability (Live Traceability™), allowing teams to visualize relationships between artifacts and perform change impact analysis efficiently. These tools help teams:

  • Instantly generate trace reports to understand how a change affects the system.
  • Perform gap analysis to ensure all impacted areas are accounted for.
  • Automate notifications to relevant stakeholders when changes occur.

By integrating robust traceability into change impact analysis, organizations can reduce risk, accelerate development cycles, and maintain compliance with confidence —ensuring that every change is made with clarity and precision.

Defining Project Scope

Every product development team talks about project scope and team members often complain about unending scope creep. The vision and scope document (often including a use case diagram and a context diagram), otherwise known as the MRD (marketing requirements document) or business case, is a key deliverable in defending against scope creep. You don’t necessarily need a standalone vision and scope document for a small project. Any project of any size, though, will benefit from such strategic guidance, even if it’s just a paragraph or two at the beginning of the requirements specification. 

Vision and Project Scope

defining project and use case diagrams-1

Both the vision and the scope are components of the project’s business requirements. I think in terms of the product vision and the project scope. I define the product vision as: “A long-term strategic concept of the ultimate purpose and form of a new system.” The product vision could also describe the product’s positioning among its competition and in its market or operating environment. 

A well-defined scope sets expectations among the project stakeholders. It identifies the external interfaces between the system and the rest of the world. The scope definition helps the project manager assess the resources needed to implement the project and make realistic commitments. The scope statement defines the boundary of the project manager’s responsibilities. 

Your scope definition also should include a list of specific limitations or exclusions—what’s out. Obviously, you can’t list everything that’s out of scope because that would include every detail in the universe except for the tiny sliver that is in scope for your project. Instead, the limitations should identify capabilities that a reader might expect to be included in the project, but which are not included. I know of a project to build a Web site for a national sports team that included the following exclusions for the initial release: 

  • There will be no virtual or fantasy games via the Web. 
  • There will be no ticketing facilities on the site. 
  • There will be no betting facilities available. 
  • The demographic details for newsletters will not be collected. 
  • Message boards are out of scope for phase 1. 

Some stakeholders involved with this project might have expected these capabilities to be included. Itemizing them as exclusions makes it clear that they won’t be. This is a form of expectation management, an important contributor to project success. 

Related: Project Management Best Practices  

Context Diagram

The venerable context diagram dates from the structured analysis revolution of the 1970s. Despite its antiquity, the context diagram remains a useful way to depict the environment in which a software system exists. Figure 1 illustrates a partial context diagram for a hypothetical corporate cafeteria ordering system. The context diagram shows the name of the system or product of interest in a circle. The circumference of the circle represents the system boundary. Rectangles outside the circle represent external entities, also called terminators. External entities could be user classes, actors, organizations, other software systems to which this one connects, or hardware devices that interface to the system. 

The interfaces between the system and these external entities are shown with labeled arrows, called flows. If the “system” is strictly an electronic system involving software and perhaps hardware components, flows will represent data or control signals. However, if the “system” includes both a software application and manual operations, flows could also represent the movement of physical objects. Two-headed flows indicate update operations involving the data object on the flow. 

defining project and use case diagrams-2

The context diagram depicts the project scope at a high level of abstraction. This diagram deliberately reveals nothing about the system internals: no information about functionality, architecture, or look-and-feel. Nor does it explicitly identify which features or functionality are in scope and which are not. The functional behavior of the system is merely implied by the labeled flows that connect the system to the external entities. Even the flows are labeled at a high level of abstraction, just to keep the diagram’s complexity manageable. 

Despite the limited view that the high level of abstraction imposes, the context diagram is a helpful representation of scope. It serves as a tool to help the project stakeholders communicate about what lies outside the system boundary. 

RELATED: High Cost of Poor Requirements Management

Use Case Diagram

Use cases are a powerful technique for exploring user requirements. The Unified Modeling Language (UML) includes a use case diagram notation. Figure 2 shows a partial use case diagram for our cafeteria ordering system. The rectangular box represents the system boundary, analogous to the circle in a context diagram. The stick figures outside the box represent actors, entities that reside outside the system’s context but interact with the system in some way. The actors correspond approximately (exactly, in this example) to the external entities shown in rectangles on the context diagram. 

defining project and use case diagrams-3

Unlike the context diagram, the use case diagram does provide some visibility into the system. Each oval inside the system boundary box represents a use case. The use case diagram shows the interactions of the system with its users and some connections between internal system operations, albeit at a high level of abstraction. 

The arrows on the use case diagram indicate which actors participate in each use case. In Figure 2, the arrow from the Patron to the oval labeled “Submit Feedback” means that the patron actor can initiate the Submit Feedback use case. The arrow from Submit Feedback to the Menu Manager actor indicates that the Menu Manager participates somehow in the execution of Submit Feedback. Arrows on the use case diagram do not indicate data flows as they do on the context diagram. In addition to showing these connections to external actors, a use case diagram could depict logical relationships and dependencies between use cases. 

RELATED: Characteristics of Effective Software Requirements and Software Requirements Specifications (SRS)

The use case diagram provides a richer scope representation than the context diagram because it provides a high-level look at the system’s capabilities, not just at its external interfaces. There is a practical limitation, though. Any sizable software system will have dozens of use cases, with many connections among them and between use cases and actors. Attempting to show all those objects inside a single system boundary box quickly becomes unwieldy. Therefore, the analyst needs to model groups of related use cases as packages or to create multiple use case diagrams. 

Many analysts have found the context diagram and use case diagram to be helpful ways to represent and communicate a shared understanding of a project’s scope. In Part 2 of this series, I’ll describe two other techniques for defining scope, feature levels and system events. 

Check out Part 2, Defining Project Scope: Feature Levels and System Events 

Check out Part 3, Defining Project Scope: Managing Scope Creep 

Download our eBook, Best Practices Guide for Requirements and Requirements Management, to learn the fundamentals of requirements and how effective requirements management can keep your projects on time and on budget.

READ THE EBOOK

This is an updated version of a 2014 article by Karl Wiegers. You can read the archived original here https://www.jamasoftware.com/blog/context-and-usecase-diagrams-defining-scope/.


Business analysts and managers sometimes ask me how long it will take to “do requirements” on their next project.

As with so many issues in software and product development, the correct answer to this question is “it depends.”

Multiple variables contribute to this issue. Various industry averages have been published to suggest what percentage of a typical project’s effort should be devoted to requirements development, which includes activities such as requirements gathering (also known as requirements elicitation).

Data from different benchmarks don’t agree very well, though, and whether these “typical” projects are similar to your own is questionable. In this article, adapted from my book, “More about Software Requirements,” I’ll offer some suggestions about how you can determine an appropriate amount of time and effort to invest in things like requirements gathering.

Industry Benchmarks

Here’s an illustration of how benchmarks may or may not be helpful. Table 1 (below) presents some industry benchmark data for the average percentage of total effort and the average schedule time that projects in several different categories devote to requirements elicitation and prototyping (data from Capers Jones’ “Software Assessments, Benchmarks, and Best Practices”). These benchmarks are for very large projects of 10,000 function points in size (approximately one million lines of code). How similar are your projects to these benchmarks?

Table 1. Benchmarks for Requirements Work on Large Projects

There’s another problem with using industry benchmarks such as these. The data doesn’t indicate how successful those projects were or define what “success” means for each project. Nor does this data indicate whether the more successful project teams devoted more of their effort to requirements elicitation activities than the less successful teams — they’re just averages of actual performance.

Whereas typical project teams devote perhaps 10% or less of their effort on things like requirements gathering, investing more has a big payoff, provided the team doesn’t get trapped in analysis paralysis. Contrary to what many people believe, spending more effort in improving your requirements development process can actually accelerate development.

A recent study by Engineering.com found that development teams without strong requirements management platforms reported more production outcome failures and reprimands by regulatory agencies.

Read it now.

While working on small projects when I was employed at Kodak, my team would typically devote 15%-to-18% of our total effort on requirements activities. We found this investment reduced the amount of post-delivery rework. It’s difficult to link causes and effects with certainty, but I believe the greatest contributing factor to our low maintenance level was the extensive user participation we cultivated.

I can’t tell you how long you should expect to spend on requirements gathering for your next project. However, Figure 1 identifies some of the conditions that can accelerate your requirements process and several other factors that lengthen the time needed for developing effective requirements.

Figure 1. Factors that influence the time needed for requirements development.

Your Own Experience

For starters, your best bet is to collect some data on how much of your own project effort is spent on requirements gathering. That’ll help you judge how well that has worked for you in the past. Use this historical data when estimating the requirements effort needed for future projects. Adjust your initial estimate by using the considerations in Figure 1 to compensate for differences between your next project and the benchmark projects. Consider any additional factors that would influence your own project. You might weight each of the factors shown in Figure 1 on a scale of 0 (no effect) to 5 (major impact). This analysis can help you spot risk factors that could prolong your requirements development work.

Another factor to consider is the development life cycle that the project is following. Not all the requirements elicitation effort should be allocated to the early stages of the project, as is the case in the sequential or waterfall life cycle (dotted line in Figure 2). Don’t think in terms of a discrete “requirements phase,” but rather about a set of requirements-related activities that span the project’s life cycle. In particular, requirements management will be performed on an ongoing basis once a set of requirements baselines emerge for the System Requirements Specification (SRS) and change requests begin to appear.

Figure 2. The distribution of requirements effort over time varies for projects that follow different development life cycles.


estimating-balance-01

In part one and part two of this series, adapted from my book Practical Project Initiation, I’ve described fifteen practices the project manager can apply to lay the foundation for a successful project, plan the project, and estimate the work to be done. In this final article I share two additional estimation practices, three good practices for tracking your progress throughout the project, and one practice for learning how to execute your future projects more successfully.

Estimating the Work (continued)

Practice #16: Use estimation tools. Many commercial tools are available to help project managers estimate entire projects. Based on equations derived from large databases of actual project experience, these tools can give you a spectrum of possible schedule and staff allocation options. They’ll also help you avoid the “impossible region,” combinations of product size, effort, and schedule where no known project has been successful. The tools incorporate a number of “cost drivers” you can adjust to make the tool more accurately model your project, based on the technologies used, the team’s experience, and other factors. You can compare the estimates from the tools with the bottom-up estimates generated from a work breakdown structure. Reconcile any major disconnects so you can generate the most realistic overall estimate.

Practice #17: Plan contingency buffers. Projects never go precisely as planned. The prudent project manager incorporates budget and schedule contingency buffers at the end of phases, dependent task sequences, or iterations to accommodate the unforeseen. Use your project risk analysis to estimate the possible schedule impact if several of the risks materialize, then build that projected risk exposure into your schedule as a contingency buffer. An even more sophisticated approach is critical chain analysis, a technique that pools the uncertainties in estimates and risks into a rational overall contingency buffer. Chapter 10 of Practical Project Initiation is all about contingency buffers.

Your manager or customer might view these contingency buffers as padding, rather than as the sensible acknowledgment of reality that they are. To help persuade skeptics, point to unpleasant surprises on previous projects as a rationale for your foresight. If a manager elects to discard contingency buffers, he has tacitly absorbed all the risks that fed into the buffer and assumed that all estimates are perfect, no scope growth will occur, and no unexpected events will take place. Sound realistic to you? Of course not. I’d rather see us deal with reality—however unattractive—than to live in Fantasyland.

Tracking Your Progress

Practice #18: Record actuals and estimates. Unless you record the actual effort or time spent on each project task and compare them to the estimates, your estimates will forever remain guesses. Someone once asked me where to get historical data to improve her ability to estimate future work. My answer was, “If you write down what actually happened today, that becomes historical data tomorrow.” It’s really not more complicated than that. Each individual can begin recording estimates and actuals, and the project manager should track these important data items on a project task or milestone basis. In addition to effort and schedule, you could estimate and track the size of the product, in terms of requirements, user stories, lines of code, function points, GUI screens, or other units that make sense for your project.

Practice #19: Count tasks as complete only when they’re one hundred percent complete. We give ourselves a lot of partial credit for tasks we’ve begun but not yet fully completed: “I thought about the algorithm for that module in the shower this morning, and the algorithm is the hard part, so I’m probably about sixty percent done.” It’s difficult to accurately assess what fraction of a sizable task has actually been finished at a given moment.

One benefit of using inch-pebbles (see Practice #6 in Part 2 of this series) for task planning is that you can break a large activity into a number of small tasks (inch-pebbles) and classify each small task as either done or not done—nothing in between. Project status tracking is then based on the fraction of the tasks that are completed and their size, not the percentage completion of each task. If someone asks you whether a specific task is complete and your reply is, “It’s all done except…,” then it’s not done! Don’t let people “round up” their task completion status. Instead, use explicit criteria to determine whether an activity truly is completed.

Practice #20: Track project status openly and honestly. An old riddle asks, “How does a software project become six months late?” The rueful answer is, “One day at a time.” The painful problems arise when the project manager doesn’t know just how far behind (or, occasionally, ahead) of plan the project really is. Surprise, surprise, surprise.

If you’re the PM, create a climate in which team members feel it is safe for them to report project status accurately. Run the project from a foundation of accurate, data-based facts, rather than from the misleading optimism that can arise from the fear of reporting bad news. Use project status information and metrics data to take corrective actions when necessary and to celebrate when you can. You can only manage a project effectively when you really know what’s done and what isn’t, what tasks are falling behind their estimates and why, and what problems, issues, and risks remain to be tackled.

The five major areas of software measurement are size, effort, time, quality, and status. It’s a good idea to define a few metrics in each of these categories. Instilling a measurement culture into an organization is not trivial. Some people resent having to collect data about the work they do, often because they’re afraid of how managers might use the measurements. The cardinal rule of software metrics is that management must never use the data collected to either reward or punish the individuals who did the work. The first time you do this will be the last time you can count on getting accurate data from the team members.

Learning for the Future

Practice #21: Conduct project retrospectives. Retrospectives (also called postmortems and post-project reviews) provide an opportunity for the team to reflect on how the last project, phase, or iteration went and to capture lessons learned that will help enhance your future performance. During such a review, identify the things that went well, so you can create an environment that enables you to repeat those success contributors. Also look for things that didn’t go so well, so you can change your approaches and prevent those problems in the future. In addition, think of events that surprised you. These might be risk factors to look for on the next project. Finally, ask yourself what you still don’t understand about the project, so you can try to learn how to execute future work better.

It’s important to conduct retrospectives in a constructive and honest atmosphere. Don’t make them an opportunity to assign blame for previous problems. Chapter 15 of Practical Project Initiation describes the project retrospective process and provides a worksheet to help you plan your next retrospective. It’s a good idea to capture the lessons learned from each retrospective exploration and share them with the entire team and organization. This is a way to help all team members, present and future, benefit from your experience.

The twenty-one project management best practices I’ve described in this series of articles won’t guarantee your project a great outcome. They will, however, help you get a solid handle on your project and ensure that you’re doing all you can to make it succeed in an unpredictable world.

Also read Project Management Best Practices, Part 1
Also read Project Management Best Practices, Part 2

Jama Software has partnered with Karl Wiegers to share licensed content from his books and articles on our web site via a series of blog posts, whitepapers and webinars.  Karl Wiegers is an independent consultant and not an employee of Jama.  He can be reached at http://www.processimpact.com.  Enjoy these free requirements management resources.

project-management-2-01

In Part 1 of this series, adapted from my book Practical Project Initiation, I shared four best practices that can help you lay the foundation for a successful project and two practices that are useful for project planning. This article continues the series by describing additional project planning best practices and several practices for estimating the work needed to complete the project.

Planning the Project 

Practice #7: Develop planning worksheets for common large tasks. If your team frequently undertakes certain common tasks—such as implementing a new class, executing a system test cycle, or performing a product build—develop activity checklists and planning worksheets for these tasks. Each checklist should include all of the steps the large task might need. These checklists and worksheets will help each team member identify and estimate the effort associated with each instance of the large task he must tackle. People work in different ways and no single person will think of all the necessary tasks, so engage multiple team members in developing the worksheets. Tailor the worksheets to meet the specific needs of individual projects. I’ve used such worksheets when creating eLearning versions of my training courses. They helped me avoid overlooking an important step in my rush to finish the project.

Practice #8: Plan to do rework after a quality control activity. I’ve seen project plans that assumed every test will be a success that lets you move on to the next development activity. However, almost all quality control activities, such as testing and peer reviews, find defects or other improvement opportunities. Your project schedule should include rework as a discrete task after every quality control task. Base your estimates of rework time on previous experience. If you collect a bit of data, you can calculate the average expected rework effort to correct defects found in various types of work products. And if you don’t have to do any rework after performing a test, great; you’re ahead of schedule on that task. This is permitted in all fifty states and in many other countries. Don’t count on it, though.

Practice #9: Manage project risks. If you don’t identify and control project risks, they’ll control you. A risk is a potential problem that could affect the success of your project. It’s a problem that hasn’t happened yet—and you’d like to keep it that way. Simply identifying the possible risk factors isn’t enough. You also have to evaluate the relative threat each one poses so you can focus your energy where it will do the most good.

Risk exposure is a combination of the probability that a specific risk could materialize into a problem and the negative consequences for the project if it does. To manage each risk, select mitigation actions to reduce either the probability or the impact. You might also identify contingency plans that will kick in if your risk control activities aren’t as effective as you hope.

A simple risk list doesn’t replace a plan for how you will identify, prioritize, control, and track risks. Incorporate risk tracking into your routine project status tracking. See Chapter 6 of Practical Project Initiation for an overview of software risk management.

Practice #10: Plan time for process improvement. Your team members are already swamped with their current project assignments. If you want the group to rise to a higher plane of software development capability, though, you’ll have to invest in process improvement. This means you’ll need to set aside some time from your project schedule for improvement activities. Don’t allocate one hundred percent of your team’s available time to project tasks and then wonder why they don’t make any progress on the improvement initiative.

Some process changes can begin to pay off immediately, but you won’t reap the full benefit from other improvements until the next project. Process improvement is a strategic investment in the organization. I liken process improvement to highway construction: It slows everyone down a little bit for a time, but after the work is done, the road is a lot smoother and the throughput is greater.

Practice #11: Respect the learning curve. The time and money you spend on training, self-study, consultants, and developing improved processes are part of the investment your organization makes in sustained project success. Recognize that you’ll pay a price in terms of a short-term productivity loss—the learning curve—when you first try to apply new processes, tools, or technologies. Don’t expect to get fabulous benefits on the first try, no matter what the tool vendor or the consultant claims. Make sure your managers and customers understand the learning curve as an inescapable consequence of working in a rapidly changing, high-technology field.

Estimating the Work

Practice #12: Estimate based on effort, not calendar time. People generally provide estimates in units of calendar time. I prefer to estimate the effort (in labor-hours) associated with a task, and then translate the effort into a calendar-time estimate. A twenty-hour task might take 2.5 calendar days of nominal full-time effort, or two exhausting days. However, it could also take a week if you have to wait for critical information from a customer or stay home with a sick child for two days. I base the translation of effort into calendar time on estimates of how many effective hours I can spend on project tasks per day, any interruptions or emergency bug fix requests I might get, meetings, and all the other places into which time disappears.

If you track how you actually spend your time at work, you’ll know how many effective weekly project hours you have available on average. Tracking time like this is illuminating. Typically, the effective project time is only perhaps fifty to sixty percent of the nominal time team members spend at work, far less than the assumed one hundred percent effective time on which so many project schedules are planned.

Practice #13: Don’t over-schedule multitasking people. The task-switching overhead associated with the many activities we are all asked to do reduces our effectiveness significantly. Excessive multitasking introduces communication and thought process inefficiencies that reduce individual productivity. I once heard a manager say that someone on his team had spent an average of eight hours per week on a particular activity, so therefore she could do five of them at once. Forty hours per week divided by eight is five, right? In reality, she’ll be lucky if she can handle three or four such tasks. There’s just too much friction associated with multitasking.

Some people multitask more efficiently than others, even thriving on it. But if certain of your team members thrash when working on too many tasks at once, set clear priorities and help them do well by focusing on just one or two objectives at a time.

Practice #14: Build training time into the schedule. Estimate how much time your team members spend on training activities each year and subtract that from the time available for them to work on project tasks. You probably already subtract out average values for vacation time, sick time, and other assignments; treat training time the same way.

Recognize that the high-tech field of software development demands that all practitioners devote time to ongoing education, both on their own time and on the company’s time. Arrange just-in-time training when you can schedule it, as the half-life of new technical knowledge is short unless the student puts the knowledge to use promptly. Attending a training seminar can be a team-building experience, as project team members and other stakeholders hear the same story about how to apply improved practices to their common challenges.

Practice #15: Record estimates and how you derived them. When you prepare estimates for your work, write down those estimates and document how you arrived at each of them. Understanding the assumptions and approaches used to create an estimate will make them easier to defend and adjust when necessary. It will also help you improve your estimation process. Train the team in estimation methods, rather than assuming that every software developer and project leader is naturally skilled at predicting the future. Develop estimation procedures and checklists that people throughout your organization can use.

The Wideband Delphi method is an effective group estimation technique. This technique asks a small team of experts to anonymously generate individual estimates from a problem description and reach consensus on a final set of estimates through iteration. Participation by multiple estimators and the use of anonymous estimates to prevent one participant from biasing another make the Wideband Delphi method more reliable than simply asking a single individual for his best guess. Chapter 11 of Practical Project Initiation presents a tutorial on the Wideband Delphi estimation method.

The final article in this series will describe two additional estimation tips, along with several good practices for tracking your progress and learning how to plan and manage future projects more effectively.

Also read Project Management Best Practices, Part 1
Also read Project Management Best Practices, Part 3

Jama Software has partnered with Karl Wiegers to share licensed content from his books and articles on our web site via a series of blog posts, whitepapers and webinars.  Karl Wiegers is an independent consultant and not an employee of Jama.  He can be reached at http://www.processimpact.com.  Enjoy these free requirements management resources.

[raw]

To read the updated version of this article, visit www.jamasoftware.com/blog/defining-project-scope-context-use-case-diagrams/.

[/raw]
Every software team talks about project scope and team members often complain about unending scope creep. Unfortunately, the software industry lacks uniform definitions of these terms, and the requirements literature is short on clear guidance regarding how to even represent scope. I confront scope head-on in this series of three articles, adapted from my book More about Software Requirements (Microsoft Press, 2006). I’ll present some definitions, describe four techniques for defining project scope, and offer some tips for managing scope creep.

Vision and Scope

defining project and use case diagrams-1

I regard the vision and scope document is a key software project deliverable. You can find a suggested template for this document at http://www.processimpact.com/goodies.shtml. Other terms for this type of guiding document are a project charter, market (or marketing) requirements document, and business case. You don’t necessarily need a standalone vision and scope document for a small project. Any project of any size, though, will benefit from such strategic guidance, even if it’s just a paragraph or two at the beginning of the software requirements specification.

Both the vision and the scope are components of the project’s business requirements. I think in terms of the product vision and the project scope. I define the product vision as: “A long-term strategic concept of the ultimate purpose and form of a new system.” The product vision could also describe the product’s positioning among its competition and in its market or operating environment. Chapter 5 of my book Software Requirements, 2nd Edition describes how to write a concise vision statement using a simple keyword template.

I’ll define project scope as: “The portion of the ultimate product vision that the current project or iteration will address. The scope draws the boundary between what’s in and what’s out for the project.” The second part of the project scope definition is most important. The scope identifies what the product is and is not, what it will and won’t do, what it will and won’t contain.

A well-defined scope sets expectations among the project stakeholders. It identifies the external interfaces between the system and the rest of the world. The scope definition helps the project manager assess the resources needed to implement the project and make realistic commitments. In essence, the scope statement defines the boundary of the project manager’s responsibilities.

Your scope definition also should include a list of specific limitations or exclusions—what’s out. Obviously, you can’t list everything that’s out of scope because that would include every detail in the universe except for the tiny sliver that is in scope for your project. Instead, the limitations should identify capabilities that a reader might expect to be included in the project but which are not included. I know of a project to build a Web site for a national sports team that included the following exclusions for the initial release:

  • There will be no virtual or fantasy games via the Web.
  • There will be no ticketing facilities on the site.
  • There will be no betting facilities available.
  • The demographic details for newsletters will not be collected.
  • Message boards are out of scope for phase 1.

Some stakeholders involved with this project might have expected these capabilities to be included. Itemizing them as exclusions makes it clear that they won’t be. This is a form of expectation management, an important contributor to project success.

The rest of this article describes two techniques I’ve found useful for depicting project scope, the context diagram and the use case diagram. In part 2 of the series, I’ll describe two additional techniques, feature levels and system events.

Context Diagram

The venerable context diagram dates from the structured analysis revolution of the 1970s. Despite its antiquity, the context diagram remains a useful way to depict the environment in which a software system exists. Figure 1 illustrates a partial context diagram for a hypothetical corporate cafeteria ordering system. The context diagram shows the name of the system or product of interest in a circle. The circumference of the circle represents the system boundary. Rectangles outside the circle represent external entities, also called terminators. External entities could be user classes, actors, organizations, other software systems to which this one connects, or hardware devices that interface to the system.

The interfaces between the system and these external entities are shown with labeled arrows, called flows. If the “system” is strictly an electronic system involving software and perhaps hardware components, flows will represent data or control signals. However, if the “system” includes both a software application and manual operations, flows could also represent the movement of physical objects. Two-headed flows indicate update operations involving the data object on the flow.

defining project and use case diagrams-2

The context diagram depicts the project scope at a high level of abstraction. This diagram deliberately reveals nothing about the system internals: no information about functionality, architecture, or look-and-feel. Nor does it explicitly identify which features or functionality are in scope and which are not. The functional behavior of the system is merely implied by the labeled flows that connect the system to the external entities. Even the flows are labeled at a high level of abstraction, just to keep the diagram’s complexity manageable. The business analyst can decompose these data flows into individual data elements in the project’s data dictionary or data model. Corresponding data inputs and outputs imply the types of operations the system will perform, but these aren’t shown explicitly in the context diagram.

Despite the limited view that the high level of abstraction imposes, the context diagram is a helpful representation of scope. It serves as a tool to help the project stakeholders communicate about what lies outside the system boundary. A BA in a requirements class I once taught showed me a context diagram for her current project. She had shown this diagram to the project manager. The manager had pointed out that one of the external entities shown on the context diagram, another information system, was now going to be part of the new system. With respect to Figure 1, this would be like moving the Payroll System inside the project circle. That is, the scope of the project just got larger than the BA expected. She had expected that external system to be someone else’s responsibility, but now it was her problem.

Use Case Diagram

Use cases are a powerful technique for exploring user requirements. The Unified Modeling Language (UML) includes a use case diagram notation. Figure 2 shows a partial use case diagram for our cafeteria ordering system. The rectangular box represents the system boundary, analogous to the circle in a context diagram. The stick figures outside the box represent actors, entities that reside outside the system’s context but interact with the system in some way. The actors correspond approximately (exactly, in this example) to the external entities shown in rectangles on the context diagram.

defining project and use case diagrams-3

Unlike the context diagram, the use case diagram does provide some visibility into the system. Each oval inside the system boundary box represents a use case. The use case diagram shows the interactions of the system with its users and some connections between internal system operations, albeit at a high level of abstraction.

The arrows on the use case diagram indicate which actors participate in each use case. In Figure 2, the arrow from the Patron to the oval labeled “Submit Feedback” means that the patron actor can initiate the Submit Feedback use case. The arrow from Submit Feedback to the Menu Manager actor indicates that the Menu Manager participates somehow in the execution of Submit Feedback. Arrows on the use case diagram do not indicate data flows as they do on the context diagram. Some analysts simply draw lines instead of arrows on the use case diagram to avoid any confusion with data flow. In addition to showing these connections to external actors, a use case diagram could depict logical relationships and dependencies between use cases.

The use case diagram provides a richer scope representation than the context diagram because it provides a high-level look at the system’s capabilities, not just at its external interfaces. There is a practical limitation, though. Any sizeable software system will have dozens of use cases, with many connections among them and between use cases and actors. Attempting to show all those objects inside a single system boundary box quickly becomes unwieldy. Therefore, the analyst needs to model groups of related use cases as packages or to create multiple use case diagrams.

Many analysts have found the context diagram and use case diagram to be helpful ways to represent and communicate a shared understanding of a project’s scope. In Part 2 of this series, I’ll describe two other techniques for defining scope, feature levels and system events.

Check out Part 2, Defining Project Scope: Feature Levels and System Events

Check out Part 3, Defining Project Scope: Managing Scope Creep


Learn how product development teams can leverage analytics to improve efficiency and quality in “The Essential Guide to Software Development Team Metrics.”

change-impact-analysis-01[1]

The need for performing impact analysis is obvious for major enhancements. However, unexpected complications can work below the surface of even minor change requests. A consulting client of mine once had to change the text of a single error message in its product. What could be simpler? The product was available in both English and German language versions. There were no problems in English, but in German the new message exceeded the maximum character length allocated for error message displays in both the message box and a database. Coping with this apparently simple change request turned out to be much more work than the developer had anticipated when he promised a quick turnaround.

Impact analysis is a key aspect of responsible requirements management. It provides accurate understanding of the implications of a proposed change, which helps the team make informed business decisions about which proposals to approve. The analysis examines the proposed change to identify components that might have to be created, modified, or discarded and to estimate the effort associated with implementing the change. Skipping impact analysis doesn’t change the size of the task. It just turns the size into a surprise. Software surprises are rarely good news. Before a developer says, “Sure, no problem” in response to a change request, he or she should spend a little time on impact analysis. This article, adapted from my book Software Requirements, 2nd Edition (Microsoft Press, 2003), describes how the impact analysis activities might work.

Impact Analysis Procedure

The chairperson of the change control board will typically ask a knowledgeable developer to perform the impact analysis for a specific change proposal. Impact analysis has three aspects:

  1. Understand the possible implications of making the change. Change often produces a large ripple effect. Stuffing too much functionality into a product can reduce its performance to unacceptable levels, as when a system that runs daily requires more than 24 hours to complete a single execution.
  2. Identify all the files, models, and documents that might have to be modified if the team incorporates the requested change.
  3. Identify the tasks required to implement the change, and estimate the effort needed to complete those tasks.

Figure 1 presents a checklist of questions designed to help the impact analyst understand the implications of accepting a proposed change. (You can download the checklists and templates described in this article from http://www.processimpact.com/goodies.shtml.) The checklist in Figure 2 contains prompting questions to help identify all of the software elements that the change might affect. Traceability data that links the affected requirement to other downstream deliverables helps greatly with impact analysis. As you gain experience using these checklists, modify them to suit your own projects.

Checklist of possible implications of a proposed change.

Checklist of possible implications of a proposed change.

Checklist of possible software elements affected by a proposed change.

Checklist of possible software elements affected by a proposed change.

Following is a simple procedure for evaluating the impact of a proposed requirement change. Many estimation problems arise because the estimator doesn’t think of all the work required to complete an activity. Therefore, this impact analysis approach emphasizes comprehensive task identification. For substantial changes, use a small team—not just one developer—to do the analysis and effort estimation to avoid overlooking important tasks.

  1. Work through the checklist in Figure 1.
  2. Work through the checklist in Figure 2, using available traceability information. Some requirements management tools include an impact analysis report that follows traceability links and finds the system elements that depend on the requirements affected by a change proposal.
  3. Use the worksheet in Figure 3 to estimate the effort required for the anticipated tasks. Most change requests will require only a portion of the tasks on the worksheet, but some could involve additional tasks.
  4. Total the effort estimates.
  5. Identify the sequence in which the tasks must be performed and how they can be interleaved with currently planned tasks.
  6. Determine whether the change is on the project’s critical path. If a task on the critical path slips, the project’s completion date will slip. Every change consumes resources, but if you can plan a change to avoid affecting tasks that are currently on the critical path, the change won’t cause the entire project to slip.
  7. Estimate the impact of the proposed change on the project’s schedule and cost.
  8. Evaluate the change’s priority by estimating the relative benefit, penalty, cost, and technical risk compared to other discretionary requirements.
  9. Report the impact analysis results to the CCB so that they can use the information to help them decide whether to approve or reject the change request.

In most cases, this procedure shouldn’t take more than a couple of hours to complete. This may seem like a lot of time to a busy developer, but it’s a small investment in making sure the project wisely invests its limited resources. If you can adequately assess the impact of a change without such a systematic evaluation, go right ahead; just make sure you aren’t stepping into quicksand. To improve your ability to estimate the impacts of future changes, compare the actual effort needed to implement each change with the estimated effort. Understand the reasons for any differences, and modify the impact estimation checklists and worksheet accordingly.

Estimating effort for a requirement change.

Estimating effort for a requirement change.

Money Down the Drain

Here’s a true story about what can happen if you don’t take the time to perform impact analysis before diving into implementing a significant change request. Two developers at the A. Datum Corporation estimated that it would take four weeks to add an enhancement to one of their information systems. The customer approved the estimate, and the developers set to work. After two months, the enhancement was only about half done and the customer lost patience: “If I’d known how long this was really going to take and how much it was going to cost, I wouldn’t have approved it. Let’s forget the whole thing.” In the rush to gain approval and begin implementation, the developers didn’t do enough impact analysis to develop a reliable estimate that would let the customer make an appropriate business decision. Consequently, the A. Datum Corporation wasted several hundred hours of work that could have been avoided by spending a few hours on an up-front impact analysis.

Impact Analysis Report Template

Figure 4 suggests a template for reporting the results from analyzing the potential impact of each requirement change. Using a standard template makes it easier for the CCB members to find the information they need to make good decisions. The people who will implement the change will need the analysis details and the effort planning worksheet, but the CCB needs only the summary of analysis results. As with all templates, try it and then adjust it to meet your project needs.

Impact analysis report template

Impact analysis report template

Requirements change is a reality for all software projects, but disciplined change-management practices can reduce the disruption that changes can cause. Improved requirements elicitation techniques can reduce the number of requirements changes, and effective requirements management will improve your ability to deliver on project commitments.

Jama Software has partnered with Karl Wiegers to share licensed content from his books and articles on our web site via a series of blog posts, whitepapers and webinars.  Karl Wiegers is an independent consultant and not an employee of Jama.  He can be reached at http://www.processimpact.com.  Enjoy these free requirements management resources.

measuring-requirements-2-illustration

This article continues the exploration of requirements-related metrics that I began in Measuring Requirements: Product Size and Requirements Quality.

Requirements Status

Track the status of each requirement over time to monitor overall project status, perhaps defining a requirement attribute to store this information. Status tracking can help you avoid the pervasive “ninety percent done” problem of software project tracking. Each requirement will have one of the following statuses at any time.

  • Proposed (someone suggested it)
  • Approved (it was allocated to a baseline)
  • Implemented (the code was designed, written, and unit tested)
  • Verified (the requirement passed its tests after integration into the product)
  • Deferred (the requirement will be implemented in a future release)
  • Deleted (you decided not to implement it at all)
  • Rejected (the idea was never approved)

Other status options are possible, of course. Some organizations use a status of “Reviewed” because they want to confirm that a requirement is of high quality before allocating it to a baseline. Other organizations use “Delivered to Customer” to indicate that a requirement has actually been released.

When you ask a developer how he is coming along, he might say, “Of the eighty-seven requirements allocated to this subsystem, sixty-one of them are verified, nine are implemented but not yet verified, and seventeen aren’t yet completely implemented.” There’s a good chance that not all these requirements are the same size, will consume the same amount of implementation effort, or will deliver the same customer value. If I were a project manager, though, I’d feel that we had a good handle on the size of that subsystem and how close we were to completion. This is far more informative than, “I’m about ninety percent done. Lookin’ good!”

Requests for Changes

Much of requirements management involves handling requirement additions, modifications, and deletions. Therefore, track the status and impact of your requirements change requests. The data you collect should let your team answer questions such as the following:

  • How many change requests were submitted in a given time period?
  • How many of these requests are open, and how many are closed?
  • How many requests were approved, and how many were rejected?
  • How much effort did the team spend implementing each approved change?
  • How long have the requests been open on average?
  • On average, how many individual requirements or other artifacts are affected by each submitted change request?

Monitor how many changes are incorporated throughout development after you baselined the requirements for a specific release. Note that a single change request potentially can affect multiple requirements of different levels and types (user requirements, functional requirements, nonfunctional requirements). To calculate requirements volatility over a given time period, divide the number of changes by the total number of requirements at the beginning of the period (for example, at the time a baseline was defined):

The intent is not to try to eliminate requirements volatility. There are often good reasons to change requirements. However, we need to ensure that the project can manage the degree of requirements changes and still meet its commitments. Changes become more expensive as the product nears completion, and a sustained high level of approved change requests makes it difficult to know when you can ship the product. Most projects should become more resistant to making changes as development progresses, meaning the trend of accepted changes should approach zero as you near the planned completion date for a given release. An iterative development approach gives the team multiple opportunities to incorporate changes into subsequent iterations, while still keeping each iteration on schedule.

If you receive many change requests, that suggests that elicitation overlooked many requirements or that new ideas keep coming in as the project drags along month after month. Record where the change requests come from: marketing, users, sales, management, developers, and so on. The change request origins will tell you who to work with to reduce the number of overlooked, modified, and misunderstood requirements.

Change requests that remain unresolved for a long time suggest that your change management process isn’t working well. I once visited a company where a manager wryly admitted that they had enhancement requests that were several years old and still pending. This team should allocate certain of their open requests to specific planned maintenance releases and convert other long-term deferred change requests to a status of rejected. This would help the project manager focus the team’s energy on the most important and most urgent items in the change backlog.

Effort

Finally, I recommend that you  record the time your team spends on requirements engineering activities. These activities include both requirements development (getting and writing good requirements) and requirements management (dealing with change, tracking status, recording traceability data, and so on).

I’m frequently asked how much time and effort a project should allocate to these functions. The answer depends enormously on the type and size of project, the developing team and organization, and the application domain. If you track your own team’s investment in these critical project activities, you can better estimate how much effort to plan for future projects.

Suppose that on one previous project, your team expended ten percent of its effort on requirements activities. In retrospect, you conclude that the requirements were too poorly defined and the project would have benefited from additional investment in developing quality requirements. The next time your team tackles a similar project, the project manager would be wise to allocate more than ten percent of the total project effort to the requirements work.

As you accumulate data, correlate the project development effort with some measure of product size. The documented requirements should give you an indication of size. You could correlate effort with the count of individually testable requirements, use case points, function points, or something else that is proportional to product size. As Figure 1 illustrates, such correlations provide a measure of your development team’s productivity, which will help you estimate and scope individual release contents. If you collect some product size data and track the corresponding implementation effort, you’ll be in a better position to create meaningful estimates for similar projects in the future.

Some people are afraid that launching a software measurement effort will consume too much time, time they feel should be spent doing “real work.” My experience, though, is that a sensible and focused metrics program doesn’t take much time or effort at all. It’s mostly a matter of developing a simple infrastructure for collecting and analyzing the data, and getting team members in the habit of recording some key bits of data about their work. Once you’ve developed a measurement culture in your organization, you’ll be surprised how much you can learn from the data.

Also read Measuring Requirements: Product Size and Requirements Quality.

Jama Software has partnered with Karl Wiegers to share licensed content from his books and articles on our web site via a series of blog posts, whitepapers and webinars.  Karl Wiegers is an independent consultant and not an employee of Jama.  He can be reached at http://www.processimpact.com.  Enjoy these free requirements management resources.

 

measuring-requirements-1-illustration-01-01

Disciplined software organizations collect a focused set of metrics about each of their projects. These metrics provide insight into the size of the product; the effort, time, and money that the project or individual tasks consumed; the project status; and the product’s quality. Because requirements are an essential project component, you should measure several aspects of your requirements engineering activities. This two-part series, adapted from my book More about Software Requirements, describes several meaningful metrics related to requirements activities on your projects. Enjoy these free requirements management resources.

Product Size

The most fundamental metric is the number of requirements in a body of work. Your project might represent requirements by using a mix of use cases, functional requirements, user stories, feature descriptions, event-response tables, and analysis models. However, the team ultimately implements functional requirements, descriptions of how the system should behave under specific conditions.

Begin your requirements measurement by simply counting the individual functional requirements that are allocated to the baseline for a given product release or development iteration. If different team members can’t count the requirements and get the same answer, you have to wonder what other sorts of ambiguities and misunderstandings they’ll experience. Knowing how many requirements are going into a release will help you judge how the team is progressing toward completion because you can monitor the backlog of work remaining to be done. If you don’t know how many requirements you need to work on, how will you know when you’re done?

Of course, not all functional requirements will consume the same implementation and testing effort. If you’re going to count functional requirements as an indicator of system size, your analysts will need to write them at a consistent level of granularity. One guideline is to decompose high-level requirements until the child requirements are all individually testable. That is, a tester can design a few logically related tests to verify whether a requirement was correctly implemented. Count the total number of child requirements, because those are what developers will implement and testers will test. Alternative requirements sizing techniques include use case points and story points. All of these methods involve estimating the relative effort to implement a defined chunk of functionality.

Functional requirements aren’t the whole story, of course. Stringent nonfunctional requirements can consume considerable design and implementation effort. Some functionality is derived from specified nonfunctional requirements, such as security requirements, so those would be incorporated appropriately into the functional requirement size estimate. But not all nonfunctional requirements will be reflected in this size estimate. Be sure to consider the impact of nonfunctional requirements upon your effort estimate. Consider the following situations:

  • If the user must have multiple ways to access specific functions to improve usability, it will take more development effort than if only one access mechanism is needed.
  • Imposed design and implementation constraints, such as multiple external interfaces to achieve compatibility with an existing operating environment, can lead to a lot of interface work even though you aren’t providing additional new product functionality.
  • Strict performance requirements might demand extensive algorithm and database design work to optimize response times.
  • Rigorous availability and reliability requirements can imply significant work to build in failover and data recovery mechanisms, as well as having implications for the system architecture you select.

You’ll also find it informative to track the growth in requirements as a function of time, no matter what requirements size metric you use. One of my clients found that their projects typically grew in size by about twenty-five percent before delivery. Amazingly, they also ran about twenty-five percent over the planned schedule on most of their projects. Coincidence? I think not.

Requirements Quality

Consider collecting some data regarding the quality of your requirements. Inspections of requirements specifications are a good source of this information. Count the requirements defects you discover and classify them into various categories: missing requirements, erroneous requirements, unnecessary requirements, incompleteness, ambiguities, and so forth. Use defect type frequencies and root-cause analysis to tune up your requirements processes so the team makes fewer of these types of errors in the future. For instance, if you find that missing requirements are a common problem, your elicitation approaches need some adjustments. Perhaps your business analysts aren’t asking enough questions or the right questions, or maybe you need to engage more appropriate user representatives in the requirements development process.

If the team members don’t think they have time to inspect all their requirements documentation, try inspecting a sample of just a few pages. Then calculate the average defect density—the number of defects found per specification page—for the sample. Assuming that the sample was representative of the entire document (a big assumption), you can multiply the number of uninspected pages by this defect density to estimate the number of undiscovered defects that could still lurk in the specification. Less experienced inspectors might discover only, say, half the defects that actually are present, so use this estimated number of undiscovered defects as a lower bound. Inspection sampling can let you assess the document’s quality so that you can determine whether it’s cost effective to inspect the rest of the requirements specification. The answer will almost certainly be yes.

Also, keep records of requirements defects that are identified after the requirements are baselined, such as requirements-related problems discovered during design, coding, and testing. These represent errors that leaked through your quality control filters during requirements development. Calculate the percentage of the total number of requirements errors that the team caught at the requirements stage. Removing requirements defects early is far cheaper than correcting them after the team has already designed, coded, and tested the wrong requirements.

Two informative metrics to calculate from inspection data are efficiency and effectiveness. Efficiency refers to the average number of defects discovered per labor hour of inspection effort. Effectiveness refers to the percentage of the defects originally present in a work product that was discovered by inspection. Effectiveness will tell you how well your inspections (or other requirements quality techniques) are working. Efficiency will tell you what it costs you, on average, to discover a defect through inspection. You can compare that cost with the cost of dealing with requirements defects found later in the project or after delivery to judge whether improving the quality of your requirements is cost effective.

The second article in this series will address metrics related to requirements status, change requests, and the effort expended on requirements development and management activities.

Also read Measuring Requirements: Status and Requests for Changes.

Jama Software has partnered with Karl Wiegers to share licensed content from his books and articles on our web site via a series of blog posts, whitepapers and webinars.  Karl Wiegers is an independent consultant and not an employee of Jama.  He can be reached at http://www.processimpact.com.  Enjoy these free requirements management resources.

This article continues the exploration of requirements-related metrics that I began in Part 1, which looked at measuring product size and requirements quality.

Requirements Status

Track the status of each requirement over time to monitor overall project status, perhaps defining a requirement attribute to store this information. Status tracking can help you avoid the pervasive “ninety percent done” problem of software project tracking. Each requirement will have one of the following statuses at any time.

  • Proposed (someone suggested it)
  • Approved (it was allocated to a baseline)
  • Implemented (the code was designed, written, and unit tested)
  • Verified (the requirement passed its tests after integration into the product)
  • Deferred (the requirement will be implemented in a future release)
  • Deleted (you decided not to implement it at all)
  • Rejected (the idea was never approved)

Other status options are possible, of course. Some organizations use a status of “Reviewed” because they want to confirm that a requirement is of high quality before allocating it to a baseline. Other organizations use “Delivered to Customer” to indicate that a requirement has actually been released.

When you ask a developer how he is coming along, he might say, “Of the eighty-seven requirements allocated to this subsystem, sixty-one of them are verified, nine are implemented but not yet verified, and seventeen aren’t yet completely implemented.” There’s a good chance that not all these requirements are the same size, will consume the same amount of implementation effort, or will deliver the same customer value. If I were a project manager, though, I’d feel that we had a good handle on the size of that subsystem and how close we were to completion. This is far more informative than, “I’m about ninety percent done. Lookin’ good!”

Requests for Changes

Much of requirements management involves handling requirement additions, modifications, and deletions. Therefore, track the status and impact of your requirements change requests. The data you collect should let your team answer questions such as the following:

  • How many change requests were submitted in a given time period?
  • How many of these requests are open, and how many are closed?
  • How many requests were approved, and how many were rejected?
  • How much effort did the team spend implementing each approved change?
  • How long have the requests been open on average?
  • On average, how many individual requirements or other artifacts are affected by each submitted change request?

Monitor how many changes are incorporated throughout development after you baselined the requirements for a specific release. Note that a single change request potentially can affect multiple requirements of different levels and types (user requirements, functional requirements, nonfunctional requirements). To calculate requirements volatility over a given time period, divide the number of changes by the total number of requirements at the beginning of the period (for example, at the time a baseline was defined):

The intent is not to try to eliminate requirements volatility. There are often good reasons to change requirements. However, we need to ensure that the project can manage the degree of requirements changes and still meet its commitments. Changes become more expensive as the product nears completion, and a sustained high level of approved change requests makes it difficult to know when you can ship the product. Most projects should become more resistant to making changes as development progresses, meaning the trend of accepted changes should approach zero as you near the planned completion date for a given release. An iterative development approach gives the team multiple opportunities to incorporate changes into subsequent iterations, while still keeping each iteration on schedule.

If you receive many change requests, that suggests that elicitation overlooked many requirements or that new ideas keep coming in as the project drags along month after month. Record where the change requests come from: marketing, users, sales, management, developers, and so on. The change request origins will tell you who to work with to reduce the number of overlooked, modified, and misunderstood requirements.

Change requests that remain unresolved for a long time suggest that your change management process isn’t working well. I once visited a company where a manager wryly admitted that they had enhancement requests that were several years old and still pending. This team should allocate certain of their open requests to specific planned maintenance releases and convert other long-term deferred change requests to a status of rejected. This would help the project manager focus the team’s energy on the most important and most urgent items in the change backlog.

Effort

Finally, I recommend that you  record the time your team spends on requirements engineering activities. These activities include both requirements development (getting and writing good requirements) and requirements management (dealing with change, tracking status, recording traceability data, and so on).

I’m frequently asked how much time and effort a project should allocate to these functions. The answer depends enormously on the type and size of project, the developing team and organization, and the application domain. If you track your own team’s investment in these critical project activities, you can better estimate how much effort to plan for future projects.

Suppose that on one previous project, your team expended ten percent of its effort on requirements activities. In retrospect, you conclude that the requirements were too poorly defined and the project would have benefited from additional investment in developing quality requirements. The next time your team tackles a similar project, the project manager would be wise to allocate more than ten percent of the total project effort to the requirements work.

As you accumulate data, correlate the project development effort with some measure of product size. The documented requirements should give you an indication of size. You could correlate effort with the count of individually testable requirements, use case points, function points, or something else that is proportional to product size. As Figure 1 illustrates, such correlations provide a measure of your development team’s productivity, which will help you estimate and scope individual release contents. If you collect some product size data and track the corresponding implementation effort, you’ll be in a better position to create meaningful estimates for similar projects in the future.

Figure 1. Correlating requirements size with project effort gives a measure of team productivity. Each point represents a separate project.

Some people are afraid that launching a software measurement effort will consume too much time, time they feel should be spent doing “real work.” My experience, though, is that a sensible and focused metrics program doesn’t take much time or effort at all. It’s mostly a matter of developing a simple infrastructure for collecting and analyzing the data, and getting team members in the habit of recording some key bits of data about their work. Once you’ve developed a measurement culture in your organization, you’ll be surprised how much you can learn from the data.

Also read Measuring Requirements, Part 1

Jama Software has partnered with Karl Wiegers to share licensed content from his books and articles on our web site via a series of blog posts, whitepapers and webinars.  Karl Wiegers is an independent consultant and not an employee of Jama.  He can be reached at http://www.processimpact.com.  Enjoy these free requirements management resources.