requirements management Archives

The need for performing impact analysis is obvious for major enhancements. However, unexpected complications can work below the surface of even minor change requests. A consulting client of mine once had to change the text of a single error message in its product. What could be simpler? The product was available in both English and German language versions. There were no problems in English, but in German the new message exceeded the maximum character length allocated for error message displays in both the message box and a database. Coping with this apparently simple change request turned out to be much more work than the developer had anticipated when he promised a quick turnaround.

Impact analysis is a key aspect of responsible requirements management. It provides accurate understanding of the implications of a proposed change, which helps the team make informed business decisions about which proposals to approve. The analysis examines the proposed change to identify components that might have to be created, modified, or discarded and to estimate the effort associated with implementing the change. Skipping impact analysis doesn’t change the size of the task. It just turns the size into a surprise. Software surprises are rarely good news. Before a developer says, “Sure, no problem” in response to a change request, he or she should spend a little time on impact analysis. This article, adapted from my book Software Requirements, 2nd Edition (Microsoft Press, 2003), describes how the impact analysis activities might work.

Impact Analysis Procedure

The chairperson of the change control board will typically ask a knowledgeable developer to perform the impact analysis for a specific change proposal. Impact analysis has three aspects:

Understand the possible implications of making the change. Change often produces a large ripple effect. Stuffing too much functionality into a product can reduce its performance to unacceptable levels, as when a system that runs daily requires more than 24 hours to complete a single execution.
Identify all the files, models, and documents that might have to be modified if the team incorporates the requested change.
Identify the tasks required to implement the change, and estimate the effort needed to complete those tasks.

Figure 1 presents a checklist of questions designed to help the impact analyst understand the implications of accepting a proposed change. (You can download the checklists and templates described in this article from http://www.processimpact.com/goodies.shtml.) The checklist in Figure 2 contains prompting questions to help identify all of the software elements that the change might affect. Traceability data that links the affected requirement to other downstream deliverables helps greatly with impact analysis. As you gain experience using these checklists, modify them to suit your own projects.

Checklist of possible implications of a proposed change.

Checklist of possible software elements affected by a proposed change.

Following is a simple procedure for evaluating the impact of a proposed requirement change. Many estimation problems arise because the estimator doesn’t think of all the work required to complete an activity. Therefore, this impact analysis approach emphasizes comprehensive task identification. For substantial changes, use a small team—not just one developer—to do the analysis and effort estimation to avoid overlooking important tasks.

Work through the checklist in Figure 1.
Work through the checklist in Figure 2, using available traceability information. Some requirements management tools include an impact analysis report that follows traceability links and finds the system elements that depend on the requirements affected by a change proposal.
Use the worksheet in Figure 3 to estimate the effort required for the anticipated tasks. Most change requests will require only a portion of the tasks on the worksheet, but some could involve additional tasks.
Total the effort estimates.
Identify the sequence in which the tasks must be performed and how they can be interleaved with currently planned tasks.
Determine whether the change is on the project’s critical path. If a task on the critical path slips, the project’s completion date will slip. Every change consumes resources, but if you can plan a change to avoid affecting tasks that are currently on the critical path, the change won’t cause the entire project to slip.
Estimate the impact of the proposed change on the project’s schedule and cost.
Evaluate the change’s priority by estimating the relative benefit, penalty, cost, and technical risk compared to other discretionary requirements.
Report the impact analysis results to the CCB so that they can use the information to help them decide whether to approve or reject the change request.

In most cases, this procedure shouldn’t take more than a couple of hours to complete. This may seem like a lot of time to a busy developer, but it’s a small investment in making sure the project wisely invests its limited resources. If you can adequately assess the impact of a change without such a systematic evaluation, go right ahead; just make sure you aren’t stepping into quicksand. To improve your ability to estimate the impacts of future changes, compare the actual effort needed to implement each change with the estimated effort. Understand the reasons for any differences, and modify the impact estimation checklists and worksheet accordingly.

Estimating effort for a requirement change.

Money Down the Drain

Here’s a true story about what can happen if you don’t take the time to perform impact analysis before diving into implementing a significant change request. Two developers at the A. Datum Corporation estimated that it would take four weeks to add an enhancement to one of their information systems. The customer approved the estimate, and the developers set to work. After two months, the enhancement was only about half done and the customer lost patience: “If I’d known how long this was really going to take and how much it was going to cost, I wouldn’t have approved it. Let’s forget the whole thing.” In the rush to gain approval and begin implementation, the developers didn’t do enough impact analysis to develop a reliable estimate that would let the customer make an appropriate business decision. Consequently, the A. Datum Corporation wasted several hundred hours of work that could have been avoided by spending a few hours on an up-front impact analysis.

Impact Analysis Report Template

Figure 4 suggests a template for reporting the results from analyzing the potential impact of each requirement change. Using a standard template makes it easier for the CCB members to find the information they need to make good decisions. The people who will implement the change will need the analysis details and the effort planning worksheet, but the CCB needs only the summary of analysis results. As with all templates, try it and then adjust it to meet your project needs.

Impact analysis report template

Requirements change is a reality for all software projects, but disciplined change-management practices can reduce the disruption that changes can cause. Improved requirements elicitation techniques can reduce the number of requirements changes, and effective requirements management will improve your ability to deliver on project commitments.

Jama Software has partnered with Karl Wiegers to share licensed content from his books and articles on our web site via a series of blog posts, whitepapers and webinars. Karl Wiegers is an independent consultant and not an employee of Jama. He can be reached at http://www.processimpact.com. Enjoy these free requirements management resources.

There has been no shortage of press surrounding the HealthCare.gov October 1^st release. Much of the debate points fingers at individuals often skewed by political leanings and party affiliation. Regardless of the political circus there are important lessons to be learned that highlight yet another glaring example of the importance of open, iterative collaboration around building products. This is not a new problem, nor is it unique to government. This is a problem faced by some of the world’s largest companies. Both Microsoft and Apple have recently experienced less than stellar rollouts: Surface estimated as a $900 million mistake and Apple Maps, which prompted a $30 billion loss in stock shares.

The fact is that when developing massively complex products in a broken system with insufficient tools, issues will be amplified. In the fallout of the much-anticipated HealthCare.gov release, the typical reactions are taking place; blame the most senior person available and throw tons of last minute resources at the problem in an attempt to fix it. None of this will work because the damage has been done – the source of the problem is cultural and ingrained in the process. Anthony Wing Kosner’s piece in Forbes discusses one aspect of the problem centered around the definition of the requirements themselves. Kosner discusses the fact that an estimated “40% of the defects that make it into the testing phase of enterprise software have their root cause in errors in the original requirements documents” and that government project requirements can be especially difficult to manage.

I have experience working on an implementation that suffered similar issues, and am especially frustrated with the state of HealthCare.gov because since then I’ve been reading about the many initiatives to incorporate Agile or Lean concepts into government software products. There are instances where taking an open, collaborative approach has paid off; one great example is the implementation of Recovery.gov and FederalReporting.gov. The success stories incorporated iterative processes that enabled changes to be more easily and effectively implemented, as opposed to HealthCare.gov, whose architects have cited changes in requirements as a root cause of issues.

Each set of changes created a new version of the document that needed to be shared across many teams. For example, Andrew Slavitt, vice-president of Optum explained to lawmakers about the late decision requiring consumers to register for an account before browsing insurance products. This highlights the typical occurrence of change, but begs the question: is the late decision the problem or is it how that decision is managed and ultimately communicated to the rest of the team? Compounding these problems is the fact that especially in government there are multiple contractors involved that are not incentivized to work together. The success of the their work is more important than the success of the project as a whole because contracts and payments are based on those deliverables. This fosters an environment of CYA as each contractor spends more time making sure that they are covered based on their individual contract and less on the idea of building the right solution.

The sad thing is that it’s hard to blame the contractors, developers, QA or even the government. The problem is in the status quo we all accept. Organizations continue to resist the inevitable need to make overarching changes to their process and tools to move away from such avoidable chaos and lack of communication. Based on this fact and my past experiences, there are 4 key lessons to be learned from the HealthCare.gov story:

Initial expectations and visions are often too vague and lofty. In the instance of HealthCare.gov, acceptance of change was not considered early on. Implementing such a complex system is not as simple as defining a set of requirements and pushing them downstream to be built. It takes a highly coordinated effort full of constant communication and realignment to change. There needs to be a conversation about how to handle unpredictability.
Allow for voices to be heard sooner. If there are no clear paths of communication that connect the business side to the implementation and test side there is an increased risk of misalignment, discontent, frustration, and delays that ultimately create a fast moving train of costly issues.
Allow for innovation to thrive throughout. Were there options to rethink the requirements themselves? One such possibility was the requirement for each user to receive an immediate confirmation based on complex integrations and system checks. Would it have sufficed to let the user know their information was accepted and would be processed over the next 7 business days?
Tools. If those who built HealthCare.gov are anything like many major organizations and companies they were likely relying heavily on Word documents to send requirements. When will we learn that this is an archaic means of communication? This method does not in any way help with managing the constant influx of changes notorious with government contracts.

My own experience working on a government healthcare software project took place 6 years ago when I worked for a contractor who was tasked with designing a state Medicaid eligibility program. The realities and problems my team faced back then have a striking similarity to the failings of the HealthCare.gov rollout. I was a developer at the time and was part of a team that had been put together based on a set of requirements that were written as an RFP and awarded to the contractor. Based on those initial requirements, estimates were created and resources were assigned to the project. As is common in RFPs, a year passed between the award and the final team being assembled, during which requirements changed. This common occurrence is most often the initial point of failure as teams quickly fall out of alignment, while scope and schedule became a constant topic of debate.

As a development team, we worked in a very Agile manner and provided frequent working software to the customer. Still, the complexity and volume of changes made it very difficult to be efficient, the schedule changed often and resources were staffed up in an effort to increase velocity. Luckily we had some flexibility with dates and a lot less visibility nationally.

Why does this status quo continue to dominate and dictate how we implement projects? The government is not alone in the challenges they are facing in so many products and programs. At my company, we often receive RFPs that are frankly outdated and irrelevant. The RFP model is broken and the idea that requirements are to be used as a binding contract that if changed puts both parties at odds, prevents teams from doing what’s right and speaking up if they sense something is wrong. RFPs are not the only culprit, requirements used as a binding contract set the tone. The agile manifesto highlights this point: “Customer collaboration over contract negotiation.” This isn’t to say there isn’t a need for contracts. HealthCare.gov may have failed just as much had there been no contracts involved. It’s easy to say generically that an agile approach would have done better, especially considering many over emphasize the desire to eliminate requirements all together in lieu of complete “we’ll figure it out as we go along” mentality.

The balance is in providing the goals (the requirements) in parallel to a more open, iterative, and collaborative process that is necessary in order to deliver products that fulfill requirements and are completed on time. The requirements and decisions themselves should have been in a place that was available to all that could handle the complexity and balance of requirements that change the speed necessary to make critical decisions when they can’t be changed. We are at a day and age where how we collaborate has fundamentally changed. Organizations need to take advantage of both the concept of agile – keeping in mind the key quote “…while there is value in the items on the right we value the items on the left more.” – and tools and technology built on modern methods of communication. Had this been in place for HealthCare.gov, changes to requirements would have been clear, communication would have been open, and decisions made would have been communicated immediately. All of that information would have been easy to track and maybe we wouldn’t be in this state of constant blame.

Tag Archive for: requirements management Page 10

Tag Archive for: requirements management