A Software Metrics Primer, Part 1
|
My friend Nicole was a quality manager at a large company recognized for its software process improvement and measurement success. Once I heard her say, “Our latest code inspection only found two major defects, but we expected to find five, so we’re trying to figure out what’s going on.” Few organizations have deep enough insight into their software development process to make such precise statements. Nicole’s organization spent years establishing effective processes, measuring critical aspects of its work, and applying those measurements on well-managed projects that develop high-quality products.
Software measurement is a challenging but essential component of a healthy and highly capable software engineering culture. This two-part series, adapted from my book Practical Project Initiation: A Handbook with Tools (Microsoft Press, 2007), describes some basic software measurement principles and suggests some metrics that can help you understand and improve the way your organization and its projects operate.
Why Measure Software
Software projects are notorious for running over schedule and budget and having quality problems to boot. Software measurement lets you quantify your schedule and budget performance, work effort, product size, product quality, and project status. If you don’t measure your current performance and use the data to improve your future work estimates, those estimates will forever remain guesses. Because today’s current data becomes tomorrow’s historical data, it’s never too late to begin recording key information about your project.
You can’t track project status meaningfully unless you know the actual effort and time spent on each task compared to your plans and the progress you’ve accomplished as a result. You can’t sensibly decide whether your product is stable enough to ship unless you’re tracking the rates at which your team is finding and fixing defects. You can’t assess how well your new development processes are working without some measure of your current performance and a baseline to compare against. Metrics help you better control your software projects and better understand how your organization works.
What to Measure
You can measure many aspects of your software products, projects, and processes. The trick is to select a small, balanced set of metrics that will help your organization track progress toward its goals. The Goal-Question-Metric (GQM) method promoted by Professor Victor Basili of the University of Maryland is an effective technique for selecting appropriate metrics to meet your needs. With GQM, you begin by selecting a few project or organizational goals. State the goals to be as quantitative and measurable as you can. They might include targets such as the following:
- Reduce maintenance costs by 50% within one year.
- Improve schedule estimation accuracy to within 10% of actuals.
- Reduce system testing time by three weeks on the next project.
- Reduce the time to close a defect by 40% within three months.
For each goal, think of questions you would have to answer to judge whether your team is reaching that goal. If your goal was “reduce maintenance costs by 50% within one year,” these might be some appropriate questions:
- How much do we spend on maintenance each month?
- What fraction of our maintenance costs do we spend on each application we support?
- How much do we currently spend on adaptive maintenance (adapting to a changed environment), perfective maintenance (adding enhancements), and corrective maintenance (fixing defects)?
Finally, identify metrics that will provide answers to each question. Some of these will be simple items you can count directly, such as the total budget spent on maintenance. These are called base measures. Other metrics are derived measures, which are calculated from one or more base measures, often as simple sums or ratios. To answer the final question in the previous list, you must know the hours spent on each of the three maintenance activity types and the total maintenance cost over a period of time.
Note that I expressed several goals in terms of a percentage change from the current level. The first step of a metrics program is to establish a current baseline, so you can track progress against it and toward your goals. I prefer to establish relative improvement goals (“reduce maintenance by 50%”) rather than absolute goals (“reduce maintenance to 20% of total effort”). You can probably reduce maintenance to 20 percent of total organizational effort within a year if you are currently at 30 percent, but not if you spend 80 percent of your effort on maintenance today.
Your balanced metrics set should eventually include items relating to the six basic dimensions of software measurement: size, time, effort, cost, quality, and status. Table 1 suggests some metrics that individual developers, project teams, and development organizations should consider collecting. Some of these metrics relate to products, others to projects, and the remainder to processes. You should track most of these metrics over time. For example, your routine project tracking activities should monitor the percentage of requirements implemented and tested, the number of open and closed defects, and so on. I recommend including the following base measures fairly early in your metrics program:
- Product size: Count lines of code, function points, object classes, number of requirements, or GUI elements.
- Estimated and actual duration: Count in units of calendar time.
- Estimated and actual effort: Count in units of labor hours.
- Work effort distribution: Record the time spent in various development and maintenance activities.
- Requirements status: Track the number of requirements that are proposed, approved, implemented, verified, deleted, and rejected.
- Defects: Track the number, type, severity, and status of defects found by testing, peer reviews, and customers.
Table 1: Appropriate Metrics for Software Developers, Teams, and Organizations
| Level | Appropriate Metrics |
| Individual Developers |
|
| Project Teams |
|
| Development Organization |
|
It’s important to remember that even if you conclude that all of the metrics in Table 1 are valuable, you cannot possibly begin a metrics program that encompasses so many dimensions of measurement. It would overwhelm the practitioners, and you’d likely end up with far more data than you’re prepared to analyze and use to steer the organization’s development activities.
The second article in this series will continue with some recommendations about the importance of creating a measurement culture in the organization, as well as presenting five tips for metrics success.
Read A Software Metrics Primer, Part 2.
Jama Software has partnered with Karl Wiegers to share licensed content from his books and articles on our web site via a series of blog posts, whitepapers and webinars. Karl Wiegers is an independent consultant and not an employee of Jama. He can be reached at http://www.processimpact.com. Enjoy these free requirements management resources.
Subject matter expert Karl Wiegers writes on best practices for writing requirements, requirements management, and requirements traceability.
Latest posts by Karl Wiegers (see all)
- Best Practices for Change Impact Analysis - September 12, 2022
- Characteristics of Effective Software Requirements and Software Requirements Specifications (SRS) - May 30, 2022
- Defining and Implementing Requirements Baselines - June 18, 2019