There has been no shortage of press surrounding the HealthCare.gov October 1st release. Much of the debate points fingers at individuals often skewed by political leanings and party affiliation. Regardless of the political circus there are important lessons to be learned that highlight yet another glaring example of the importance of open, iterative collaboration around building products. This is not a new problem, nor is it unique to government. This is a problem faced by some of the world’s largest companies. Both Microsoft and Apple have recently experienced less than stellar rollouts: Surface estimated as a $900 million mistake and Apple Maps, which prompted a $30 billion loss in stock shares.
The fact is that when developing massively complex products in a broken system with insufficient tools, issues will be amplified. In the fallout of the much-anticipated HealthCare.gov release, the typical reactions are taking place; blame the most senior person available and throw tons of last minute resources at the problem in an attempt to fix it. None of this will work because the damage has been done – the source of the problem is cultural and ingrained in the process. Anthony Wing Kosner’s piece in Forbes discusses one aspect of the problem centered around the definition of the requirements themselves. Kosner discusses the fact that an estimated “40% of the defects that make it into the testing phase of enterprise software have their root cause in errors in the original requirements documents” and that government project requirements can be especially difficult to manage.
I have experience working on an implementation that suffered similar issues, and am especially frustrated with the state of HealthCare.gov because since then I’ve been reading about the many initiatives to incorporate Agile or Lean concepts into government software products. There are instances where taking an open, collaborative approach has paid off; one great example is the implementation of Recovery.gov and FederalReporting.gov. The success stories incorporated iterative processes that enabled changes to be more easily and effectively implemented, as opposed to HealthCare.gov, whose architects have cited changes in requirements as a root cause of issues.
Each set of changes created a new version of the document that needed to be shared across many teams. For example, Andrew Slavitt, vice-president of Optum explained to lawmakers about the late decision requiring consumers to register for an account before browsing insurance products. This highlights the typical occurrence of change, but begs the question: is the late decision the problem or is it how that decision is managed and ultimately communicated to the rest of the team? Compounding these problems is the fact that especially in government there are multiple contractors involved that are not incentivized to work together. The success of the their work is more important than the success of the project as a whole because contracts and payments are based on those deliverables. This fosters an environment of CYA as each contractor spends more time making sure that they are covered based on their individual contract and less on the idea of building the right solution.
The sad thing is that it’s hard to blame the contractors, developers, QA or even the government. The problem is in the status quo we all accept. Organizations continue to resist the inevitable need to make overarching changes to their process and tools to move away from such avoidable chaos and lack of communication. Based on this fact and my past experiences, there are 4 key lessons to be learned from the HealthCare.gov story:
- Initial expectations and visions are often too vague and lofty. In the instance of HealthCare.gov, acceptance of change was not considered early on. Implementing such a complex system is not as simple as defining a set of requirements and pushing them downstream to be built. It takes a highly coordinated effort full of constant communication and realignment to change. There needs to be a conversation about how to handle unpredictability.
- Allow for voices to be heard sooner. If there are no clear paths of communication that connect the business side to the implementation and test side there is an increased risk of misalignment, discontent, frustration, and delays that ultimately create a fast moving train of costly issues.
- Allow for innovation to thrive throughout. Were there options to rethink the requirements themselves? One such possibility was the requirement for each user to receive an immediate confirmation based on complex integrations and system checks. Would it have sufficed to let the user know their information was accepted and would be processed over the next 7 business days?
- Tools. If those who built HealthCare.gov are anything like many major organizations and companies they were likely relying heavily on Word documents to send requirements. When will we learn that this is an archaic means of communication? This method does not in any way help with managing the constant influx of changes notorious with government contracts.
My own experience working on a government healthcare software project took place 6 years ago when I worked for a contractor who was tasked with designing a state Medicaid eligibility program. The realities and problems my team faced back then have a striking similarity to the failings of the HealthCare.gov rollout. I was a developer at the time and was part of a team that had been put together based on a set of requirements that were written as an RFP and awarded to the contractor. Based on those initial requirements, estimates were created and resources were assigned to the project. As is common in RFPs, a year passed between the award and the final team being assembled, during which requirements changed. This common occurrence is most often the initial point of failure as teams quickly fall out of alignment, while scope and schedule became a constant topic of debate.
As a development team, we worked in a very Agile manner and provided frequent working software to the customer. Still, the complexity and volume of changes made it very difficult to be efficient, the schedule changed often and resources were staffed up in an effort to increase velocity. Luckily we had some flexibility with dates and a lot less visibility nationally.
Why does this status quo continue to dominate and dictate how we implement projects? The government is not alone in the challenges they are facing in so many products and programs. At my company, we often receive RFPs that are frankly outdated and irrelevant. The RFP model is broken and the idea that requirements are to be used as a binding contract that if changed puts both parties at odds, prevents teams from doing what’s right and speaking up if they sense something is wrong. RFPs are not the only culprit, requirements used as a binding contract set the tone. The agile manifesto highlights this point: “Customer collaboration over contract negotiation.” This isn’t to say there isn’t a need for contracts. HealthCare.gov may have failed just as much had there been no contracts involved. It’s easy to say generically that an agile approach would have done better, especially considering many over emphasize the desire to eliminate requirements all together in lieu of complete “we’ll figure it out as we go along” mentality.
The balance is in providing the goals (the requirements) in parallel to a more open, iterative, and collaborative process that is necessary in order to deliver products that fulfill requirements and are completed on time. The requirements and decisions themselves should have been in a place that was available to all that could handle the complexity and balance of requirements that change the speed necessary to make critical decisions when they can’t be changed. We are at a day and age where how we collaborate has fundamentally changed. Organizations need to take advantage of both the concept of agile – keeping in mind the key quote “…while there is value in the items on the right we value the items on the left more.” – and tools and technology built on modern methods of communication. Had this been in place for HealthCare.gov, changes to requirements would have been clear, communication would have been open, and decisions made would have been communicated immediately. All of that information would have been easy to track and maybe we wouldn’t be in this state of constant blame.