Three Primary Principles of a Software Release

“Delivering a release is a little like wrapping up a present and giving it to our customers” – Maarika Krumhansl, Release Manager at Jama Software

When I mention to folks outside of Jama that I’m a Release Manager at Jama, the most common reaction is “Interesting!” and then shortly thereafter “…What does a Release Manager do?”

Release Management means slightly different things at different companies. Some companies employ DevOps Release Engineers instead of Release Managers. Some companies roll the Release Management function into the Product Team. Other companies have their build, test, deploy, documentation, and customer communication so streamlined that they have no need for a Release Manager. I personally come at Release Management from a DevOps background. In a previous job as a Deployment Developer I had the opportunity to build that company’s first Continuous Integration pipeline. I was also responsible for releasing and packaging a Java application for production deployments. I am a huge advocate for Agile methodologies and my Release Management philosophy is heavily based on personal experience as well as learning from the industry leaders.

Regardless of who or what process performs the role of Release Management, it is based on three primary principles: Traceability, Reproducibility, and Measurability.

Traceability: The ability to see how one piece of information – e.g. a requirement, a story, a git commit, an automated test run – connects to any and all other relevant pieces of information in a release, either upstream or downstream in the item hierarchy (or forwards/backwards in the workflow). For example, a release is traceable if any member of the organization is able to see which epics are shipping with a release, the specific stories in those epics, and any bugs or defects slated to be fixed. For each ticket (story or defect) in a release, it is also possible to determine exactly which git commit(s) represents the work done to satisfy the requirements, who performed the code review and the desk review, and whether the automated unit-, integration and functional tests passed against that commit.

Reproducibility: At its core, this is about the ability to generate an exact copy of (i.e. reproduce) a release of Jama. A release is made up of multiple components, including the actual binary artifacts, the deployment method/scripts, the documentation, and the environment configurations. Binary repositories (e.g. Nexus, Artifactory, etc) provide reproducibility of artifacts, and by keeping build and deployment scripts – as well as standard environment configurations – in source control (“Infrastructure as Code“) we can guarantee reproducibility of installs / instances of a release.

Measurability: The ability to determine the “state” of a release at any moment, either in development or in production. While a release is in development, it is important for all stakeholders to have a clear view of the progress being made and the “health” of a release, including things like: How many tickets are still open/in development/in testing? What is the test coverage? What are the results of the automated regression and performance testing and how do the results compare to previous runs? Once a release is live, it is our responsibility to monitor and measure its performance compared to previous releases and to remediate any unexpected behavior (if needed). Numerous tools exist to help with application performance monitoring, server-side resource monitoring, logging and parsing of errors, etc, but these tools are only helpful if 1.) they are measuring the right things, 2.) they have visibility (e.g. alerts/triggers set up, well-designed dashboards, people actually looking at them, etc.) and 3.) they are reliable (e.g. provisioned with enough resources, few numbers of false positives).

It is the Release Manager’s job to ensure the Traceability, Reproducibility and Measurability of software releases. Ideally this is done by implementing tooling and automation but in the worst case some of it must be done manually until the pain of NOT automating the task is far greater than the up-front cost of scripting it. Case in point: Currently at Jama the process of producing a Manifest Check (i.e. the document that proves that each ticket slated for the release has at least one git commit implementing it, as well as verifying that each git commit is implementing a ticket planned for the release) is manual and tedious, involving:

running a bash script to diff the commits in the current release from the last release,
parsing the commit messages for ticket numbers and loading those ticket numbers into a spreadsheet,
cross-checking the tickets in the spreadsheet with the tickets intended for release, as reported by our internal install of Jama,
working with Engineering and Product to resolve any discrepancies by either adding tickets to the release that were overlooked originally, or identifying which commits may have implemented code for multiple tickets.

As you can imagine, this process is time-intensive and non-scalable, since Jama already has multiple code repositories. As we plan to move towards a Service Oriented Architecture (e.g. “microservices“) the number of repos is expected to explode. Clearly the current manual process is no longer tenable. At a recent Jama Hackathon a team of developers and QA engineers developed a proof of concept service that will automate all of the tasks in the above list, and Product has added this work to the overall product backlog (to be prioritized against other strategic initiatives) as an add-on service for Jama.

What I love about Release Management at Jama is the diversity of responsibilities and technical challenges. It is fascinating to witness and assist Jama transform from a monolithic architecture to a service-oriented architecture, and ultimately support a more container-ized, continuous deployment paradigm for our Hosted releases. Additionally, I am learning an enormous amount about the state of the art in on-premises deployment technology – i.e. Replicated and Docker. As a Release Manager concerned with Traceability, I am fortunate to be able to use Jama to build Jama, since this is exactly what Jama was built to do! I work with people from across the organization daily as I perform general project management for releases, and I get to be a spokesperson for process improvements and CI optimization, helping to drive initiatives such as modernizing our binary repo and establishing and enforcing our git release branching strategy. We are also starting to implement slow rollouts of some of our features to small subsets of our customers, also known as “Canary Releases“, and we are pleased with the feedback and data we have been receiving about this effort.