Agile development has proven a huge success for building functioning, innovative products quickly, especially software.
It is not surprising, then, that many organizations are looking for ways to apply it in areas beyond software.
In fact, this is already happening. WikiSpeed, for instance, is designing, building, testing and selling cars on a weekly scrum cycle. But would you feel comfortable in a WikiSpeed car?
There’s a long tradition of developing safety-critical systems like planes, trains, cars or medical devices. (There is the saying that federal aviation regulations were “written in blood.”)
So, it is understandable that we do not want to discard decades of experience at the risk of human life.
Not Either/Or: Both!
The foundation for functional safety development is IEC 61508 or a related standard. Key to these types of standards is the management of functional risk, which in turn requires a clear structure of the development artifacts to act as the foundation for risk analysis. Almost all standards build on the V-Model as the supporting structure, but the V-Model is often misunderstood.
Contrast this with Agile approaches, which rely on rapid development cycles. Agile methods de-emphasize the development structures, as stated in the Agile Manifesto: “Working software over comprehensive documentation.” To be fair, specific frameworks like the Scaled Agile (SAFe) fill this gap.
The challenge is to combine the principles of functional safety development with the advantages of Agile product development, without sacrificing the former. The following figure visualizes this conceptually:
We see the V-Model is the underlying structure, overlaid with an Agile continuous engineering process. For organizations that currently develop products, the V-Model is already established. The challenge is to overlay it with the Agile engineering process, without jeopardizing the existing achievements with respect to functional safety. Here are five catalysts that can help you get there.
Enabler 1: Fine-Granular Items
A surprisingly high number of organizations still use documents for various development artifacts: requirements specification, system requirements, design, etc. For a small number of items, a Word or Excel file seems adequate. But there are two problems: First, with increasing complexity, the number of items increases. Using documents does not scale.
The second is directly related to the topic of this article: Document-based development makes each iteration very expensive and creates a large overhead. Specifically, with document-based work, there is no clear understanding on what has changed within the document, and how it affects related artifacts. This, in turn, means that the complete system description must be reviewed and analyzed, not just the pieces that have changed.
Item-based work requires appropriate tool support, but also a corresponding mindset.
Enabler 2: Up to Date, Actionable Traceability
Once an organization adopts an item-based mindset, it is critical to understand and capture the relationship between the items. This is commonly known as traceability.
If maintained properly, traces help to drastically reduce the effort for an additional iteration: They point to the impact of changes from iteration to iteration, and ensure that only the “delta” between iterations need to be analyzed.
Traceability should cover the whole development chain from requirement to test – end-to-end traceability. This ensures that only the changes have to be re-tested, not everything else.
The hardest thing about traceability is keeping it up to date. This requires the right mindset (again!), as well as proper tool support that does not get in the way and is “actionable” — i.e. problems in the traceability itself and easy access for fixing it.
Enabler 3: Seamless Collaboration
With traditional development, problems are often discovered only at the end of a development cycle, especially if they relate to other subsystems. But collaboration across subsystems should happen continuously, so that issues and questions are resolved as soon as possible.
Again, this requires the right mindset, in particular the willingness for transparency and open communication. But the tools and processes must support this as well. If the tooling alerts developers immediately to the change of an upstream requirement, engineers can resolve potential problems right away.
Enabler 4: Decouple the (Sub-)Systems
It’s a nightmare scenario to have one technical change result in an avalanche that makes the whole product obsolete. A safeguard against this is to decouple the various systems and subsystems as much as possible. Treat every subsystem as a “black box” with a clearly defined interface and behavior that does not depend on the implementation.
This approach needs a good understanding of behavior-driven development by the team, but if implemented right, it can become a game changer. In addition to making the system much more robust, it also makes reuse much easier and efficient.
For sophisticated systems, effective black-box design may require specialized Model Based Systems Engineering (MBSE) approaches and tools.
Enabler 5: Different Speed for Software and Hardware
In addition to decoupling, software and hardware development can happen at a different pace. It’s a fact that hardware changes slower than software, and the cycle speed should reflect this.
Working at different speeds makes it even more important to align on a regular basis, and to collaborate continuously. In practice, this means that hardware and software have common stakeholder requirements, and that changes in those must trigger a collaboration with software and hardware groups to react accordingly.
It took decades to learn how to build safe products – let’s not discard what we have learned. Instead, let’s augment existing practices with new insights from Agile development.
To apply them, we need a new mindset, new tools and the willingness to incrementally transform the way we develop products. The five enablers shown here provide some guidance on how to achieve this.
If these ideas resonate with you, consider attending ECS 2018 in Stockholm, Europe’s largest embedded conference. There, author Michael Jastram will deliver a talk, “Agile Development of Embedded Systems for Functional Safety – a Contradiction in Terms?”
Putting the development methodology into consistent practice is a lot more difficult to sustain than many realize, and teams often have trouble sticking with it.
Plenty of patience and discipline is needed to really see results from Agile. If your team has neither of those qualities — and, given increasingly demanding timelines, that’s not unusual — you’ll likely see regression into familiar patterns. You may also end up with some sort of hybrid, homebrewed methodology that’s unique to your team and falls somewhere in between Agile and Waterfall (“Scrummerfall,” if you like) but hasn’t been optimized for success.
According to author and consultant Greg Cohen, engineering processes associated with Agile — such as Extreme Programming (XP) — need to be maintained for months and months before progress is made. We talked with him a little bit about that, as well as why Agile might not be the best fit for every team.
Jama Software:What are some trends you’re seeing within product development teams at the moment?
Greg Cohen: What’s more interesting is the negative trend. And that’s the idea that seemingly all teams at this point almost have to say they’re Agile.
Everyone’s claiming to be Agile, but what I see is that the majority of teams are really struggling with it. And I heard a great quote from one of my clients recently. They described their processes as “somewhere between Agile and the Wild West.”
What I find is that there are far too many teams that are more ad hoc and not Agile. You know, they’ve thrown away the discipline and structure of Waterfall, but they haven’t really adopted the discipline of Agile either.
So they’ve gotten rid of process and haven’t embraced the empirical process that Agile requires. And it really is a very disciplined process and requires an immense amount of work to do well. And to stay focused and keep up with everything you need to do. So I think teams struggle with that.
JS: That’s not all teams, of course.
GC: The teams that I see that are highly performing, highly productive, they really get the productivity boost out of Agile. They get the quality boost out of it. But in order to get all that, you have to do the engineering practices, which these days often include terms like “Scrum with XP,” and particularly test-driven development and continuous integration.
And I feel, in some ways, Scrum did a disservice because it’s a process framework. And they never said you need to do these engineering practices. They didn’t want to specify that to the teams.
So at the same time teams adopted Agile, they sort of took on what I would say are the easier events and rituals of Scrum, without making those deeper changes in engineering that are really required to become a high-performing team.
JS: What are some of the challenges teams run into when transitioning to Agile, where they land in between methodologies?
GC: To do incremental development, you need to know that what you’re putting into production, or even QA, isn’t going to break the work that’s already been done. And if you don’t have that ability, it’s just not scalable to do it with standard testing or big regression tests.
I think that’s the problem teams get themselves into: they just can’t move fast. It’s hard and it gets chaotic.
From a development standpoint, there are engineering practices that are essential. Those are like XP practices, test driven development, and continuous integration. And these are what allow teams to work iteratively and safely.
So if you have a large legacy code base? It takes time to build up the test cases. And sometimes you have to create an abstraction layer so you can sort of isolate that legacy code. And slowly replace and refactor it, as you work on upgrades and new features and capabilities without risking some unintended consequence in that code base.
When you talk about older organizations, they’re dealing with legacy code, and that’s a much harder problem than just greenfield development on a new product.
JS: What are some development environments where you might say Agile might not be the best fit?
GC: I had this client once and there were two projects we were talking about.
One of them was critically important to the company and it needed to get to market quickly and it was new development.
They had another product that had been around for many years and the team did one major release a year. They followed a Waterfall method. They consistently released on time and they released with the planned scope. It was a well-oiled machine. And those releases came at an appropriate cadence for the market. It was an older code base. It was legacy customers. They weren’t out marketing and selling this product. They were just supporting their customers on it and continuing to improve it as those customers wanted.
If you’re talking about going to go through this change in process by moving to Agile, you’re going to experience an initial drop in performance. Because any time you do a big change, people have to learn new roles and skills.
For this client, would I recommend they switch to Agile for that legacy product? I think that’s a very tough sell.
Could they ultimately improve the productivity of that team? Absolutely, but they’re selling to an already captive audience. The market is static; it’s not changing quickly and there’s little competitive pressure. If you look at that, and ask which project should I apply Agile to, it’s not the legacy project.
Maybe if the entire company masters Agile development through their cutting-edge product? Maybe at some point it’s worth switching this other team over to Agile, but it’s not where I would start.
JS: No need to fix what’s not broken if they’re hitting their marks.
GC: Exactly. They’re hitting their marks, the market wasn’t very dynamic and there wasn’t a lot of competitive pressure. If that’s the case, stick with Waterfall because they were executing very well with it.
If you are building complex systems, then you have probably heard of the V-Model. Before we explain it, a word of caution: There are different, partially contradicting definitions of the V-Model. Further, it’s easily misunderstood, which can be more harmful than helpful. This is also the source of the myths of the V-Model. Understanding the Myth of the V-Model helps you apply it correctly, thereby making you more effective.
The V-Model in Brief
Above is one visualization of the V-Model: To understand it, we start in the top left. Here you find the high-level requirements (“Concept of Operations”). Following the left arm of the V, the product definition is refined until we reach the implementation. The right arm of the V contains the corresponding items for validation and verification, like tests, analysis or inspection.
What’s unclear about the model (and frequently a source of confusion) are the relationships between the elements, visualized through arrows. Note that the various arrows have different meanings:
The solid lines connecting the boxes represent dependencies. For instance, there is a relationship between “Requirements” and “Detailed Design” (the Detailed Design is a possible solution). If the requirement changes, then usually the design needs to change as well. For instance, changing the battery life from 24 to 48 hours (requirement) will probably require a larger battery (design).
The dashed “Verification & Validation” lines also represent a dependency, but this time for test coverage. Note that the arrows in the figure represent a “tests” relationship (“System Verification & Validation tests Requirements & Architecture”). The direction of the dependency is in the other direction (e.g., changing the battery life from 24 to 48 hours will trigger an update to the corresponding Systems Test).
The outer dashed lines denote activities, and broadly represent time. But here we have to be very careful because they do not represent a strict time sequence. This is a huge source of confusion.
Now that we have a rough understanding of the V-Model, let’s look at four of its common myths:
Myth: The V-Model is Only an Extended Waterfall
If we take the time axis too literally, then the V-Model appears to be the Waterfall Model (the left side of the V), with the verification “folded” up. Just as a reminder, the Waterfall model is a linear development approach. It consists of different phases (requirements, design, implementation, etc.), where each one is completed before the next starts. The main difference between the V-Model and Waterfall is the fact that the V-Model iteratively increases the maturity of the development, affecting all items of the product description.
Myth: The V-Model is Simply a Process
This statement is correct, but not complete. Yes, the V-Model is a process. In fact, it is widely used, cited and standardized in various ways: There is the German “Das V-Modell,” a government standard and PRINCE2 method, which is used in the UK and US. Many standards reference it, including ISO/IEC/IEEE 15288.
But this view focuses on the process and ignores the fact that the V-Model also represents the artifacts of product development and their relationships. This second view can generate additional insights, especially if the artifacts are fine-grained: For instance, traditionally the box “requirements” in the figure above would represent a requirements process, with a process flow to the design process.
But if these represent individual requirements, with a precise traceability to design elements, this relationship can suddenly be used for coverage analysis (“ensure all requirements are covered”), change management (“after this requirement change, which design elements are affected?”) and much more.
Looking at both — the process and data model — significantly increases the value of the V-Model.
Myth: The V-Model Cannot be Applied in an Agile Environment
This myth is related to the previous one: The existing methods based on the V-Model were created in the ‘90s, before the Agile Manifesto was created.
But the V-Model itself is not just compatible with Agile methods. It’s (almost) a prerequisite! To be more precise, a robust relationship model, like the V-Model, is the foundation for making changes with confidence. And frequent changes are one of the aspects of Agile development. A robust relationship model reflects the triangular relationship between requirement, test and implementation.
Myth: During Development, We Move from Left to Right
As already indicated in the two myth we busted, the V-Model requires jumping back and forth during development. Nevertheless, the assumption is that we need requirements before we can create the corresponding tests.
But the V-Model is well-suited to also support behavior-driven development (BDD). In BDD, the expected behavior is defined in a testable manner. Design only starts after the behavior is defined.
This corresponds to test-driven software development, where an automated test is written before the software implementation is created. For BDD to work, the relationships between the items concerned must be clearly defined and available for analysis and change management.
The V-Model provides the necessary structures for this process. In fact, this triangular relationship is visible in the V-Model’s triangular form, except that BDD uses a slightly different terminology.
The V-Model and Product Development Platforms
The V-Model can be confusing, as it relates to both process and data models. Product development platforms like Jama Software distinguish between these two aspects and address them separately:
Works with a flexible data model consisting of item types and relationships. These can be tailored to existing artifacts and workflows. This allows the product description to evolve naturally, while managing the relationships automatically. The results are a clearer understanding of coverage without the fear of changes.
Allows workflows to guide the maturity of the product description, aligned with existing processes. This creates the confidence in the quality of the product, makes it easy to work in a compliant environment and takes the anxiety out of audits.
Conclusion
The V-Model is a powerful concept. But it has been around for a long time: It was created in a time where document-based work was the norm, and where Agile was not yet an established approach. Applied in a modern context, the V-Model is the enabler for the faster development of better products. But in order to take advantage of this, it must be understood.
Heightened efficiency, quicker time to market and risk reduction are all compelling benefits. That’s provided teams can actually stick with the discipline required to make the methodology work for them.
Either way, organizations shouldn’t substitute Agile as a product strategy, according to author and consultant Greg Cohen. Although that hasn’t stopped some from trying.
“You can be so responsive with Agile, you can get away without a strategy in the way you couldn’t in the Waterfall world,” Cohen says.
While Agile as a development methodology can give you a distinct edge in many areas, it’s woefully inadequate as a standalone strategy for market success.
A team may think it can rapidly develop and release a product, for instance, leaving strategic decisions like plans for rollout, marketing, distribution and partnerships until it’s available to the general public. Unfortunately, that’s not a path to profit.
We recently chatted with Cohen, who runs Agile Excellence, and he expanded on this key distinction.
Jama Software: You’ve said before Agile is not a substitute for product strategy. Could you elaborate a little bit on that?
Greg Cohen: If you think about what Agile is, it’s absolutely a superior process for development and working in an uncertain and dynamic environment. And without a doubt a high performing Agile team is a competitive advantage.
So in that sense, a superior process can be part of a bigger strategy. Agile lets you learn, out develop, out innovate your competition — if you can use this process better than your competitors.
But even knowing that, do you know how many times the word ”strategy” shows up in the Agile Manifesto or The Scrum Guide?
JS: Zero.
GC: Right, exactly zero times. And, in that sense, Agile isn’t a strategy. Its creators didn’t consider it that way at all.
And that’s why I wrote this new book that came out in September, Strategy Excellence for Product Managers. Because what I saw was teams were losing the hard skills of product strategy and just sort of relying on Agile flexibility to react to the market. And no longer leading it.
But product strategy is about analysis. It’s a deep analytic, intellectual exercise. And there are five elements in it:
Your customers and their unmet needs
What’s going on in the market – is it growing or shrinking?
What’s your competition doing and what advantages do they have over you versus you over them?
Technology trends – how quickly is it being adopted and how fast is the cost curve dropping on any given technology
And, ultimately, what is your business strategy because product strategy needs to feed into the business.
As the other thing to realize is what I’ll call the instantiated product: what’s coming out of development? It’s only a small part of an overall product strategy.
So product managers need to concern themselves with what we call the “whole product,” which is this set of services that surrounds the product and turns it into a solution.
Because customers buy solutions to their problems. They don’t buy products.
And product managers, therefore, have a lot of what I call leavers to pull to influence the success of a product.
There’s what’s coming out of development, the feature set, but there’s also things to consider, such as:
How is it priced?
How do we promote it in the marketplace?
What are our distribution channels?
What’s the buyer experience before you’re even using the product?
How do we onboard our customers?
How do we train them?
How do we support them and what experience do we give there?
And even bigger things like partner ecosystems, which turn our instantiated product into a bigger solution and fit to solve a customer need.
You have to get all of that right to be successful. So in that sense, Agile addresses the development side. Certainly, Agile-thinking can permeate into larger decisions about how the company structures, does governance and organizes, but it is not a product strategy and no one should confuse the two.
Just because your embedded systems team commits itself to adopting Agile, there’s no guarantee your next release will hit all its targets. Product delays, quality issues, and missed market opportunities may still be lingering issues.
This isn’t an uncommon scenario. While there are many benefits to Agile (e.g. better products, more efficient development, increased transparency) it might take more time than expected before your organization is fully on board. Other times, Agile just isn’t the best fit for your team.
Either way, if your business is experiencing roadblocks with Agile, it’s worth examining your development process before scrapping it, as there could be other issues plaguing your execution.
For instance, try assessing whether or not the requirements you’re working under are too vague. Those kinds of communication issues can lead to wrong builds, which will cause delays and force teams to roll past due dates. In that type of scenario— where market opportunities are being wasted each day the product isn’t released— pressure to deliver may translate into significant stress on team members. An environment like that can, in turn, increase the likelihood of product defects and failure.
Agile is supposed to help break this vicious cycle. In our recent webinar, James Grenning — founder of Wingman Software and co-author of the Agile Manifesto — lays out some of the common pitfalls of Agile for embedded systems. We’ve listed a few of the problems below, and they’re broad enough to apply to several different development disciplines. If you’re interested in diving deeper, definitely check out the webinar.
Not Testing As You Go
By now, most development teams understand the need to be testing throughout the process. Since Agile was created to reduce inefficiencies, it’s still worth repeating: if you’re not testing as you go, you’re running the risk of causing major backups later, whether it’s bug hunting or discovering that what used to work is now broken. Whatever the issue, it’s usually a lot easier to prevent a problem at the beginning of a development cycle than scramble for a fix at the end.
Communication Breakdowns
Agile is intended to increase transparency and visibility throughout the development process. Some of this can be managed through data, but it’s also reliant on team members being proactive when communicating progress. For example, if a feature is lagging behind schedule, team members need to have the confidence and freedom from harsh consequences to say so. Conversely, if a project is running ahead of schedule, making stakeholders aware allows more time for additional work on other areas.
Everything All At Once
In order to maximize the accuracy Agile can offer, focus on near-term rather than long-term plans. That way, you’ll be able to pace your way through a project, deliver working capability quicker, and refine the bigger plan as you go. Plus, prioritizing items incrementally will help you stay on course and reduce the amount of scope creeping in.
After learning about the Agile Fluency Model at ProductCamp, I began to wonder where Jama Software’s product-development organization fell within the framework. While I could see evidence of our fluency in the first and second levels, I was uncertain about our competency in aspects of the third level.
At Jama Software, Agile Software Development practices have been in place for several years. In the past few, we have made a series of investments in both our engineers and the broader organization.
In particular, we’ve focused on refactoring workshops, continuous integration, test-driven development, pair-programming, collective code ownership, and DevOps culture. We ship software at a cadence that makes sense for the various markets we serve. Some markets have a tolerance for monthly releases, whereas others operate on a biannual schedule in accordance with their regulatory environments or risk tolerances.
Jama Software’s Approach to Agile Software Development
One practice we’ve adopted, which is consistent with a three-star team, is we have a system of self-organization within our engineering group. Engineers are free to move to different projects and focuses as their interests change. At the same time, we provide strong support for gelled-teams, which consist of individuals whom enjoy working with one other and have gained an efficiency in doing so in particular areas of the product.
An important tenet of a three-star team is that business experts have permanent memberships. So each development team on a consumer-facing project works with a product manager to represent customer and business needs, for instance, or a UX designer to ensure a feature meets design quality. However, those individuals aren’t embedded or collocated with the team. Instead, business experts often work with multiple teams at the same time.
Though we don’t fulfill the criteria of having dedicated business experts and designers embedded on our teams, at this juncture in Jama Software’s growth, it’s appropriate we operate this way. While we’re certainly no longer a neophyte startup, we try to stay lean in our operations. This ensures team members at all levels of the process have a better understanding of what we’re building and why. Practices such as including the engineering team early in the user research process, and brokering out what exactly is going to be built, is an important counter-weight to the direct embedding of an individual for a specific project.
Staying flexible with Agile Methodologies
As we continue to progress as an organization, there will be some practices of a three-star organization that we’ll strive to achieve for the benefit of our own agility. As I mentioned above, we release on a monthly cadence to our hosted customers. For our on-premise clients, we follow a six-month schedule. While Jama Software has a release schedule that may appear longer when compared to consumer market applications, “move-fast and break things” isn’t a mindset that would be appreciated by the companies using our product.
Jama Software builds a solution for customers developing complex, mission-critical products, many of which are regulated. Bugs in our product development platform can lead to bugs in their products, which can result in both financial and fatal consequences. Many of the companies with this kind of regulatory overhead engage in mutual validation of our product when new releases are made available, and only bring them into use across their organizations once it has passed internal controls.
To this end, while continuous integration is an important part of our engineering ethos, and we would like to reap all the benefits of continuous delivery, it’s unlikely that we will push updates of our product to customers multiple times per day.
For software companies making use of Agile Methodologies, it would be a disservice to consider their application an all-or-nothing situation. Applying the practices appropriate for your business context is the most pragmatic approach. However, it’s also important to continually challenge traditional methods of project management to gain the benefits provided by Agile.
One suggestion for those who want to up their game in Agile is to choose a practice and focus on integrating it into your process until you achieve mastery in it. Just as with software development, process improvement is a practice best undertaken with incremental and steady approaches.
Being one of the people that participated in the creation of the Agile Manifesto, I find myself very disappointed by the reaction of engineers to the question “Are you practicing Agile?” Their shoulders drop. They start to slowly shake their heads. They mumble; they grumble. They tell me agile is horrible. I ask why. Reasons I hear most often are:
We’re being micromanaged
The pressure is constantly on, these two-week deliveries are too short
All they care about is the date
We only have time for cr**py code, we can’t do it right!
None of those management behaviors are part of Agile (read the Agile Manifesto and its related principles for yourself). Unfortunately, the dysfunction the authors of the Agile Manifesto were rebelling against are alive and well today in most places that claim to be Agile.
Too many supposedly Agile shops are missing a lot. They are not even doing plain old Scrum. If they were, programmers would be better off, for example by:
Self-organizing teams manage themselves
Choosing your work lets you follow your interest (sometime you have to take less desirable work)
Committing to how much can be done in an iteration
Updating each other in the stand up. (It’s not a manager update, though they participate.)
Talking responsibility for design and other technical decisions (being professionals)
Facilitate continuous improvement by doing meaningful retrospectives (being honest with yourself)
The motions of Scrum can help, but they are not enough.
Engineers, programmers, developers! We need to build trust. When we ask for three months to deliver, what happens after 2.5 months? We give the bad news that we’ll need an extra month. Bad news late is the worst kind of bad news. Your trust takes a hit. Then it happens again as you near the new deadline. Your reputation is weakened further. It looks like you are really busy now! You claim to be done and then the bugs start rolling in. You fix the bugs and unknowingly break previously working features your customer relies on. Your trust and reputation is in the trash.
Can we get out of this mess?
With Agile, the cadence of delivery is changed, so must your practices change. Kent Beck told me long ago (paraphrasing): “You’ll never be able to figure it all out up-front. You’ll never be able to stop changes even with a signed requirements document and contract defined penalties. Things will change. That’s the world. If you can’t beat ’em, join ’em. Get good at dealing with change.”
Agile 2016
As it turns out, Agile as practiced is dominated with management in 2016. Most the people that were involved with the Agile Manifesto were programmers, doing what they could to make the world better for programmers and the people that need programs.
Programmers, you have a professional responsibility to improve how you work. We (programmers) better get our act together; we’re just a few more software caused problems away from having the lawyers and government in our business. We better learn to be professional, or some lawyer or government bureaucrat will tell us what professional is. Agile might not be the whole answer, but it does advance the state of the art. It introduces the two week cycle and we can and should use that cycle for continuous improvement. Serious technical improvement, not lip service.
Managers, Scrum masters, and POs do you encourage your team to learn the technical practices of Agile? Do you even know what they are and why they are important? Here are two key areas you cannot afford to ignore if you want your team be more successful and make your customers happier:
These are only a start. But they are foundational.
Agile 2017?
That is up to you.
About the Author: James Grenning trains, coaches and consults worldwide. James’ mission is to bring modern technical and management practices to to product development teams, especially embedded systems development team. He is the author of Test-Driven Development for Embedded C. He is a co-author of CppUTest, a popular unit test harness for embedded C and C++. He invented Planning Poker, an estimating technique used around the world, and participated in the creation of the Manifesto for Agile Software Development.
It is no secret that Agile methodologies have changed the way software teams organize and have had an even greater impact on the speed and flexibility of the product cycle. In the old days, software was developed in silos between independent teams and put together at the very end. This resulted in clunky, redundant programs that had many errors that had to be reworked. Early versions of Microsoft Windows with tens of millions of lines of extra code is the classic example of a poor development process.
Today, Agile software development has taken over. Agile teams build their products in concert with one another and have visibility into everyone’s progress. Feedback is constant and any problems or duplications are discovered before the product is ready to launch. That allows a much more elegant product to emerge in a much shorter amount of time.
Most people associate the Agile method as a practice followed by software teams. This is changing thanks in large part to the overwhelming market for smart, connected products. Case in point, look at the automotive industry. It’s no secret that it is undergoing a revolution. Between radical technological innovation, new safety standards and shifting consumer attitudes, manufacturers must rethink and adjust their strategy. These changes have made automotive development much more complicated, forcing design teams to understand these complexities and adapt, adhering to new safety regulations such as ISO 26262, and prove compliance. As a result, there is a far greater need to use modern tools and technology to further evolve how they align teams, verify functional safety and ensure they are building the right products.
Applying Agile development to hardware requires a few basic parameters. First, everyone needs to be on the same team to see plans, progress and calculate the physical effects of one part on the rest. The high-costs associated with hardware development means it is especially important to have transparency. Altering a product once the architectural decisions are completed does not mean erasing a few bits of code and rewriting them. Instead, you are in the world of atoms and you may have to order new parts, apply heat to weld or add more microchips. All these add expenses and end up affecting the final price, and impact your bottom line. Applying Agile development processes will inevitably lower the price for hardware devices and result in more profit for the companies or lower prices for end consumers.
Hardware Agile development includes seven steps, as software Agile development does. Yet, the process is slightly different. Hardware Agile development starts with a design interface and moves on to a low functionality emulator. That is followed by a medium and finally high functioning emulator for the team to see. There are two layers of full functionality prototypes and finally a fully integrated testable device. The team shares an open platform to get to the product they are seeking.
The manufacturing process itself has also improved by leaps and bounds in recent years. Ever since Motorola developed its Six Sigma standard, companies have endeavored to upgrade their manufacturing speed, quality and accuracy while reducing price. Agile development is the next stage in this process as all team members can contribute to productivity improvements on the manufacturing line.
Test Driven Development (TDD) was introduced more than a decade ago by Kent Beck and has been widely adopted by effective, agile development teams. Still, after all this success, it is surprising how often we miss the reason for TDD. Here’s the surprising thing: Tests are not the point of Test Driven Development, they are a useful byproduct. When plants perform photosynthesis they produce oxygen as a byproduct, and we are all thankful for that. We are also thankful that TDD produces vital tests for our code, but the point of TDD, and the reason you should adopt it is something else.
TDD Refresher
TDD has 3 simple steps performed in short, tight cycles (perhaps as short as 30 seconds each when you are an expert):
In step 1, you are asking yourself, how do I communicate with this code and what is the most useful result I can expect back? Because you are writing the test first, you start by envisioning the best possible interface to the code and the simplest, most useful results. As you implement the code, reality and other constraints may nudge you off of this ideal, but it is always the starting point.
For step 2, you write just enough code to make the test pass. Often, this will be a completely fake implementation just to get the test passing as quickly as possible. Red (i.e. failing) tests will tend to make you twitch in discomfort. If the real code necessary to turn the code green isn’t right at your fingertips, implement something less real but satisfying the test’s demands. Now you have a functioning test that is protecting you from straying off the path of green.
Step 3: You have a passing test, congrats! Now with the code constrained by the need to keep the test green, you refactor the code to make it conform to the rules of good design and craftsmanship. When your implementation is satisfactory, return to Step 1 by asking yourself, “What is the next test I need to write?”
Misconceptions
Although the TDD rules are simple, there are frequent misconceptions.
You write all the tests first before writing any code. No. As described above, write one small test, then one bit of code to make it pass. The tests and the code are implemented together in tight iterations.
TDD takes too long. If you haven’t been writing tests at all, then yes, more time is added to your development effort, but this is true if you write them first or last. What TDD teams realize once they start using it expertly is that overall development is faster because of reduced rework, refactoring, and bug fixing.
TDD is primarily a testing process. This is the point of this entire post, so let’s explore it in detail…
The Evolution of a Test Driven Developer
With any skill, it takes time to become an expert. Until you reach a certain level, the true benefits of that skill don’t reveal themselves. With TDD, there are 4 stages you must grow through.
Stage 1: Write Unit Tests
You have to start somewhere. Learn the unit testing framework(s) available for your development language. Leverage the shortcuts and plugins in your IDE that make unit testing as quick and easy as possible.
Stage 2: Write Good Unit Tests
What makes for good unit tests is the topic for another blog post, but in general, remember that unit tests test a unit of functionality, not a unit of code. Your tests are applying inputs and examining results for testing a small bit of functionality. How this functionality is realized in code is not their concern. Unit tests also run as independent units in isolation from one another. They can be run in any order and always get the same results. And they are fast. You’ll be running them frequently and so they must be as fast as possible.
Stage 3: Write Good Unit Tests First
You are now asking, ‘What’s the next test I need to write?’ instead of ‘What do I need to add to the code?’. When you’ve run out of ideas for tests, you are done with the code. You are always thinking first about the next bit of functionality that must be tested for, not about how it will be implemented. You will remain (happily and productively) at this stage for a long time.
Stage 4: The tests drive design decisions
It’s at this final step in your TDD evolution that you realize – it’s not about the tests. You now rely on TDD to drive good design. So finally we reach the point of this post: Test Driven Development is not a test tool, it’s a design tool.
Here are a few ways this works:
Rapid Feedback
The short feedback loop of the TDD discipline gives you a quick and constant evaluation of your decisions. If tests are getting hard to write, there’s something wrong with the code, back up and take a look at it. If the tests are red for too long while you try to figure out an implementation, your design needs attention. Maybe break things into smaller, more easily understood chunks.
Pressure Against Gold-plating/Future-proofing
It’s tempting to write or extend code to handle conditions that might (but probably won’t) be needed in the future, and then, if you are doing test-last, forgetting to write a test for the currently unneeded features. When your tests are driving your development, you’re always trying to tease out the design details of your current concern – what is being asked for right now. When those concerns are met, you’re done. If design changes are requested in the future, you start from a great place with well-written tests to guide you, and clean code to work with.
Pressure Against Complexity
One of the challenges of test-last when you are trying to achieve a certain target for lines covered by tests, is reaching every dark corner of deeply nested branches and every last private method that the code under test relies on. Hallmarks of tests written after the fact for legacy code include a huge amount of setup and an alarming amount of mock-object use, all done to help the tests reach deep down into the complex code.
When a test is driving development, every branch and every method is by default easily reachable. The tests dictate their creation, and there is no need to go back and figure out how to exercise a particular branch through the code. The well-written tests also serve as documentation about how the code is used. Also, because you are keeping the tests simple and concentrating on small increments of functionality, you are less likely to create complex cyclomatic craziness.
Promoting Cohesion over Coupling
I’ve recently reached a new stage in my testing philosophy. My practice has always been (in the Java world) to create a one-to-one relationship between the test class and the code under test. That is, for a Foo.java class, I would have a FooTest.java class containing all the tests for all the methods in Foo. I am now coming around to the idea that it is better to group tests by functionality. A test class that is focusing on a specific piece of functionality will naturally bring together, or bring about the creation of, objects that cohesively work together to perform that functionality. A test that is solely mapped to one object will naturally prefer that object to perform the entire function itself (even if it shouldn’t), leading to coupling and violations of the single responsibility principle.
Test Driven Development is a guide. It will help you make good design decisions but will not guarantee it. TDD provides positive pressure – pressure than can be ignored or misinterpreted. Writing good software is an art requiring experience and discipline to maintain the correct balance between all the competing, conflicting pressures.
Your Tests are Talking To You
When you focus on quick TDD cycles and are sensitive to times when you struggle to write a test or struggle to complete the code that makes your test pass, then you will immediately be aware of design problems and will course correct. Every test you write is telling you something about your design. Every test you write first is trying to influence your design to be better.
Better design, cleaner code, and a suite of tests. That’s a pretty good deal.
As Jama grew, our Engineering department began to change overnight. Teams were outputting consistent velocities, we had upped our abilities technically on how we measured areas such as performance and quality, we began 8-week planning cycles, we doubled our Engineering team and we had a strong roadmap to follow but for some reason it didn’t feel right. Identifying what that was exactly is where the challenges began. Why was it that we’ve come so far as an Engineering team but didn’t feel we had reached the Agile utopia we were seeking?
So What Were the Issues?
First and foremost, we called ourselves an Engineering Team, which meant that we were all in this together and operating as the collective whole, however when you really observed what was going on you saw static individual teams focused on only their projects. This became clear one day when I was in a meeting with Product and was told that the “top priority project” needed protection to get it out the door. I put it in quotes because at that time I thought there were multiple priority projects but no clear communication on which one was higher than the other. I then decided to test this with my team. In 1:1’s I asked a sampling of my team members, not on that project, what they thought was the top priority. The way in which I received the answers told me we had an issue. Half of the people I asked didn’t know and thought that what they were working on was a priority project, the other half answered correctly but with a very guarded response, as if they were going to be judged if they got it wrong. I then asked one key question to some of them, “Do you feel empowered to help that priority project if it needs it”. The answer was “no”. Not because they didn’t want to do the right thing, but because of the pressure they felt to get back to the project they left and get it on track again. The teams felt they were being measured on hitting certain sprint goals even if the business made the decision for them to do something different that affected it. We needed to get to a model that allowed teams the flexibility to protect our priorities without a feeling of guilt for those ones they left.
Teams believed that with more measurement came scrutiny. I started hearing comments that teams felt the business cared about velocity numbers more than the value being delivered from them and because of this they were more focused on how to make their burndowns look good, rather than taking chances and shifting during a sprint if that affected how the burndown presented itself. Hearing this cut to the core of my being because I wanted teams to take chances, innovate and change the sprint if it no longer made sense. Why continue to build in a direction when you know it’s the wrong path? Also, I know that there is always a story to tell behind burndowns regardless of how they look. I’ve had plenty of teams show me poor burndowns, but then the explanation as to why, is where I find the true value of a burndown, because the team iterated and learned something about the project that forced a shift. This is what we needed to reward the continual learning and change to reach the overarching goal as we drove to the end of the project, not every burndown’s success.
Another issue was around Engineering Priorities. Some examples include items such as infrastructure improvements, removal of tech debt, automating certain processes or refactoring problematic areas of the code that slow us down. We tried our best to recognize these issues within our team and tell teams to budget a certain amount of time out of their sprints to take this on. This “peanut butter” approach seemed like the right way to go but it was very apparent progress on those items was slow, or abandoned all together, because it was too hard to balance those priorities with the project work teams needed to get done. More often than not, a team would begin their sprints, find new work because of discovery and this additional work would overrun the time we allocated for the Engineering Priorities. Now in this model we make projects out of them and Engineering gets a budget to form a team and take these on every cycle. This brings more focus instead of the context switching within a sprint from project work to something that may not be related.
Cross-pollination of technologies, team processes, best practices and overall learning about the application was low. We had our Agile ceremonies and other meetings to talk about these in a cross-functional setting, but in the end teams would go back and continue to practice what they always had done. As a smaller engineering team I had newer employees tell me they didn’t even know their peers that well on other projects. This just seemed strange given we’re all within a small vicinity of each other. So why not give teams a chance to change up their team composition based off of people they wanted to learn from, processes they wanted to try or technologies they wanted to learn?
I Wasn’t Alone
After feeling this way for some time and hearing more and more feedback on how teams felt I began to wonder if it was just me or did others see it. I needed to see if Product felt this way, so I reached out to Derwyn Harris, one of our co-founders that’s in the Product team. Over lunch we discussed what we saw happening with the teams, challenged ourselves on what we thought we wanted out of the teams and talked about teams self-organizing around projects they could make the most impact on. The conversation was all laid on the foundation of empowerment and autonomy with the goal of increasing quality and decreasing lead time. In the end we walked away with a common goal and understanding of what we saw and what we wanted to change.
But wait, there’s more! Cristian Fuentes, one of our Engineering Managers, was also thinking of a way to further protect our team’s priorities to get work out the door, and Shane Patrick, our Sr. QA Manager, was building momentum around a centralized QA Team to help offload regression cycles from the other teams to protect project work. All of our ideas, thoughts and observations were coming to a head, which made it very clear that there was time for change at Jama.
“WE” Weren’t Alone
There was also validation in the industry of others trying models like this and having success. Two of the major players were Spotify and Valve, each doing something a little different but again built on the goal of empowering their engineers to make choices. Using these as case studies to gain confidence in our direction was key.
It Wasn’t Easy
Even though we had all this momentum from a leadership perspective we needed to make sure our teams understood what we were trying to do and why, gain alignment and test our theories. Derwyn created a Circle of Influence diagram to help us identify key stakeholders in the process and their overall support for it. A lot of conversations began with team members to get their feedback, and concerns, that helped us refine our approach. All of this happened over the course of a couple of months, because we knew we needed to make sure this was the right thing to do for Jama and our teams. A successful rollout required patience, listening and a build-up to buy-in.
Let’s Do This!
Now that you have some of the backstory let’s roll into how the teams formed around our priority projects. As of right now, we have an 8-week planning cycle that we plan for. Within those 8 weeks we run sprint lengths of 2 weeks. The goal of this planning cycle is not to load up all the sprints, because we all know that’s impossible and so much changes early on that rearranges your later sprints anyway, the goal is to provide enough clarity so the teams can have solid direction and know the release goals their striving for.
First we need to know our priorities. Product utilizes a process to help calculate this called Weighted Shortest Job First. This method combines a number of different criteria to help us evaluate a project’s importance. We use this method not only on roadmap features, but also on our Engineering priorities. It’s not an exact science but it gets us close.
Once those Product and Engineering priorities have been identified we provide one list for the teams to look at. This helps to bring clarity on the priorities and highlights ones we need to protected so we can get them out the door. Protection comes in different forms but simply put, if there is something a priority project needs, or if there is something that could potentially interrupt that project, teams around them are empowered to figure out creative ways to absorb those risks and keep them on track.
For us to get a good idea for the size of the projects they must also have an estimated duration in sprints and potential roles needed on the project to make it successful. For example, Project Unicorn could take up to 2 sprints to accomplish and it needs 3 Engineers and 2 QA Engineers on it to make it successful. This helps to recognize if a project is staffed appropriately and also exposes potential future scenarios for teams to roll off onto other projects.
Team members then get three votes after the Product Managers, or potentially a Technical Leader of the project, have had a chance to pitch their ideas to the teams. We are now doing this in a Science Fair format before the 8-week planning day. Once they have heard all they need to about the projects, they get to place their #1, #2 and #3 votes on the ones they are the most interested in, or feel they can provide the most value to the business. Once this is done the results tell us a variety of different stories:
Were teams able to staff the priority projects so they could be successful?
Did team members choose to move away from a project because they could influence higher priority ones?
Did we pick up Engineering projects that help with our efficiencies? (Examples: removing technical debt, shoring up infrastructure, increasing automation, etc.)
What were the projects that didn’t get chosen and why?
We then record the results and input them into a spreadsheet so we can further evaluate the choices. The goal is to not only make sure the top priorities are staffed, but we also want to do the best we can to place team members on projects that they have chosen (more to come on this). What’s fascinating is the number of scenarios and possible team combinations that you can form by reviewing the choices.
After we’ve done our best to predict where team members next tour of duty might be we open it up for collaboration and discussion. There may be more concerns that we didn’t think about, or other dependencies identified after self-selection has finished, but we try to get as close as we can so people know who they’ll be planning with when we roll into the 8 week planning day.
The Results
We are now in our fourth 8-week cycle utilizing this process. We continue to evaluate, change, iterate and find better ways of utilizing this model. Here are some observations after practicing this for a while now.
Priority projects are getting picked up and staffed with the talent they need to succeed. Through the first three 8-week cycles we staffed a lot of projects, and sometimes it was questionable if we were taking on too much. Because of this Product Management wanted to make sure we were not spreading ourselves thin, so we’ve started to implement WIP limits to meet our teams capacity and self-select on fewer projects. At the time of this post we will be starting our first cycle with these limits in place.
Cross-pollination is happening. Team members are learning more from each other. Some may be back end developers getting exposure to the front end, or just different technologies all together. Others may be on teams that are trying new processes such as Mob programming and pairing techniques that follow them to their next team. Agile techniques such as retrospectives, planning, estimating, etc. that are done differently from team to team are being shared. And knowledge of the application as a whole is increasing as team members with diverse experience of the application come together to solve common goals.
Teams are aware of the top priorities and have conversations around how to protect them. Because of this we feel more aligned as a team and are willing to accept project impacts if a team shifts off of theirs to assist in getting the higher ones out the door. The process increases fluidity in our teams to do this when we need to.
Still Working Out the Kinks
We’ve come a long way as a team with this model but are still evaluating and adjusting as we go. We’re far from perfect but I’d like that to be our goal. Here are some areas we continue to refine and recognize:
Not everyone receives a project they voted on. Because we don’t have infinite skill sets amongst team member’s certain projects may require someone with deep domain knowledge or technical skill because without them certain projects would cause risk. The good news is, this has only happened to a very small number of people, for the most part the majority of people get to be on a project that they picked. However, we try to remain sensitive to those that didn’t receive their pick so we can make sure we’re giving them their pick next time we run through our project selection. Long story short, no guarantees depending on business priorities. We have to keep the lights on, right?
We’re trying to “gauge happiness” by sending out surveys asking questions to our engineers. We did our first one in March and if you’re to look at the answers from a 10,000 foot view things look positive, and I do believe the results are exposing a level of happiness, however I’ve retrospected over the survey with my team and the next round we’re going to be more explicit on the questions we were asking, as they were too subjective and open for interpretation. Regardless, we’re trying to measure how people feel in this new model and are paying attention.
We’re getting better at trying to define Engineering Priorities and the value behind them. We want to do a lot of “good stuff” but also realize we only have so many people to do it with. We have to be careful with what we choose, so we can get the biggest bang for our buck.
Metrics, metrics, metrics. We have abandoned some of our metrics because we weren’t sure of the value they were giving us and instead felt more administrative. But we still have to prove the process is working so we’re continually reviewing how to best measure and represent epic lead time and quality measures. Also we still use agile Velocity and Burndowns to see how teams are doing, but those are only snapshots in time so we try to make sure we’re keeping our eye on the prize which is overall health of our projects as they reach completion.
We continue to evaluate work in progress project limits to find the magic number the teams can handle. This model is built to have fluidity in it so we must make sure we protect that as much as possible and aren’t spreading ourselves too thin.
How Do I Get Started?
Maybe this process sounds good to you and you want to try it out, but how do you know if it’s right for your company? I’m sorry to say that I can’t answer that for you. Only you know your culture, problems you have and how this could potentially help you solve them. But what I can do is provide you with some potential thought points for you to think about as you determine if this will work or not.
What problems would you be solving by trying something like this? Make sure you have a clear understanding of these by sharing them with your peers and leadership so you can validate them. This is the start of the buy-in process. If others around you identify the same issues you already have momentum.
Have clear goals that you’re trying to achieve. For example, our goal was to empower our employees to make choices which in theory should decrease epic lead time and increase quality. We’re still measuring these as I pointed out earlier but we are seeing results.
How flexible is your company? Do you already work in a company that’s Agile and accepting of change or do you work in a regulated environment that is change adverse because it may introduce too much risk and uncertainty? If it’s the second, rollout will be slow and painful so find ways within it to make very small, incremental changes that in the end align to your goals.
I mentioned this briefly in the “what problems are you trying to solve” paragraph, but felt it is important enough to give it focus. Make sure you have leadership support. It was key at Jama that we had this from our Engineering and Product VP’s as well as the Director of Product. Without their willingness to let us turn the world upside down and try something different, the idea would have died after initial conversations. They’ve been instrumental in giving us the freedom to implement this while also being very involved in its success.
And last but certainty not least, do you have TRUST? Trust from the business and trust from the people around you? If we didn’t have this, I don’t know where we’d be today. For example, imagine going to your Product Management organization and saying “Hey, those static teams that you’ve relied upon for the last few years, we’re going to blow them up and allow them to do what they want based off of priority”. Talk about a scary message to accept, but we were trusted by those around us that we have the talent and understanding to do the right thing.
So what are you waiting for? Don’t get bogged down and accept your problems that in the end lead to low morale and poor quality products. You’ve hired smart people to do great things. Empower them to do so regardless if it’s something like this model or another idea you have. Be brave, give it a try, iterate on it and change your organization for the better!
The Results