As Jama grew, our Engineering department began to change overnight. Teams were outputting consistent velocities, we had upped our abilities technically on how we measured areas such as performance and quality, we began 8-week planning cycles, we doubled our Engineering team and we had a strong roadmap to follow but for some reason it didn’t feel right. Identifying what that was exactly is where the challenges began. Why was it that we’ve come so far as an Engineering team but didn’t feel we had reached the Agile utopia we were seeking?
So What Were the Issues?
First and foremost, we called ourselves an Engineering Team, which meant that we were all in this together and operating as the collective whole, however when you really observed what was going on you saw static individual teams focused on only their projects. This became clear one day when I was in a meeting with Product and was told that the “top priority project” needed protection to get it out the door. I put it in quotes because at that time I thought there were multiple priority projects but no clear communication on which one was higher than the other. I then decided to test this with my team. In 1:1’s I asked a sampling of my team members, not on that project, what they thought was the top priority. The way in which I received the answers told me we had an issue. Half of the people I asked didn’t know and thought that what they were working on was a priority project, the other half answered correctly but with a very guarded response, as if they were going to be judged if they got it wrong. I then asked one key question to some of them, “Do you feel empowered to help that priority project if it needs it”. The answer was “no”. Not because they didn’t want to do the right thing, but because of the pressure they felt to get back to the project they left and get it on track again. The teams felt they were being measured on hitting certain sprint goals even if the business made the decision for them to do something different that affected it. We needed to get to a model that allowed teams the flexibility to protect our priorities without a feeling of guilt for those ones they left.
Teams believed that with more measurement came scrutiny. I started hearing comments that teams felt the business cared about velocity numbers more than the value being delivered from them and because of this they were more focused on how to make their burndowns look good, rather than taking chances and shifting during a sprint if that affected how the burndown presented itself. Hearing this cut to the core of my being because I wanted teams to take chances, innovate and change the sprint if it no longer made sense. Why continue to build in a direction when you know it’s the wrong path? Also, I know that there is always a story to tell behind burndowns regardless of how they look. I’ve had plenty of teams show me poor burndowns, but then the explanation as to why, is where I find the true value of a burndown, because the team iterated and learned something about the project that forced a shift. This is what we needed to reward the continual learning and change to reach the overarching goal as we drove to the end of the project, not every burndown’s success.
Another issue was around Engineering Priorities. Some examples include items such as infrastructure improvements, removal of tech debt, automating certain processes or refactoring problematic areas of the code that slow us down. We tried our best to recognize these issues within our team and tell teams to budget a certain amount of time out of their sprints to take this on. This “peanut butter” approach seemed like the right way to go but it was very apparent progress on those items was slow, or abandoned all together, because it was too hard to balance those priorities with the project work teams needed to get done. More often than not, a team would begin their sprints, find new work because of discovery and this additional work would overrun the time we allocated for the Engineering Priorities. Now in this model we make projects out of them and Engineering gets a budget to form a team and take these on every cycle. This brings more focus instead of the context switching within a sprint from project work to something that may not be related.
Cross-pollination of technologies, team processes, best practices and overall learning about the application was low. We had our Agile ceremonies and other meetings to talk about these in a cross-functional setting, but in the end teams would go back and continue to practice what they always had done. As a smaller engineering team I had newer employees tell me they didn’t even know their peers that well on other projects. This just seemed strange given we’re all within a small vicinity of each other. So why not give teams a chance to change up their team composition based off of people they wanted to learn from, processes they wanted to try or technologies they wanted to learn?
I Wasn’t Alone
After feeling this way for some time and hearing more and more feedback on how teams felt I began to wonder if it was just me or did others see it. I needed to see if Product felt this way, so I reached out to Derwyn Harris, one of our co-founders that’s in the Product team. Over lunch we discussed what we saw happening with the teams, challenged ourselves on what we thought we wanted out of the teams and talked about teams self-organizing around projects they could make the most impact on. The conversation was all laid on the foundation of empowerment and autonomy with the goal of increasing quality and decreasing lead time. In the end we walked away with a common goal and understanding of what we saw and what we wanted to change.
But wait, there’s more! Cristian Fuentes, one of our Engineering Managers, was also thinking of a way to further protect our team’s priorities to get work out the door, and Shane Patrick, our Sr. QA Manager, was building momentum around a centralized QA Team to help offload regression cycles from the other teams to protect project work. All of our ideas, thoughts and observations were coming to a head, which made it very clear that there was time for change at Jama.
“WE” Weren’t Alone
There was also validation in the industry of others trying models like this and having success. Two of the major players were Spotify and Valve, each doing something a little different but again built on the goal of empowering their engineers to make choices. Using these as case studies to gain confidence in our direction was key.
It Wasn’t Easy
Even though we had all this momentum from a leadership perspective we needed to make sure our teams understood what we were trying to do and why, gain alignment and test our theories. Derwyn created a Circle of Influence diagram to help us identify key stakeholders in the process and their overall support for it. A lot of conversations began with team members to get their feedback, and concerns, that helped us refine our approach. All of this happened over the course of a couple of months, because we knew we needed to make sure this was the right thing to do for Jama and our teams. A successful rollout required patience, listening and a build-up to buy-in.
Let’s Do This!
Now that you have some of the backstory let’s roll into how the teams formed around our priority projects. As of right now, we have an 8-week planning cycle that we plan for. Within those 8 weeks we run sprint lengths of 2 weeks. The goal of this planning cycle is not to load up all the sprints, because we all know that’s impossible and so much changes early on that rearranges your later sprints anyway, the goal is to provide enough clarity so the teams can have solid direction and know the release goals their striving for.
First we need to know our priorities. Product utilizes a process to help calculate this called Weighted Shortest Job First. This method combines a number of different criteria to help us evaluate a project’s importance. We use this method not only on roadmap features, but also on our Engineering priorities. It’s not an exact science but it gets us close.
Once those Product and Engineering priorities have been identified we provide one list for the teams to look at. This helps to bring clarity on the priorities and highlights ones we need to protected so we can get them out the door. Protection comes in different forms but simply put, if there is something a priority project needs, or if there is something that could potentially interrupt that project, teams around them are empowered to figure out creative ways to absorb those risks and keep them on track.
For us to get a good idea for the size of the projects they must also have an estimated duration in sprints and potential roles needed on the project to make it successful. For example, Project Unicorn could take up to 2 sprints to accomplish and it needs 3 Engineers and 2 QA Engineers on it to make it successful. This helps to recognize if a project is staffed appropriately and also exposes potential future scenarios for teams to roll off onto other projects.
Team members then get three votes after the Product Managers, or potentially a Technical Leader of the project, have had a chance to pitch their ideas to the teams. We are now doing this in a Science Fair format before the 8-week planning day. Once they have heard all they need to about the projects, they get to place their #1, #2 and #3 votes on the ones they are the most interested in, or feel they can provide the most value to the business. Once this is done the results tell us a variety of different stories:
- Were teams able to staff the priority projects so they could be successful?
- Did team members choose to move away from a project because they could influence higher priority ones?
- Did we pick up Engineering projects that help with our efficiencies? (Examples: removing technical debt, shoring up infrastructure, increasing automation, etc.)
- What were the projects that didn’t get chosen and why?
We then record the results and input them into a spreadsheet so we can further evaluate the choices. The goal is to not only make sure the top priorities are staffed, but we also want to do the best we can to place team members on projects that they have chosen (more to come on this). What’s fascinating is the number of scenarios and possible team combinations that you can form by reviewing the choices.
After we’ve done our best to predict where team members next tour of duty might be we open it up for collaboration and discussion. There may be more concerns that we didn’t think about, or other dependencies identified after self-selection has finished, but we try to get as close as we can so people know who they’ll be planning with when we roll into the 8 week planning day.
The Results
The ResultsWe are now in our fourth 8-week cycle utilizing this process. We continue to evaluate, change, iterate and find better ways of utilizing this model. Here are some observations after practicing this for a while now.
Priority projects are getting picked up and staffed with the talent they need to succeed. Through the first three 8-week cycles we staffed a lot of projects, and sometimes it was questionable if we were taking on too much. Because of this Product Management wanted to make sure we were not spreading ourselves thin, so we’ve started to implement WIP limits to meet our teams capacity and self-select on fewer projects. At the time of this post we will be starting our first cycle with these limits in place.
Cross-pollination is happening. Team members are learning more from each other. Some may be back end developers getting exposure to the front end, or just different technologies all together. Others may be on teams that are trying new processes such as Mob programming and pairing techniques that follow them to their next team. Agile techniques such as retrospectives, planning, estimating, etc. that are done differently from team to team are being shared. And knowledge of the application as a whole is increasing as team members with diverse experience of the application come together to solve common goals.
Teams are aware of the top priorities and have conversations around how to protect them. Because of this we feel more aligned as a team and are willing to accept project impacts if a team shifts off of theirs to assist in getting the higher ones out the door. The process increases fluidity in our teams to do this when we need to.
Still Working Out the Kinks
We’ve come a long way as a team with this model but are still evaluating and adjusting as we go. We’re far from perfect but I’d like that to be our goal. Here are some areas we continue to refine and recognize:
Not everyone receives a project they voted on. Because we don’t have infinite skill sets amongst team member’s certain projects may require someone with deep domain knowledge or technical skill because without them certain projects would cause risk. The good news is, this has only happened to a very small number of people, for the most part the majority of people get to be on a project that they picked. However, we try to remain sensitive to those that didn’t receive their pick so we can make sure we’re giving them their pick next time we run through our project selection. Long story short, no guarantees depending on business priorities. We have to keep the lights on, right?
We’re trying to “gauge happiness” by sending out surveys asking questions to our engineers. We did our first one in March and if you’re to look at the answers from a 10,000 foot view things look positive, and I do believe the results are exposing a level of happiness, however I’ve retrospected over the survey with my team and the next round we’re going to be more explicit on the questions we were asking, as they were too subjective and open for interpretation. Regardless, we’re trying to measure how people feel in this new model and are paying attention.
We’re getting better at trying to define Engineering Priorities and the value behind them. We want to do a lot of “good stuff” but also realize we only have so many people to do it with. We have to be careful with what we choose, so we can get the biggest bang for our buck.
Metrics, metrics, metrics. We have abandoned some of our metrics because we weren’t sure of the value they were giving us and instead felt more administrative. But we still have to prove the process is working so we’re continually reviewing how to best measure and represent epic lead time and quality measures. Also we still use agile Velocity and Burndowns to see how teams are doing, but those are only snapshots in time so we try to make sure we’re keeping our eye on the prize which is overall health of our projects as they reach completion.
We continue to evaluate work in progress project limits to find the magic number the teams can handle. This model is built to have fluidity in it so we must make sure we protect that as much as possible and aren’t spreading ourselves too thin.
How Do I Get Started?
Maybe this process sounds good to you and you want to try it out, but how do you know if it’s right for your company? I’m sorry to say that I can’t answer that for you. Only you know your culture, problems you have and how this could potentially help you solve them. But what I can do is provide you with some potential thought points for you to think about as you determine if this will work or not.
What problems would you be solving by trying something like this? Make sure you have a clear understanding of these by sharing them with your peers and leadership so you can validate them. This is the start of the buy-in process. If others around you identify the same issues you already have momentum.
Have clear goals that you’re trying to achieve. For example, our goal was to empower our employees to make choices which in theory should decrease epic lead time and increase quality. We’re still measuring these as I pointed out earlier but we are seeing results.
How flexible is your company? Do you already work in a company that’s Agile and accepting of change or do you work in a regulated environment that is change adverse because it may introduce too much risk and uncertainty? If it’s the second, rollout will be slow and painful so find ways within it to make very small, incremental changes that in the end align to your goals.
I mentioned this briefly in the “what problems are you trying to solve” paragraph, but felt it is important enough to give it focus. Make sure you have leadership support. It was key at Jama that we had this from our Engineering and Product VP’s as well as the Director of Product. Without their willingness to let us turn the world upside down and try something different, the idea would have died after initial conversations. They’ve been instrumental in giving us the freedom to implement this while also being very involved in its success.
And last but certainty not least, do you have TRUST? Trust from the business and trust from the people around you? If we didn’t have this, I don’t know where we’d be today. For example, imagine going to your Product Management organization and saying “Hey, those static teams that you’ve relied upon for the last few years, we’re going to blow them up and allow them to do what they want based off of priority”. Talk about a scary message to accept, but we were trusted by those around us that we have the talent and understanding to do the right thing.
So what are you waiting for? Don’t get bogged down and accept your problems that in the end lead to low morale and poor quality products. You’ve hired smart people to do great things. Empower them to do so regardless if it’s something like this model or another idea you have. Be brave, give it a try, iterate on it and change your organization for the better!