Why Software Teams Fail To Improve Delivery
I've observed teams in many companies invest in deployment tools and yet fail to improve delivery. Why does this happen and what can be done?
This post was originally posted in August of 2022 and has been edited and republished to improve reading and coherence.
My recurring observation across organisations is that teams invest in deployment tools but struggle to improve their pace of delivery as desired. Why does this happen? Why do teams strive for something and consistently fall short across many organisations?
In my experience, here are some interrelated reasons why this happens:
Mismatch of Aspirations and Expectations
Misalignment of the Goal
Mismatch of Aspirations and Expectations
When I coach a team plagued with quality issues, it’s common to find frustrated team members. They often experience added frustration because each team member has their own aspirations for different aspects of the development process. Their aspirations may be in different timescales — some shorter term, others longer.
Mismatches of aspirations can contribute to friction. Without a mechanism for prioritising, each planning session becomes a calamity of individual ideas. What can result is a stalemate where compromise is not the focus but instead becomes a Frankenstein’s monster of ill-fitting software requirements stitched together. Over time the team can become demoralised by the process, frustrated with each other, and achieve less impact and a slow rate of improvement.
Identifying and articulating opportunities and coordinating each team member's readiness to engage is a critical factor in the success of any collaboration for improvement. This is an idea I cover in more detail in this post:
Also important is a joint agreement across the team on a single focus for the work, which brings me to the following common issues that conspire to lead teams astray - misalignment of the goal.
Misalignment of the Goal
The example I teased at the outset of this article is that teams build deployment tools that aren’t fit for purpose—i.e., they fail to help make software deployment faster and more reliable.
This happens because the ‘destination’ or goal becomes building the tooling rather than the effect it’s supposed to provide. This is often the result of a pet project from one of the team members or a collective desire to experiment with new technology.
Why the goal shouldn’t be to deliver the tool
The new version of the deployment tool may be technically more sophisticated but fails to fix the underlying issues, preventing the team from improving its delivery rate. When the goal is to build the tool, the project can’t fail even if it doesn’t address the problems, slowing their delivery. A tool will be delivered whether it solves the driving need or not.
Being able to deploy more frequently never really eventuates, or it does, but the ability to deliver value or deliver reliably hasn’t improved. Either way, we fall short of what we set out to achieve.
The team always has more tweaks to apply to the toolchain, which can occupy them, but if the issue is somewhere else in their process outside their tooling, then adjustments to the technology may not address the root issue. They stay very busy improving the tool, but no positive impact can be observed for anyone outside of the team.
What happens if the goal is to deliver the tool rather than the outcome?
When the goal is to deliver the tool, it creates blind spots for the team for common issues, contributing to slow or infrequent delivery. One example of a blindspot: The problem may be how the team frames and vertically slices stories to ensure work is small and self-contained enough to deploy confidently.
A new tool won’t provide much benefit if the team doesn’t change how it works upstream of the tool in that case. If that was the cause for slow iteration before, it won’t change if only deployment is optimised.
Common gaps when framing a goal for this type of improvement work are the absence of:
Holistic analysis of the issues,
Clear framing of the problem,
A well-defined goal and how it will translate to success.
Deployment tools seem like a natural solution to delivery frequency, and many teams go straight to this as the solution to delivery challenges. It’s a comfortable domain for them and avoids less comfortable discussions, such as finding agreement on renegotiating how work is done. The tooling may be part of the answer, but improving a team’s ability to deploy value more frequently must consider the people and process elements.
The problem is that the team’s focus is not guided by a meaningful improvement for them or their customers. Without a measure of progress towards that improvement or a way to assess whether the rate is adequate to realise a significantly improved situation in the future. To solve the right problem at the right time.
What might be a better goal for improving software delivery?
Teams may seek to improve their deployment frequency and reliability to increase their delivery pace. It helps to be clear why this investment will help. The imperatives for this investment may differ for a given team.
One reason could be a desire for increased responsiveness. Reducing the lead time to ship an improvement or fix may be the motivation.
Another reason could be learning, which can occur at different levels. Being able to deploy changes more frequently may help us know whether or not they are meeting a customer's needs because they can get into the user’s hands more quickly. It might also be learning relating to the implementation, such as assumptions about how the software might work with other dependencies discovered sooner because they were brought together sooner.
These can be implicit benefits of improving delivery, but by making them explicit, you can start to identify ways to test whether you are progressing towards achieving this.
This may lead you to value reducing the time until software can be integrated and socialised with other software. It may also lead you to value the lead time from when an idea is conceived to when a user can experience it and provide feedback.
These may be influenced as much by upstream factors, such as the amount of work in progress at once, the reliability of the deployment mechanism, the time associated with preparing for deployment, and how well the work is broken down into parts that can represent meaningful progress.
Other factors may be critical to helping this happen, such as the reliability of the deployment mechanism, the time associated with preparing for deployment, and all the other steps that contribute to the elapsed time.
Each organisation's gaps and needs differ, so analysis and agreement on what success will be for your team in your organisation’s context will be critical.
How to improve with a higher chance of success
The journey of a team I coached through this
On one of the occasions I observed a team facing these struggles, I was in a leadership position, having recently joined the company. There was already an inflight initiative to try and improve delivery frequency from monthly to more frequent so more experiments could be run with users.
As I’ve described in this post, the team faced the issues I’ve described above. There was a lot of focus on building a better deployment tool. It was better in every way than before, but it still didn’t allow the team to deploy more frequently than monthly. They would have more ideas for adding features to the tool, yet nothing improved for months.
I started coaching the team as I moved from an observational role into more active leadership (I was nearing the 90-day mark of being at the new company). I explored with them what they were trying to achieve, and gaps where focus on improving tooling would fall short of helping them. This eventually led to some ‘aha!’ moments for the team.
What the Team did to get Back on Track
Once they realised the error, the team returned to the drawing board and mapped out a typical deployment package, usually user stories and bug fixes, where the time was spent in all aspects of product creation, not just coding.
Think of this as research or data collection. Over time, they also took each of the steps in the post—using the data they collected to frame the problem better, identify better goals, and identify better progress indicators than task completion. This led to a crude version of three of the four DORA metrics (before the first State of DevOps / DORA and Accelerate was published) working from first principles.
It started with simply counting deployments. A script only a few lines long was inserted in the build to update a counter, which they used to see if anything they were changing helped them deploy more often. They took this to a level that, in hindsight, we all would agree was ridiculous. They deployed over a hundred times a day.
They brought this feat to me, asking if I thought it was good and if that was what I wanted because they weren’t so sure. I asked them what they thought, and while proud of deploying so many times, proving they could do what seemed unachievable before, it also felt not quite right.
This was because they knew some of it was from gaming the system - they could trigger a deployment from commits of any size. They also broke the build a lot more and deployed breaking changes to the service. Previous advice now made sense :
about focusing on the goal, not the measures
using measurement as evidence rather than as the goal,
and finally using measures in balance rather than juicing one at the expense of others.
I suggested they consider monitoring the lead time of work moving through the system (DORA defines Change Lead Time; we were pulling lead times for work from our boards to try to identify other factors influencing frequency) and Mean Time To Recovery (MTTR) of services they were deploying to (DORA has since moved away from this measure for a more specific deployment recovery one, which makes sense, I suppose) and put some friction in such as needing to huddle any time they broke the build.
From this, we can conclude the following aspects are essential:
Ensure there is agreement that this is a problem worth solving and that the next problem is the best one.
Commence when the team members can collaborate on this improvement.
Analyse the situation holistically, considering people, processes, and technology elements. Conduct research and collect data.
Understand and frame the problem and the issues at play.
Set a clear goal describing the new reality being aimed for.
Identify measures of progress that are symptomatic of progress towards the goal.
Track progress towards the goal continuously using these measures.
The effort for the above should be commensurate with the ‘size of the prize’ and iterative.
Have you experienced this with a team you have worked with or participated in? Do you think identifying critical problems and imagining a future with those problems addressed helps better frame the work to improve the situation? Please share your experiences in the comments.