The sad reality is that though well-intentioned, most agile transformations fail. By fail I mean they are canceled, reversed or fall short of delivering the desired results.
To be clear, agile transformation success is different than agile project success. Success rates for agile tend to be significantly higher than those for traditional approaches, and there are plenty of data points to support that. Unfortunately, a lot less data is available on agile transformation success.
I do find that the majority of attendees to my training classes belong to organizations that have attempted some type of transformation. Most of them report limited success, and in many cases, a complete reversal of the agile initiatives.
Agile transformations are major programs of change and represent risk. Read on to learn more about the top 10 reasons that agile transformations fail. And download this handy diagram outlining those 10 reasons:
#1 – Agile Transformations Fail Because They Take Too Long
The primary reason that I believe agile transformations fail is that they take a long time. As humans, our expectations for things have dramatically changed over the last five to 10 years.
Today we expect things immediately. Amazon will now deliver almost anything, including groceries, to my home in two hours or less. We have become conditioned to near-instant satisfaction, and we are losing the ability to defer gratification.
In organizations, the challenge of wanting instant results is exacerbated by the short-term tenure of executives and a somewhat myopic focus on short-term results. The average tenure for a CIO is about four years.
That isn’t a lot of time to make a deep and lasting change like a transformation to agile. Executives find that they need to hit the ground running and deliver results fast. Most don’t have the patience or the time to take on an agile transformation. So they push for the quick fix, the one-time big bang transformation. And those big bang agile transformations are quite likely to fail.
A true agile transformation in an organization is anything but fast. Change takes time, for all organizations. Most people believe it takes three to five years. That is beyond the horizon for most executives.
Those transformations that get started will likely be halted mid-stream once the focus shifts to a different shiny object. Or the agile champion leaves the organization.
#2 – Agile Transformation is Limited to Process Change Only
Many people see agile as simply an alternative way of running projects. They view it as another tool in their toolkit, alongside traditional project management practices like those described in A Guide to the Project Management Body of Knowledge (PMBOK® Guide).
This view of agile as a process change misses the most critical aspect of agile, namely that it is fundamentally a culture and mindset change. Treating agile transformation as a process change alone is short-sighted and will likely cause your agile transformation to fail.
Without a change in mindset and culture, agile will become an empty set of rituals that fail to deliver the promised benefits. Pretty soon, the organization will snap back to its tried-and-true methods, like a rubber band that has been stretched.
Consider the example of FinTech giant ING. It invested heavily in agile development processes in 2011 and then further into DevOps.
Years later, it recognized that it didn’t get the benefits because it hasn’t fundamentally changed the organization structure and relationship with the internal business customers. It had simply bolted agile onto the existing organization.
#3 – Lack of Executive Leadership
Another common reason for agile transformation failure is a lack of leadership support. And not just support, actual leadership from the top. This frequently happens when agile was initiated as a bottom-up effort at the department, project, or team level.
Agile may have worked great initially for that department or business unit, and so the leaders agree to “do agile” and then sit by and wait it out. That is, they tolerate agile ways of working.
The bottom-up approach reminds me of a bunch of kids building a tent fort in the living room. They have had great fun and they believe that their parents are going to let them leave it up forever. They are somewhat shocked when mom and dad want to put the living room back the way it is “supposed to be.”
Like parents, managers and leaders may tolerate things being done differently, but only for so long. As soon as there is a bump in the road (and according to the Satir Change Process Model, there will be problems, see diagram below) they are going to change the priorities of the current sprint, pull team members to work on special projects, demand an MS Project Schedule or take some other action to undermine the agile transformation.
#4 You Don’t Get It – Lack of Agile Understanding
The root cause of the previous two items could be that there is a lack of understanding of exactly what agile means.
Despite the fact that agile software development has been around for nearly 20 years (and Scrum and other predecessors even longer), some people have not taken the time to really understand it.
I see this all the time. Invariably when I visit a new client, they tell me that all the leaders are onboard and they understand agile.
The problem is, they really don’t get it! Some do, of course, but most have only read about it or experienced a version of “agile” that wasn’t really agile at all.
Where possible, I strive to encourage them to learn about what agile actually means before telling their teams that they need to adopt it.
I often have to convince them that an investment in agile training to establish a common set of terms and understanding would be a good idea.
#5 – Copying Others Instead of Experimenting and Thinking
One of my personal pet peeves is when people mindlessly parrot the ideas of others without thinking too deeply about them. This is likely to cause an agile transformation to fail.
If you call your development teams “squads”, I am talking about you.
Transformations don’t begin in a vacuum. In most cases, someone from outside the organization brings in a specific set of practices that worked in their previous company.
Or they are a coach and they tell the organization what is best for them. Or someone in the organization reads an article about transformation at Capital One or ING, or watches a Spotify video; they use that as a blueprint for their transformation.
In its simplest form, agile is about thinking, running small experiments, and improving continuously. It is about building the muscle of learning. It is not about copying what another organization did and implementing it as a cargo cult.
[See my related article: There is No Spotify Model] This is one reason I insist my clients develop their own internal coaches and champions to help lead their transformation. It is exactly those people within the organization that can help drive change from the inside out. External coaches like myself can help to kick-start the change, but the people closest to the work need to own it and that includes making informed decisions about the transformation.
#6 – Lack of Compelling Why
Simon Sinek is probably best known for his book Start with Why. And with good reason. Before starting any change initiative, you have to know WHY you are doing it.
It isn’t enough to point at others and say that you want to transform because others are using Agile and Scrum. And a why isn’t “to Adopt Scrum”.
Organizations need some sort of burning platform if they are really going to engage and attract others to the change initiative. They need to see both how detrimental it will be to not change and have a vision for an attractive future state. Otherwise, people will be consumed with the urgencies of the day and not be compelled to change.
#7 – No Guiding Coalition
Change expert and Harvard Business Professor John Kotter says that it is imperative to have a group of committed people leading it. It is not enough to have one executive leader. And you can forget it if you only have a team doing bottom-up, team-level agile.
The coalition needs to have representation from across the organization, in different roles, and with different perspectives and skill sets.
#8 – Agile Everywhere
Some organizations go overboard with agile and there are zealots who believe it belongs everywhere.
More likely there are areas where it fits well – like product development or software – and others where it doesn’t work so well. In their book, Doing Agile Right, Daryl Rigby, and his co-authors call this Agile Everywhere.
Rigby goes so far as to call Agile Everywhere one of the three toxic mistakes that organizations make. They argue that agile ways of working don’t fit every situation and in some areas like finance or compliance, a more bureaucratic approach is actually OK.
The challenge in short is not to replace bureaucracy with agile everywhere but to find the right balance between the two.
Daryl Rigby et al, Doing Agile Right.
#9 – Agile in Name Only
This is so prevalent I should have put it right at the top! ‘Agile In Name Only is the term for those who simply rename the things that they are doing with agile names. Status meetings become standup meetings, phases become sprints…you get the picture.
This is dangerous for a couple of reasons. First, it reinforces the status quo. When you simply rebrand what you are doing today with Scrum terms, you keep on keeping on as they say.
Second, you eliminate the opportunity to learn what true agile means. By calling your existing project managers “Scrum Masters”, you remove the need for anyone to learn how to be an effective Scrum Master.
Agile and Lean Expert Craig Larman calls this out in his laws of organizational behavior. His first and second rules are shown below:
1. Organizations are implicitly optimized to avoid changing the status quo middle- and first-level manager and “specialist” positions & power structures.
2. As a corollary to (1), any change initiative will be reduced to redefining or overloading the new terminology to mean basically the same as status quo.
Craig Larman, Larman’s Laws of Organizational Behavior
#10 – Waterfalling Your Agile Transformation
If you have a Gantt chart for your transformation, you’ve already failed.
Some organizations treat agile transformation as a waterfall project and they drive toward specific outcomes and timelines. That won’t work.
Transformation is an ongoing process of experimentation, learning, and improvement. It is not a fixed scope, fixed timeline project to hit a specific target.
Transformation expert Michael Sahota described this well in his Certified Agile Leadership training that I took a few years ago. Sahota provided the following image of what transformation looks like:
What Is Your Experience with Agile Transformation?
I am very interested in hearing from you about your experience with agile transformation. If you’ve been part of a transformation before or are currently involved in one, what have you learned? Are my observations about what causes agile transformations to fail on target?
An earlier version of this article was published on ProjectManagement.com.
Like this post? Check out our other blogs that have similar free downloads:
Anthony, the last paragraph of #3 really resonated with me and my current situation.
Our department has been making consistent strides in our Agile journey for a while now, but recently Senior Management requested an initiative be completed in a very tight time frame. Despite feedback (facilitated by SM’s) from Scrum teams on how to best meet the goal (i.e. – eliminate dependencies, improve focus), management decided to fall back to a PM-based model, and increase and change team rosters to better meet their “plan”.
I’ve tried to offer my observations and suggestions, but they seem intent on sticking with their traditional PM approach. Morale has dipped, meetings and status are now sapping our already limited capacity, and management is reporting our status as “yellow”, further exacerbating the situation (“Why are you guys falling behind?”).
Tim, thanks for your comments. It sounds like your situation is both frustrating and all too common. Change is difficult and old habits die hard. The desire to impose top down control is too attractive.
I like how you pointed out the vicious downward spiral that can occur when managers are overly involved. Their attempts to ‘help’ the teams to hit aggressive dates will not only backfire, they create more work for everyone and put them on the defensive. Then there are more meetings to explain the situation, more time spent defending and justifying and less overall productivity.
Thanks for your comments Tim!
Very good summary, I made roughly the same observations. I question myself still: once companies fall back into their old routines – can they survive in increasingly complex environments with customer demands changing faster than ever? Maybe some of those high-profile dinosaurs need to become extinct, and then the sense of urgency can be created?
Hi Daniel, thanks for your comment. I don’t know how companies will be able to use traditional approaches to stay competitive when customer demands and technology disruptions are upheaving entire industries. I agree with you that it is likely that many will become extinct or find their backs against the wall before they embrace new ways of thinking or working.
There are many studies out there showing that the average life of companies in the S&P 500 is rapidly shrinking. That may provide a bellwether of the state of survival. Here is one example
Thanks again for commenting!
Hi Antony, Great article from you.
How do you avoid reversal of transformation?
How does transformations will be successful in spite of shorter tenure of CIOs/key sponsors?
There are different percentages of success and failure ,but do we really know on what basis whether the transformation is considered a failure or success..?
Is there any thing SCRUM or agile organizations can do to help other organizations with their agile transformation?
I have thought through this and one of the solution which I have come up to solve this is ,with a new role called AGILE AUDITOR
My research paper
Hi Sreenivasa, thanks for your comment. I don’t know if a new role is the solution or not but glad you weighed in and I will read your research paper.
Would the role of Agile Auditor also have a certification component? Because we need more agile certifications!
Thank you for this article. I was just doing some research to prepare a training session for our leadership team on agile. We have started the transition in the Tech Team in May and with only myself, the former Project Manager retrained as ScrumMaster, to drive it, backed up by the Tech Team Lead. We have adopted Scrum to get us going and help change the mindset and culture, which I see as the most difficult part. I can see us moving away from Scrum in the future and let each Team decide how to work as long as they keep adhering to the Agile principles.
I have watched the Spotify videos, it’s true, but since the start I’ve been advocating coming up with our own model rather than copying it. What I see, however, is a lack of enthusiasm from people. I think it’s so much easier to adopt agile in a startup as you don’t have to deal with the “tiredness” of the tech team members who have been squeezed and overworked for so long and take everything new with skepticism.
It would be great to hear how others have overcome the challenges you list.
Agree with the others. Very nice summary of the big issues. Kudos.
The BIG problem is – All companies have been “sold” the idea Agile is faster delivery. A transformation requires an amount of work to be completed. If you have 100 days of work in a waterfall project – you still have the same 100 days of work on an Agile project. This is including only delivering an MVP.
Agile has been “sold” as a faster delivery methodology, therefore the executive expects a faster delivery.
The main point is any delivery methodology has its positive and negative points. Agile is NOT the “silver bullet”
The Agile principles below are ignored:
I have capitalized the main points below.
1. Early and Continuous Delivery of VALUABLE Software.
2. WORKING software is the primary measure of progress.
3. Continuous attention to TECHNICAL EXCELLENCE and GOOD DESIGN enhances agility
4. SIMPLICITY – the art of maximizing the amount of work not done—is essential
If ALL these four points (above) are not followed, your project / programme will fail. In most cases NONE of the points above are even considered.
This principle below has a problem.
5. “The best architectures, requirements, and designs emerge from self-organizing teams.”
In a programme, if you have 5 “self-organizing teams” and they are all working differently, (they all think their way is “best”) – how can you expect to have success.
The problem that nobody wants to discuss about Agile is the skill quality of the team members. In most cases they can write code (a lot of times, not very well either) but cannot design or test their code properly, you need all three to be effective(Design – a system, Build – the system, Test – the system fully).
Now – A lot of people are going to say this is not true – fine – work on a programme with the big consulting firms, look at the quality of software delivered, look at when the defects are found (normally in the production system stage). look at how the teams work. The defects are down to bad design, slopy code, and lack of testing,
We know that using agile product development is the best approach for our clients but when bidding on a project they both want and need to have some estimations of time and cost. Here we have our biggest challenges. Any suggestions?