How do you Measure the Effectiveness of Agile Approaches?

Using the correct measurements is the key to success with agile approaches. If we don’t know what we are trying to measure, how can we be successful?

Don’t Measure the Wrong Things

I meet with a lot of organizations who want to be Agile. When I ask why, most will respond with mechanical and tactical reasons:

• Improve On Time Delivery
• Align Business and Technology
• Flexibility to Respond to Change 

Are these the right things to measure?  If you achieve this, will you create a competitive advantage or even parity?  Do these represent business agility? Or even show the effectiveness of your agile approaches?

Applying Lean Thinking to Your Value Stream

Lean thinking would say to evaluate your value stream.  That is, look at the activities and the timeline from when the customer places an order to when the customer receives the goods and pays. This is called the order to cash cycle. Lean organizations work ruthlessly to shorten this cycle.

We could apply this thinking to the work of developing technology solutions in organizations. The customer order is the need or request from the external customer or an internal business stakeholder. And the satisfaction of that need is when the technology team delivers the solution and it is tested and accepted by the requester.

Based on discussions with organizations, I don’t think many give much thought to the actual steps in the process or the timeline from request to satisfaction. They often think about “on time” delivery and look at Scrum or other agile approaches to speed the delivery part of the cycle.

But they often ignore the wasted effort and time associated with the processes they use to identify, prioritize, and approve technology. And they rarely have any idea how long it takes to go from idea or request to a delivered solution.

We can use lean tools like value stream mapping to visualize and analyze our order to cash cycle. We recently mapped the value stream for a client and found that requests were typically satisfied in 14 to 26 months.

The picture at the top of this post is a generalized example of what most organizations probably follow.  In that cycle you will find that:

• 50% of the cycle is just spent gathering and analyzing these requests, and meeting to discuss and approve them. This includes the annual budgeting process which consumes lots of time and energy and produces zero value in the eyes of the customer.
• The next 25% of the time is spent on initiation activities. This includes identifying deliverables, developing resource plans, and creating schedules. There is likely also another approval step. Like the previous step, these activities consume time and energy and produce no value in the eyes of the customer.
• The last 25% of the time is when a team is given the work, they develop and test the solution, and they work with the requester to test and get acceptance. These activities actually produce value in the eyes of the customer.

Agile Approaches will Help Shorten Your Cycle

Scrum and other agile techniques can help us to improve that last 25%. But it sort of misses the point. We can deliver fast and efficiently for the last 3-4 months but we are still working on items that were identified 14-15 months earlier and working their way through slow and ineffective review processes.

Forget on time success measures.  Measuring “on time” delivery misses the point. You can deliver on time, based on your project schedule, but that is still 14 to 26 months after the need was identified.

Instead, start measuring cycle time, that is, the time from when an item is requested until it is delivered. True business agility is meeting business needs, as quickly as possible.