Value Stream Metrics: If You Don’t Know Where You Are, How Do You Know Where to Go?

Demands from within the company, industry newsletters, pitches from consultants and vendors; a technology leader’s bulging inbox reminds them on a daily basis that they must do something to keep up with leading digital innovators. Migrating to the cloud, hiring more developers, investing in AI, machine learning, new software tooling and best practices (DevOps, SAFe®, ITIL); there seems no end to the myriad magical solutions out there to help. 

But where to start? Which problem to focus on today? And how do we fix it? Making the right decisions around business model modernization has never been more important nor challenging. But it is manageable with the right mindset and metrics. There is a way to find calm in the storm. We would do well to heed the advice of TUI Group CTO of Technology, Pieter Jordaan: “Take your business and break it down into smaller chunks and start building things. And don’t solve problems that don’t exist yet, that you haven’t yet faced”.

This past year, TUI Group—a global travel company—took proactive steps during the pandemic to redefine the digital experience for customers and employees. In leadership meetings, Jordaan and his counterparts broke down their business into manageable chunks to surface the most pressing issues. It soon became clear that they needed to unify the way that different parts of business digitally engaged with customers across multiple regions. Jordaan spearheaded the development of a centralized cloud platform to consolidate and simplify the way business units support the customer journey. Embracing a more modern digital experience, TUI was generating 50% of its revenues through the cloud platform in less than six months. A simply staggering achievement.

One of the key lessons of this story is that the company didn’t just “move to the cloud” because it’s “the thing to do”. They had a clear business goal: they knew that the cloud platform would enable their teams to better support customers and that leadership would be able to measure this technology decision (a better platform for marketing and selling to customers) against business results (i.e., leads and sales). But how did they prioritize that investment? Under immense pressure amid conflicting priorities, how did they decisively pull the trigger?

The answer is an old one; they had the metrics and visibility to baseline their product portfolio against business OKRs to ensure they weren’t marching the wrong way or chasing their tails. They also put in place the necessary feedback loops to help them learn faster what customers needed, guide their product development process and incrementally track the impact of their changes. They leveraged data-driven value stream management via the Flow Framework® to—in the words of Jordaan—to make their critical “blow up in your face areas visible and measurable”.

The beauty of data-driven VSM is that it forces us to identify the products that are the lifeblood of our business, preventing wasted time on projects that are not aligned to a business outcome (such as revenue, customer retention or product usage). It encourages us to ask a few simple questions: 

• What are the most important products in the eyes of our customers?
• What are all the activities that go into delivering and maintaining the product?
• How do we accelerate the speed of these activities, or remove some completely, to improve the flow of work across the value stream to meet customer needs better and sooner?
• Do we have the rapid end-to-end feedback loops for our customers and teams to help measure and learn at lightning speeds?

Once we know where to focus our energies, we can double down on the product value streams that are vital to our survival and ongoing success. As Brian Solis (Global Innovation Evangelist at Salesforce) emphasizes, “If you don't know how your product portfolio supports and retains customers, you need to baseline its performance and fast. Introduce measurement mechanisms that focus on your software-driven product value streams.” It’s through such mechanisms that we can track our decision making more clearly, helping us to monitor our progress as we seek to continuously improve and innovate faster.

You Can’t Manage What You Can’t Measure

“Using VSM management paradigms like the Flow Framework, IT organizations can make intangible software work visible and measurable from a business context. Using agreed outcome-based metrics that focus on business results, Technology teams can collaborate better with business leadership to remove the bottlenecks slowing their software delivery teams down.” Brian Solis, Global Innovation Evangelist, Salesforce - Customer loyalty is up for grabs; retention is now mission-critical, CIO.com

To measure the success of any endeavor, you need a set of consistent metrics for your current and future states. Too often, digital initiatives start with pervasive changes to people, process and technology, without first measuring the baseline. This is tantamount to training for a marathon without tracking your speed and distances over time.

Before starting any transformation effort, leaders should:

1. Agree on a core set of metrics for the journey
2. Gather accurate baseline metrics of the current state
3. Establish targets for improvement in the future state 

Leaders should also avoid pinning the success of their transformations on proxy metrics, i.e., measuring process, activity and operational efficiencies that are indicative of local optimizations that can often be siloed. While useful for particular stages of work,  proxy metrics are misleading, as they do not tie directly to business results and cannot be relied upon to present an accurate, holistic picture. A technology team can be deploying a hundred times per day, but if their work is not connected to the needs of the business, the results will not materialize for the business either.

Flow Metrics measure the current state of a product value stream. This allows teams to analyze and propose countermeasures and other potential improvements which form the basis of experiments. After an experiment is run—such as focusing on paying down tech debt in a quarter—Flow Metrics are used to help determine the impact. You have the means to ascertain whether you met the customer’s needs faster, and if not, why not? They encourage you to carry out further investigation to find the real root cause of your acceleration problems. Perhaps there’s an issue in QA? Or maybe the hold-up is on the business end, with product teams waiting for approval. Or maybe your development teams are overloaded as they seek to meet the flurry of requests from the business? 

Whatever it is, it prevents knee-jerk reactions like chucking developers and technology at the problem. The end-to-end focus helps you pinpoint the exact area causing you problems, and test and track your hypothesis until you get it right and move on to your next bottleneck. Successful outcomes benefit teams by providing direction for next steps and provide opportunities for other teams to learn and grow through the sharing of these success (or failure) stories across the enterprise. 

Continual Learning, Step by Step

Seeing change is not only a powerful motivator and storytelling tool; a 1% change every day adds up. As my colleague John Golke emphasizes, “Amazon’s developers are not 30x better and faster developers than others. However, they treat continuous improvement and digital innovation like interest, recognizing that it’s a compounding game.” A key part of re-baselining after every change and measuring progress is the linchpin of validating hypotheses and experiments against business results. 

It may sound obvious, but continual learning must be front and center of any successful product evolution. Because behind any continuous improvement to a product’s time to value is what Justin Watts, Director of Agile Change at Lloyds Banking Group, describes as “time to learning”. In the “Build, Learn, Measure” model, Steve Blank reiterates that our focus must be on maximum learning through incremental and iterative engineering. That’s why end-to-end value stream management and closed feedback loops are so important. The insights from the data into flow and from customer feedback informs our vision and enables companies to deliver faster against product value stream OKRS (such as customer usage) and business/financial OKRs (such as customer retention and revenue). This correlation helps bring business and technology leadership into the equation, enabling hard conversations around priorities, trade-offs and demand vs. capacity.  

Visibility for Leadership, Actionable for Teams

Another reason Flow Metrics are so popular is the fact that they provide shared visibility into business and development metrics for everyone involved in the value stream. That includes leadership, who are often balancing resources and budgets across a portfolio. The analysis helps leadership to consider which budgets produce the highest ROI in terms of improved flow. It stops you from investing on a hunch in the wrong area; the insights provide data-backed evidence to ensure you’re always measuring what matters in software delivery from a customer/business perspective. 

With key products and their value streams identified and aligned to business outcomes, you can start to analyze the flow business value by asking questions like:

• Are shorter Flow Times in Q1 correlated with increased revenue in Q2? 
• Does an investment in debt in Q1 (Flow Distribution) result in improved quality (e.g., fewer escaped defects) in subsequent releases? 
• Does an improvement in Flow Efficiency in Q3 result in reduced cost in Q4? 
• Does a reduction in Flow Load in Q2 correlate with an improvement in employee NPS scores in Q2 and lower delivery time to market (Flow Time)?

With this big picture view, you can begin to calibrate your value stream accordingly — allocate more resources, set priorities, change workflows or modify tooling to drive improvements in flow and the correlated business results. Like all CI initiatives, VSM is a journey that requires continuous guidance. By baselining and tracking your progress as you go in real-time, you will quickly have a better idea of where to go and how to get there faster than your competitors. As Blank explains, this is particularly important as you look to enter new, disruptive markets that do not yet exist but offer your business immense potential. It helps you understand what to measure through your customers’ eyes as you experiment with your products to “invent the future”.