Over the years, as a DevOps practitioner, I’ve had the opportunity to observe how DevOps has evolved from being a movement birthed to bring teams in contention together to something broader, wider, deeper. Pain with waterfall approaches (overruns, burnout) led to the adoption of (incremental, sustainable) agile practices that kicked off a battle between the two main groups in technology; development was practicing agile and IT Operations was not. In many organizations that conflict is resolved, but the DevOps journey has just begun.
DevOps Automation
It’s not just been about collaboration and culture of course; our technology implementations have come on in leaps and bounds too. Years ago, development teams started to use version control, practiced continuous integration, then added tests: unit tests, then integration then user experience. CICD pipelines proliferated across the enterprise and, increasingly, were delivered as a self-service internal platform by a separate enabling team.
Gaps were spotted in the CICD pipelines: non-functional requirements and tests (security especially) and the service and support desk. Monitoring and observability. We extended the CICD pipeline until it became an end-to-end DevOps toolchain, tracing the travel of a feature from its inception (when it was an idea) to when its value was realized with the customer. Then we connected the ends and made a circle. Then we understood we had a value chain or value stream.
Value Stream Mapping
While all that was happening, practitioners were increasingly turning to Value Stream Mapping as a tool to trigger or accelerate the adoption of DevOps principles and practices. The DevOps Handbook’s fifth and sixth chapters provide guidance on how to select a starting value stream and how to understand the work in it and make it visible. In part one, the authors explain:
“The same principles and patterns that enable the fast flow of work in physical processes are equally applicable to technology work (and, for that matter, for all knowledge work.) In DevOps, we typically define our technology value stream as the process required to convert a business hypothesis into a technology-enabled service that delivers value to the customer.”
As a DevOps coach, I used value stream mapping extensively with clients over several years, but there were drawbacks to the approach that appeared over and over:
Don’t get me wrong, there were huge benefits to practising these exercises:
Essentially, we have a human problem. Standard value stream mapping is an opinion driven endeavor and, whilst the gathered metrics frequently proved startlingly accurate when we managed to gather data to test it against, it doesn't truly constitute empirical evidence. And it’s human hungry - it wants and needs lots of our time and our attention and we just don’t have a lot of that. Making time for improvement in all businesses is tough. We want to spend our time in it, not on it.
DevOps Metrics
Alongside all of this, I was running DevOps metrics workshops with clients as part of their enablement. The annual State of DevOps Reports had distilled the core metrics for our industry down to just four, commonly known as the DORA metrics:
But these weren’t enough for most of the teams I was working with. We wanted some value outcome metrics too - how did we tell that the customer was delighted with the new feature we’d delivered to them? It seemed like lead time was interesting - but so was cycle time. We wanted to know how long it took to get the whole idea through the system (like we were learning in value stream mapping) not just from when a developer hit merge to trunk; we wanted flow metrics too. We realised that deployment frequency became a lot less interesting when it was pretty frequent, not just once a quarter. And practices like CICD, canary testing and AIOps were rapidly dropping MTTR. There were still ways to improve the flow from idea to value realization - the problem was identifying what they are.
Where We Were
The Epiphany
Enter, stage left: Value Stream Management. Here was a software platform that:
As Gartner’s said in their January 2020 paper, ‘The Future of DevOps Toolchains Will Involve Maximizing Flow in IT Value Streams’:
“DevOps toolchains are changing, and the discrete automation silos of the past are evolving into platforms that orchestrate application delivery as a value stream… Maximize flow in DevOps value streams by using DevOps value stream management platforms for integration, orchestration, automated compliance and value stream mapping.”