Welcome!

@DevOpsSummit Authors: Liz McMillan, Yeshim Deniz, Elizabeth White, Pat Romanski, Zakia Bouachraoui

Related Topics: @DevOpsSummit, Linux Containers, Containers Expo Blog

@DevOpsSummit: Blog Feed Post

Measuring #DevOps Success | @DevOpsSummit #CD #DX #Agile #Serverless

When and where is activity occurring in my delivery pipeline?

Measuring DevOps Success in Your Software Delivery Pipeline
By Jason Lenny

When it comes to measuring the success of your DevOps rollout, it can be challenging to identify the right metrics that will provide intelligence while avoiding the trap of vanity metrics that indicate action—but not necessarily progress—towards the outcome you’re looking for.

In my experience, the most valuable metric of all is the lead time between when you make a commit in source control and when that change makes it to your consumers. Some very mature organizations have even been able to link this metric to validated learning or planned outcomes in production (i.e., user engagement, revenue, or even a pivot decision). This sort of full-cycle measurement closes the DevOps build-measure-learn loop and gives you unparalleled insight into the performance of your overall delivery metrics.

This sort of advanced metric, however, can be very difficult to implement; it often requires process, technology, and cultural changes within your team to be successful. If your organization is just beginning its DevOps journey, it can be unclear where you can start getting value immediately. For teams at that point, I have listed a few ideas here that are other good areas to begin.

When and where is activity occurring in my delivery pipeline?
Measuring when individual parts of your delivery pipeline are heavily utilized can offer surprising insights on when teams may be overloaded. If your source control, build, and testing systems remain idle until late in your sprints then suddenly become overwhelmed with changes, it would suggest changes to your planning or software development process could yield real benefits. For example, implementing something like Kanban into your development process could smooth the flow of work into the system and resolve what teams may be experiencing as late-sprint chaos. Within XL Release, this sort of metric can be clearly visible within the longest task/phase reports.

What kind of activities across systems are failing and how frequently?
The reliability of the various technical processes within your pipeline can have an outsized effect on the flow of changes through your system—not to mention team morale. Unreliable tests, flaky build systems, or other components of your pipeline that are unreliable will quickly form a bottleneck to teams looking to deliver more quickly and independently. Keep a close eye on these to ensure you aren’t tripping over your own feet.

Longest releases-phases-tasks views in XL Release provide DevOps intelligence

The longest releases/phases/tasks views within XL Release can show you the way to inefficient activities.

How long does it take to get value into a consumer’s hands?
This is the cycle-time metric I mentioned earlier. Even if you aren’t yet ready to implement the full build-measure-learn loop, you are probably still ready to break your pipeline down into interesting pieces. For example, you can measure the duration and activities within your development, testing, and release phases individually to gain insight on where within these processes you have opportunity to improve. Your delivery teams probably already have insight into where the least efficient parts of your pipeline are. Begin by modeling these and you’ll be surprised what you learn.

Are we releasing to customers more frequently?
Another core DevOps metric is how frequently you release to your customers. A very important contributor to lead time, releasing more frequently will bring down the number of changes in each release, resulting in more frequent but less risky change.

XL Release Releases Per Month Report provides DevOps intelligence

The Releases per Month report in XL Release offers a window into how you’re performing.

Are we taking less time to perform these releases?
This metric goes hand in hand with the frequency of release metric above. As you release more frequently, it is important to monitor the overall level of effort and duration.

XL Release average release duration and automation report provides DevOps intelligence

XL Release can also offer insights into duration through the Average Release Duration/Automation report.

Have you identified any other key metrics that are important for teams to track, or do you have any thoughts on the ones we’ve shared above? Please share your insights in the comments section below.

The post Measuring DevOps Success in Your Software Delivery Pipeline appeared first on XebiaLabs.

More Stories By XebiaLabs Blog

XebiaLabs is the technology leader for automation software for DevOps and Continuous Delivery. It focuses on helping companies accelerate the delivery of new software in the most efficient manner. Its products are simple to use, quick to implement, and provide robust enterprise technology.

@DevOpsSummit Stories
DXWordEXPO New York 2018, colocated with CloudEXPO New York 2018 will be held November 11-13, 2018, in New York City and will bring together Cloud Computing, FinTech and Blockchain, Digital Transformation, Big Data, Internet of Things, DevOps, AI, Machine Learning and WebRTC to one location.
In this presentation, you will learn first hand what works and what doesn't while architecting and deploying OpenStack. Some of the topics will include:- best practices for creating repeatable deployments of OpenStack- multi-site considerations- how to customize OpenStack to integrate with your existing systems and security best practices.
Most DevOps journeys involve several phases of maturity. Research shows that the inflection point where organizations begin to see maximum value is when they implement tight integration deploying their code to their infrastructure. Success at this level is the last barrier to at-will deployment. Storage, for instance, is more capable than where we read and write data. In his session at @DevOpsSummit at 20th Cloud Expo, Josh Atwell, a Developer Advocate for NetApp, will discuss the role and value extensible storage infrastructure has in accelerating software development activities, improve code quality, reveal multiple deployment options through automated testing, and support continuous integration efforts. All this will be described using tools common in DevOps organizations.
Modern software design has fundamentally changed how we manage applications, causing many to turn to containers as the new virtual machine for resource management. As container adoption grows beyond stateless applications to stateful workloads, the need for persistent storage is foundational - something customers routinely cite as a top pain point. In his session at @DevOpsSummit at 21st Cloud Expo, Bill Borsari, Head of Systems Engineering at Datera, explored how organizations can reap the benefits of the cloud without losing performance as containers become the new paradigm.
The current age of digital transformation means that IT organizations must adapt their toolset to cover all digital experiences, beyond just the end users’. Today’s businesses can no longer focus solely on the digital interactions they manage with employees or customers; they must now contend with non-traditional factors. Whether it's the power of brand to make or break a company, the need to monitor across all locations 24/7, or the ability to proactively resolve issues, companies must adapt to the new world.