Welcome!

@DevOpsSummit Authors: Mano Marks, PagerDuty Blog, Aruna Ravichandran, Elizabeth White, Liz McMillan

Related Topics: @DevOpsSummit, Java IoT, Microservices Expo, Linux Containers, @CloudExpo, SDN Journal

@DevOpsSummit: Blog Feed Post

Measuring Your DevOps Success

When you start delivering your DevOps initiatives, the measurement starts immediately and is constant

A survey from the Vanson Bourne market research agency (with CA) late in 2013 indicated that 39% of those surveyed had adopted some form of DevOps and 27% were planning to do so in the near future. Despite this being such a hot topic in the IT sector, and with a high level of take-up, the question we are still most commonly asked is: “Where do we start?”

Our answer is always that an organization’s current position must be baselined first. Having a baseline means you can build a business case, apply targets and goals to your projects and measure your success as you progress through your project with the ultimate goal being to report back to the board on how you used the money to save or make more money – and improved your teams’ satisfaction.

DevOps Metrics for Baselining and Measuring Success
There are hard, quantifiable technical and financial metrics we can track, such as:

  • Number and frequency of software releases
  • Volume of defects
  • Time/cost per release
  • MTTR*
  • Number and frequency of outages / performance issues
  • Revenue/profit impact of outages / performance issues
  • Number and cost of resources

It’s worth noting that one of the biggest inhibitors to success of DevOps and related tooling projects is people’s perceptions that they are at risk of losing their jobs as their work becomes automated. Often, particularly in areas like release and deployment management, we find that there are specific individuals who hold all the knowledge around a current process (they wrote all the scripts for example) and who are viewed as heroes when they are the only person who can fix an issue and often do it out of office hours and at short notice – but are, in fact, bottlenecks. These individuals are often highly talented, but feel secure in the indispensable role they have created for themselves. Though they will often be happier when freed up to do more creative and rewarding work, they are often fearful and this needs to be addressed.

Cultural Metrics
Although cultural metrics are difficult to apply hard dollar value to, DevOps is about resolving conflict in the workplace, eliminating stress and avoiding burnout – and they are measurable. Happy people are more productive – their health is better, they have more ideas, work more effectively and will put in the extra mile. You can measure across a number of key cultural indicators around feelings about change, failure, going to work, what a typical day’s work entails, in addition to a number of cultural attributes such as:

  • Cross-skilling, knowledge sharing and pairing between teams
  • Working in a fluid but focussed manner
  • Working in multidisciplinary teams
  • Organizing teams around projects rather than skill-sets
  • Constantly dancing on the edge of failure (in a good way)
  • Position around business demand
  • Extraneous lines of code
  • Attitude to continuous improvement
  • Obsession with metrics
  • Technological experimentation
  • Team autonomy

You can also look at a number of team features such as:

  • Rewards and feelings of success
  • Hierarchical and political obstacles and annoyances
  • Inspiring and fostering creativity

Process Metrics
DevOps is not a process or a tool – but there are a number of processes in the software development lifecycle (SDLC) that affect both traditional development and operations staff to greater or lesser degrees that need to be taken into consideration. All of these process components can be optimized, and all of them can then be improved upon further using appropriate software tooling. An ultimate goal of a typical DevOps project is often to attain true continuous delivery (CD) by linking these processes and tools together to allow fully tested, production-ready, committed code to proceed to live without impediment – we often refer to the software infrastructure piece of this as the DevOps toolchain. When baselining current state, it’s useful to measure these component processes and their relative maturity (taking into account use of existing tools and success of implementation). Typically, we look at:

  • Requirements elicitation and management
  • Agile development
  • Build
  • Release and deployment
  • Unit testing
  • User Acceptance Testing
  • Quality Assurance
  • Application Performance Monitoring
  • Cloud

How to Influence Metrics
Once you’ve baselined your current position, it’s time to think about your desired future state. Your baselining exercise will probably have highlighted where the key bottlenecks are and areas on which to concentrate. Although we preach, “People, then Process, then Tools”, there are tools, in particular Application Performance Management that can help discover bottlenecks and issues upfront – although it’s imperative you have the right people then using and acting on this data and put the right processes in behind in terms of defect tracking, corrective development, versioning build and deployment.

While you might be stumped on ideas for influencing cultural, especially in well-established enterprises, it is by no means impossible. The key is understanding the current culture. For instance, does a culture of “blame” exist? What happens when there is a production outage? How motivated and rewarded do staff feel? Once you’ve identified the prevailing culture you can establish a program of cultural initiatives to move the culture to one that is more productive and collaborative.

DevOps Tools for Change
There are a number of tools that can influence the harder and software metrics – for example:

Application Performance Monitoring (APM)

  • Reduces MTTR
  • Makes it easier to create a collaborative approach in teams dealing with issues
  • Identifies root cause fast; eliminating blame games

Application Release Automation (ARA)

  • Enables development to seamlessly transition code to operations who can quickly and consistently deploy into production
  • Enables instant rollback or redeploy when an error is identified in production
  • Reduces fear of failure as rollback/redeploy is so easy

Integration Testing & Virtualization

  • Mimics the production environment so successful test are guaranteed to run
  • Allows testers to ‘shift left’ in test process and collaborate with developers early
  • Fast testing enables fast, confident throughput of change

When to Measure, How to Tweak
When you start delivering your DevOps initiatives, the measurement starts immediately and is constant. You will be looking for upward trends, as well as downward trends. Make sure you share reports regularly with the team – try weekly with the core team and monthly with the extended team. Highlight success and elicit ideas for improvement where areas have proved more challenging. Try things – tweak, monitor, tweak again. But remember: “Any improvement not made at the constraint is an illusion.”

What to Do with Success
Celebrate success! Create rewards and incentive programmes for teams when metrics targets are achieved. Part of the DevOp’s agenda is about improving working conditions – depressurising and destressing environments and having everyone working together in harmony and eliminating disasters, catastrophes, blame and brinkmanship.

Why ask for an external DevOps Maturity Assessment
While no one’s going to understand your business as well as yourselves, we often meet organizations who are struggling to find the time – they know there are improvements to be made but they are so busy with firefighting they can’t conceive of stopping and taking stock of their current position. Also, as human beings are emotional creatures — it’s natural for all working environments to have some level of politics or hierarchies going on — it’s often helpful to have an outsider take a pragmatic, neutral view of a situation. So if you are ready to baseline your current DevOps state and identify the DevOps initiatives that will have real, positive impact on your business, but feel you do not have the time to figure it all out yourselves – please do get in touch with us @ranger4ltd.

* The MTTR is the Mean Time to Repair, Resolve or Resolution – each of the definitions mean the same thing and can be used interchangeably. This term is more commonly used when talking about Application Performance Management and the speed at which an outage or performance issue can be fixed, but equally can be used when talking about testing and eliminating defects

 

Take five minutes to get complete visibility into the performance of your production applications with AppDynamics today.

This is guest post by our partners at Ranger 4. Find out more about them on by following @ranger4ltd on twitter.

The post Measuring Your DevOps Success written by Helen Beal appeared first on Application Performance Monitoring Blog from AppDynamics.

Read the original blog entry...

More Stories By Jyoti Bansal

In high-production environments where release cycles are measured in hours or minutes — not days or weeks — there's little room for mistakes and no room for confusion. Everyone has to understand what's happening, in real time, and have the means to do whatever is necessary to keep applications up and running optimally.

DevOps is a high-stakes world, but done well, it delivers the agility and performance to significantly impact business competitiveness.

@DevOpsSummit Stories
When you focus on a journey from up-close, you look at your own technical and cultural history and how you changed it for the benefit of the customer. This was our starting point: too many integration issues, 13 SWP days and very long cycles. It was evident that in this fast-paced industry we could no longer afford this reality. We needed something that would take us beyond reducing the development lifecycles, CI and Agile methodologies. We made a fundamental difference, even changed our culture.
"There's a growing demand from users for things to be faster. When you think about all the transactions or interactions users will have with your product and everything that is between those transactions and interactions - what drives us at Catchpoint Systems is the idea to measure that and to analyze it," explained Leo Vasiliou, Director of Web Performance Engineering at Catchpoint Systems, in this SYS-CON.tv interview at 18th Cloud Expo, held June 7-9, 2016, at the Javits Center in New York City, NY.
The 20th International Cloud Expo has announced that its Call for Papers is open. Cloud Expo, to be held June 6-8, 2017, at the Javits Center in New York City, brings together Cloud Computing, Big Data, Internet of Things, DevOps, Containers, Microservices and WebRTC to one location. With cloud computing driving a higher percentage of enterprise IT budgets every year, it becomes increasingly important to plant your flag in this fast-expanding business opportunity. Submit your speaking proposal today!
@DevOpsSummit taking place June 6-8, 2017 at Javits Center, New York City, is co-located with the 20th International Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. @DevOpsSummit at Cloud Expo New York Call for Papers is now open.
SYS-CON Events announced today that Dataloop.IO, an innovator in cloud IT-monitoring whose products help organizations save time and money, has been named “Bronze Sponsor” of SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Dataloop.IO is an emerging software company on the cutting edge of major IT-infrastructure trends including cloud computing and microservices. The company, founded in the UK but now based in San Francisco, is developing the next generation of cloud monitoring required for microservices and DevOps.
SYS-CON Events announced today that Linux Academy, the foremost online Linux and cloud training platform and community, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Linux Academy was founded on the belief that providing high-quality, in-depth training should be available at an affordable price. Industry leaders in quality training, provided services, and student certification passes, its goal is to change lives by teaching Linux and cloud technology to the tens of thousands of students that learn at the Linux Academy.
20th Cloud Expo, taking place June 6-8, 2017, at the Javits Center in New York City, NY, will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud strategy.
SYS-CON Events announced today that Super Micro Computer, Inc., a global leader in Embedded and IoT solutions, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 7-9, 2017, at the Javits Center in New York City, NY. Supermicro (NASDAQ: SMCI), the leading innovator in high-performance, high-efficiency server technology, is a premier provider of advanced server Building Block Solutions® for Data Center, Cloud Computing, Enterprise IT, Hadoop/Big Data, HPC and Embedded Systems worldwide. Supermicro is committed to protecting the environment through its “We Keep IT Green®” initiative and provides customers with the most energy-efficient, environmentally friendly solutions available on the market.
Updating DevOps to the latest production data slows down your development cycle. Probably it is due to slow, inefficient conventional storage and associated copy data management practices. In his session at @DevOpsSummit at 20th Cloud Expo, Dhiraj Sehgal, in Product and Solution at Tintri, will talk about DevOps and cloud-focused storage to update hundreds of child VMs (different flavors) with updates from a master VM in minutes, saving hours or even days in each development cycle. He will also discuss how the "Ops" side of DevOps is making their life easier and becoming invisible to developers for storage-related provisioning and application performance.
Buzzword alert: Microservices and IoT at a DevOps conference? What could possibly go wrong? In this Power Panel at DevOps Summit, moderated by Jason Bloomberg, the leading expert on architecting agility for the enterprise and president of Intellyx, panelists peeled away the buzz and discuss the important architectural principles behind implementing IoT solutions for the enterprise. As remote IoT devices and sensors become increasingly intelligent, they become part of our distributed cloud environment, and we must architect and code accordingly. At the very least, you'll have no problem filling in your buzzword bingo cards. Evangelist for F5 Networks
Containers have changed the mind of IT in DevOps. They enable developers to work with dev, test, stage and production environments identically. Containers provide the right abstraction for microservices and many cloud platforms have integrated them into deployment pipelines. DevOps and containers together help companies achieve their business goals faster and more effectively. In his session at DevOps Summit, Ruslan Synytsky, CEO and Co-founder of Jelastic, reviewed the current landscape of DevOps with containers and the benefits. In addition, he discussed known issues and solutions for enterprise applications in containers.
DevOps is being widely accepted (if not fully adopted) as essential in enterprise IT. But as Enterprise DevOps gains maturity, expands scope, and increases velocity, the need for data-driven decisions across teams becomes more acute. DevOps teams in any modern business must wrangle the ‘digital exhaust’ from the delivery toolchain, "pervasive" and "cognitive" computing, APIs and services, mobile devices and applications, the Internet of Things, and now even blockchain. In this power panel at @DevOpsSummit 19th Cloud Expo, moderated by DevOps Conference Chair Andi Mann, panelists will explore this emerging use of Big Data generated by the digital business to complete the DevOps feedback loop, and inform operational and application decisions.
Every successful software product evolves from an idea to an enterprise system. Notably, the same way is passed by the product owner's company. In his session at 20th Cloud Expo, Oleg Lola, CEO of MobiDev, will provide a generalized overview of the evolution of a software product, the product owner, the needs that arise at various stages of this process, and the value brought by a software development partner to the product owner as a response to these needs.
Cloud Expo, Inc. has announced today that Andi Mann returns to 'DevOps at Cloud Expo 2017' as Conference Chair The @DevOpsSummit at Cloud Expo will take place on June 6-8, 2017, at the Javits Center in New York City, NY. "DevOps is set to be one of the most profound disruptions to hit IT in decades," said Andi Mann. "It is a natural extension of cloud computing, and I have seen both firsthand and in independent research the fantastic results DevOps delivers. So I am excited to help the great team at @DevOpsSummit and Cloud Expo tell the world how they can leverage this emerging disruptive trend." Andi Mann, Chief Technology Advocate at Splunk, is an accomplished digital business executive with extensive global expertise as a strategist, technologist, innovator, and communicator. For over 30 years across five continents, Andi has built success with Fortune 500 corporations, vendors, go...
SYS-CON Events announced today that Catchpoint Systems, Inc., a provider of innovative web and infrastructure monitoring solutions, has been named “Silver Sponsor” of SYS-CON's DevOps Summit at 18th Cloud Expo New York, which will take place June 7-9, 2016, at the Javits Center in New York City, NY. Catchpoint is a leading Digital Performance Analytics company that provides unparalleled insight into customer-critical services to help consistently deliver an amazing customer experience. Designed for digital business, Catchpoint is the only end-user experience monitoring (EUM) platform that can simultaneously capture, index and analyze object level performance data inline across the most extensive monitor types and node coverage, enabling a smarter, faster way to preempt issues and optimize service delivery. More than 350 customers in over 30 countries trust Catchpoint to strengthen their ...
The cloud market growth today is largely in public clouds. While there is a lot of spend in IT departments in virtualization, these aren’t yet translating into a true “cloud” experience within the enterprise. What is stopping the growth of the “private cloud” market? In his general session at 18th Cloud Expo, Nara Rajagopalan, CEO of Accelerite, explored the challenges in deploying, managing, and getting adoption for a private cloud within an enterprise. What are the key differences between what is available in the public cloud and the early private clouds?
"Splunk basically takes machine data and we make it usable, valuable and accessible for everyone. The way that plays in DevOps is - we need to make data-driven decisions to delivering applications," explained Andi Mann, Chief Technology Advocate at Splunk and @DevOpsSummit Conference Chair, in this SYS-CON.tv interview at @DevOpsSummit at 19th Cloud Expo, held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA.
Adding public cloud resources to an existing application can be a daunting process. The tools that you currently use to manage the software and hardware outside the cloud aren’t always the best tools to efficiently grow into the cloud. All of the major configuration management tools have cloud orchestration plugins that can be leveraged, but there are also cloud-native tools that can dramatically improve the efficiency of managing your application lifecycle. In his session at 18th Cloud Expo, Alex Lovell-Troy, Director of Solutions Engineering at Pythian, presented a roadmap that can be leveraged by any organization to plan, analyze, evaluate, and execute on moving from configuration management tools to cloud orchestration tools. He also addressed the three major cloud vendors as well as some tools that will work with any cloud.
"Logz.io is a log analytics platform. We offer the ELK stack, which is the most common log analytics platform in the world. We offer it as a cloud service," explained Tomer Levy, co-founder and CEO of Logz.io, in this SYS-CON.tv interview at DevOps Summit, held November 3-5, 2015, at the Santa Clara Convention Center in Santa Clara, CA.
In his session at @DevOpsSummit at 19th Cloud Expo, Robert Doyle, lead architect at eCube Systems, will examine the issues and need for an agile infrastructure and show the advantages of capturing developer knowledge in an exportable file for migration into production. He will introduce the use of NXTmonitor, a next-generation DevOps tool that captures application environments, dependencies and start/stop procedures in a portable configuration file with an easy-to-use GUI. In addition to capturing configuration information between Development, Test and Production, the case study shows how NXTmonitor can create dependencies, automate health scripts and scalable performance groups to handle peak production loads.
Containers have changed the mind of IT in DevOps. They enable developers to work with dev, test, stage and production environments identically. Containers provide the right abstraction for microservices and many cloud platforms have integrated them into deployment pipelines. DevOps and Containers together help companies to achieve their business goals faster and more effectively. In his session at DevOps Summit, Ruslan Synytsky, CEO and Co-founder of Jelastic, reviewed the current landscape of DevOps with containers. In addition, he will discuss known issues and solutions for enterprise applications in containers.
In 2014, Amazon announced a new form of compute called Lambda. We didn't know it at the time, but this represented a fundamental shift in what we expect from cloud computing. Now, all of the major cloud computing vendors want to take part in this disruptive technology. In his session at 20th Cloud Expo, John Jelinek IV, a web developer at Linux Academy, will discuss why major players like AWS, Microsoft Azure, IBM Bluemix, and Google Cloud Platform are all trying to sidestep VMs and containers with heavy investments in serverless computing, when most of the industry has its eyes on Docker and containers.
DevOps and microservices are permeating software engineering teams broadly, whether these teams are in pure software shops but happen to run a business, such Uber and Airbnb, or in companies that rely heavily on software to run more traditional business, such as financial firms or high-end manufacturers. Microservices and DevOps have created software development and therefore business speed and agility benefits, but they have also created problems; specifically, they have created software security issues.
"There is a huge interest in Kubernetes. People are now starting to use Kubernetes and implement it," stated Sebastian Scheele, co-founder of Loodse, in this SYS-CON.tv interview at DevOps at 19th Cloud Expo, held November 1-3, 2016, at the Santa Clara Convention Center in Santa Clara, CA.
All organizations that did not originate this moment have a pre-existing culture as well as legacy technology and processes that can be more or less amenable to DevOps implementation. That organizational culture is influenced by the personalities and management styles of Executive Management, the wider culture in which the organization is situated, and the personalities of key team members at all levels of the organization. This culture and entrenched interests usually throw a wrench in the works because of misaligned incentives.