Welcome!

DevOps Journal Authors: Pat Romanski, Elizabeth White, David Cauthron, Carmen Gonzalez, Greg Ness

Related Topics: DevOps Journal, Java, SOA & WOA, Linux, Cloud Expo, SDN Journal

DevOps Journal: Blog Feed Post

Measuring Your DevOps Success

When you start delivering your DevOps initiatives, the measurement starts immediately and is constant

A survey from the Vanson Bourne market research agency (with CA) late in 2013 indicated that 39% of those surveyed had adopted some form of DevOps and 27% were planning to do so in the near future. Despite this being such a hot topic in the IT sector, and with a high level of take-up, the question we are still most commonly asked is: “Where do we start?”

Our answer is always that an organization’s current position must be baselined first. Having a baseline means you can build a business case, apply targets and goals to your projects and measure your success as you progress through your project with the ultimate goal being to report back to the board on how you used the money to save or make more money – and improved your teams’ satisfaction.

DevOps Metrics for Baselining and Measuring Success
There are hard, quantifiable technical and financial metrics we can track, such as:

  • Number and frequency of software releases
  • Volume of defects
  • Time/cost per release
  • MTTR*
  • Number and frequency of outages / performance issues
  • Revenue/profit impact of outages / performance issues
  • Number and cost of resources

It’s worth noting that one of the biggest inhibitors to success of DevOps and related tooling projects is people’s perceptions that they are at risk of losing their jobs as their work becomes automated. Often, particularly in areas like release and deployment management, we find that there are specific individuals who hold all the knowledge around a current process (they wrote all the scripts for example) and who are viewed as heroes when they are the only person who can fix an issue and often do it out of office hours and at short notice – but are, in fact, bottlenecks. These individuals are often highly talented, but feel secure in the indispensable role they have created for themselves. Though they will often be happier when freed up to do more creative and rewarding work, they are often fearful and this needs to be addressed.

Cultural Metrics
Although cultural metrics are difficult to apply hard dollar value to, DevOps is about resolving conflict in the workplace, eliminating stress and avoiding burnout – and they are measurable. Happy people are more productive – their health is better, they have more ideas, work more effectively and will put in the extra mile. You can measure across a number of key cultural indicators around feelings about change, failure, going to work, what a typical day’s work entails, in addition to a number of cultural attributes such as:

  • Cross-skilling, knowledge sharing and pairing between teams
  • Working in a fluid but focussed manner
  • Working in multidisciplinary teams
  • Organizing teams around projects rather than skill-sets
  • Constantly dancing on the edge of failure (in a good way)
  • Position around business demand
  • Extraneous lines of code
  • Attitude to continuous improvement
  • Obsession with metrics
  • Technological experimentation
  • Team autonomy

You can also look at a number of team features such as:

  • Rewards and feelings of success
  • Hierarchical and political obstacles and annoyances
  • Inspiring and fostering creativity

Process Metrics
DevOps is not a process or a tool – but there are a number of processes in the software development lifecycle (SDLC) that affect both traditional development and operations staff to greater or lesser degrees that need to be taken into consideration. All of these process components can be optimized, and all of them can then be improved upon further using appropriate software tooling. An ultimate goal of a typical DevOps project is often to attain true continuous delivery (CD) by linking these processes and tools together to allow fully tested, production-ready, committed code to proceed to live without impediment – we often refer to the software infrastructure piece of this as the DevOps toolchain. When baselining current state, it’s useful to measure these component processes and their relative maturity (taking into account use of existing tools and success of implementation). Typically, we look at:

  • Requirements elicitation and management
  • Agile development
  • Build
  • Release and deployment
  • Unit testing
  • User Acceptance Testing
  • Quality Assurance
  • Application Performance Monitoring
  • Cloud

How to Influence Metrics
Once you’ve baselined your current position, it’s time to think about your desired future state. Your baselining exercise will probably have highlighted where the key bottlenecks are and areas on which to concentrate. Although we preach, “People, then Process, then Tools”, there are tools, in particular Application Performance Management that can help discover bottlenecks and issues upfront – although it’s imperative you have the right people then using and acting on this data and put the right processes in behind in terms of defect tracking, corrective development, versioning build and deployment.

While you might be stumped on ideas for influencing cultural, especially in well-established enterprises, it is by no means impossible. The key is understanding the current culture. For instance, does a culture of “blame” exist? What happens when there is a production outage? How motivated and rewarded do staff feel? Once you’ve identified the prevailing culture you can establish a program of cultural initiatives to move the culture to one that is more productive and collaborative.

DevOps Tools for Change
There are a number of tools that can influence the harder and software metrics – for example:

Application Performance Monitoring (APM)

  • Reduces MTTR
  • Makes it easier to create a collaborative approach in teams dealing with issues
  • Identifies root cause fast; eliminating blame games

Application Release Automation (ARA)

  • Enables development to seamlessly transition code to operations who can quickly and consistently deploy into production
  • Enables instant rollback or redeploy when an error is identified in production
  • Reduces fear of failure as rollback/redeploy is so easy

Integration Testing & Virtualization

  • Mimics the production environment so successful test are guaranteed to run
  • Allows testers to ‘shift left’ in test process and collaborate with developers early
  • Fast testing enables fast, confident throughput of change

When to Measure, How to Tweak
When you start delivering your DevOps initiatives, the measurement starts immediately and is constant. You will be looking for upward trends, as well as downward trends. Make sure you share reports regularly with the team – try weekly with the core team and monthly with the extended team. Highlight success and elicit ideas for improvement where areas have proved more challenging. Try things – tweak, monitor, tweak again. But remember: “Any improvement not made at the constraint is an illusion.”

What to Do with Success
Celebrate success! Create rewards and incentive programmes for teams when metrics targets are achieved. Part of the DevOp’s agenda is about improving working conditions – depressurising and destressing environments and having everyone working together in harmony and eliminating disasters, catastrophes, blame and brinkmanship.

Why ask for an external DevOps Maturity Assessment
While no one’s going to understand your business as well as yourselves, we often meet organizations who are struggling to find the time – they know there are improvements to be made but they are so busy with firefighting they can’t conceive of stopping and taking stock of their current position. Also, as human beings are emotional creatures — it’s natural for all working environments to have some level of politics or hierarchies going on — it’s often helpful to have an outsider take a pragmatic, neutral view of a situation. So if you are ready to baseline your current DevOps state and identify the DevOps initiatives that will have real, positive impact on your business, but feel you do not have the time to figure it all out yourselves – please do get in touch with us @ranger4ltd.

* The MTTR is the Mean Time to Repair, Resolve or Resolution – each of the definitions mean the same thing and can be used interchangeably. This term is more commonly used when talking about Application Performance Management and the speed at which an outage or performance issue can be fixed, but equally can be used when talking about testing and eliminating defects

 

Take five minutes to get complete visibility into the performance of your production applications with AppDynamics today.

This is guest post by our partners at Ranger 4. Find out more about them on by following @ranger4ltd on twitter.

The post Measuring Your DevOps Success written by Helen Beal appeared first on Application Performance Monitoring Blog from AppDynamics.

Read the original blog entry...

More Stories By Sandi Mappic

Sandi Mappic has a passion for making apps go faster. She works with AppDynamics around the clock to help customers resolve performance pain and master application performance management. (This is AppDynamics blog feed written by several different AppDynamics bloggers.)

@DevOpsSummit Stories
Leysin American School is an exclusive, private boarding school located in Leysin, Switzerland. Leysin selected an OpenStack-powered, private cloud as a service to manage multiple applications and provide development environments for students across the institution. Seeking to meet rigid data sovereignty and data integrity requirements while offering flexible, on-demand cloud resources to users, Leysin identified OpenStack as the clear choice to round out the school's cloud strategy. Additionally, the school sought a partner to provide OpenStack infrastructure deployment and operations expert...
SYS-CON Media announced today that Aruna Ravichandran, VP of Marketing, Application Performance Management and DevOps at CA Technologies, has joined DevOps Journal’s authors. DevOps Journal is focused on this critical enterprise IT topic in the world of cloud computing. DevOps Journal brings valuable information to DevOps professionals who are transforming the way enterprise IT is done. Aruna's inaugural article "Four Essential Cultural Hacks for DevOps Newbies" discusses how to demonstrate the value of DevOps behaviors, tools and processes. She states: "DevOps is not only hot it's the approa...
The 4th International DevOps Summit, co-located with16th International Cloud Expo – being held June 9-11, 2015, at the Javits Center in New York City, NY – announces that its Call for Papers is now open. Born out of proven success in agile development, cloud computing, and process automation, DevOps is a macro trend you cannot afford to miss. From showcase success stories from early adopters and web-scale businesses, DevOps is expanding to organizations of all sizes, including the world's largest enterprises – and delivering real results. Among the proven benefits, DevOps is correlated with 2...
Software-driven innovation is becoming a primary approach to how businesses create and deliver new value to customers. A survey of 400 business and IT executives by the IBM Institute for Business Value showed businesses that are more effective at software delivery are also more profitable than their peers nearly 70 percent of the time (1). DevOps provides a way for businesses to remain competitive, applying lean and agile principles to software development to speed the delivery of software that meets new market requirements. IBM's new DevOps Innovation Services help address the challenge of s...

ARMONK, N.Y., Nov. 20, 2014 /PRNewswire/ --  IBM (NYSE: IBM) today announced that it is bringing a greater level of control, security and flexibility to cloud-based application development and delivery with a single-tenant version of Bluemix, IBM's platform-as-a-service. The new platform enables developers to build ap...

"Our premise is Docker is not enough. That's not a bad thing - we actually love Docker. At ActiveState all our products are based on open source technology and Docker is an up-and-coming piece of open source technology," explained Bart Copeland, President & CEO of ActiveState Software, in this SYS-CON.tv interview at DevOps Summit at Cloud Expo®, held Nov 4-6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
DevOps Summit 2015 New York, co-located with the 16th International Cloud Expo - to be held June 9-11, 2015, at the Javits Center in New York City, NY - announces that it is now accepting Keynote Proposals. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time to wait for long development cycles that produce software that is obsolete at launch. DevOps may be disruptive, but it is essential.
At first glance, it might seem that the goals of DevOps and regulatory compliance are inherently at odds. Whereas much of the buzz around DevOps advocates delivering software at dizzying rates, compliance and security are concerned with proper oversight of the change management process to ensure that the enterprise isn’t opening itself up to potential vulnerabilities. With the tedious amount of rules and regulations that enterprises are subjected to, compliance can end up being a governor on the release process, defeating the potential benefits from adopting DevOps practices.
SYS-CON Media announced today the IBM, which offers the world’s deepest portfolio of technologies and expertise that are transforming the future of work, has launched ad campaigns on SYS-CON’s numerous online magazines such as Cloud Computing Journal, DevOps Journal, Virtualization Journal, and IoT Journal. IBM's campaigns focus on application testing, improving application development processes, common challenges in testing composite applications, continuous testing as part of the DevOps lifecycle, and deploying higher quality software faster through continuous integration testing.
Considered by many as the next step beyond Agile, DevOps has proven to be effective at accelerating development cycles, improving performance, reducing bugs and overall improving the innovation and velocity of development teams. There are a couple ways to look at DevOps: first, DevOps is a development methodology based on continuous integration and continuous delivery supported by set of configuration management tools such as Chef, Puppet, Salt and Ansible. We can also think of DevOps as a simpler set of principles that guide development and deployment practices – automate everything, monitor...
Cloud computing started a technology revolution; now DevOps is driving that revolution forward. By enabling new approaches to service delivery, cloud and DevOps together are delivering even greater speed, agility, and efficiency. No wonder leading innovators are adopting DevOps and cloud together! In his session at DevOps Summit, Andi Mann, Vice President of Strategic Solutions at CA Technologies, explored the synergies in these two approaches, with practical tips, techniques, research data, war stories, case studies, and recommendations on how to: Embrace, source, consume, and expose shared...
Software is eating the world. Companies that were not previously in the technology space now find themselves competing with Google and Amazon on speed of innovation. As the innovation cycle accelerates, companies must embrace rapid and constant change to both applications and their infrastructure, and find a way to deliver speed and agility of development without sacrificing reliability or efficiency of operations. In her Day 2 Keynote DevOps Summit, Victoria Livschitz, CEO of Qubell, discussed how IT organizations can automate just-in-time assembly of application environments - each built fo...
Our expectations for buying all sorts of consumer goods has gone through a radical transformation we now take for granted. Why should we not expect this same level of service from IT businesses? We accept the status quo for how software delivery exists today but would reject it without hesitation if it were applied to pretty much any other online consumer experience. Take pizza delivery as an example. Fifteen years ago ordering a pizza meant trying to choose an item from a grease-stained menu somebody shoved under your door. You'd make a phone call and end up speaking to somebody who sounded ...
AppDynamics is the next-generation application performance management solution that simplifies the management of complex, business-critical apps. No one can stand slow applications - not IT operations and development teams, not the Chief Information Officer, and definitely not end users. With AppDynamics, no one has to tolerate slow performing apps ever again. AppDynamics customers include TiVo, AMICA Insurance, Expedia and StubHub.
Roger: Could you explain for our audience the significance of containerization, and how it differs from virtualization? Ben: Traditional virtualization was created over a decade ago, when applications were long-lived, monolithic, and deployed to a single server. In this world, when the problem to be solved was proliferation of single purpose physical servers – e.g. one server for Microsoft exchange, one server for Mac Print, one server for a custom Unix inventory -- it made sense to turn all of those single-purpose physical servers into single purpose virtual servers. The VM was created, whi...
In his keynote at DevOps Summit, David Tesar, Microsoft Technical Evangelist on Microsoft Azure and DevOps, will discuss how Microsoft teams who have made huge progress with a DevOps transformation effectively utilize operations staff and how challenges were overcome. Regardless of whether you are a startup or a mature enterprise, whether you are using PaaS, Micro Services, or Containerization, walk away with some practical tips where Ops can make a significant impact working with the development teams. Operational teams and functions are increasingly more important as the industry delivers so...
Founded in 1997, ActiveState is a global leader providing software application development and management solutions. The Company's products include: Stackato, a commercially supported Platform-as-a-Service (PaaS) that harnesses open source technologies such as Cloud Foundry and Docker; dynamic language distributions ActivePerl, ActivePython and ActiveTcl; and developer tools such as the popular Komodo Edit and Komodo IDE. Headquartered in Vancouver, Canada, ActiveState is trusted by customers and partners worldwide, across many industries including telecommunications, aerospace, software, fina...
In today's application economy, enterprise organizations realize that it's their applications that are the heart and soul of their business. If their application users have a bad experience, their revenue and reputation are at stake. In his session at 15th Cloud Expo, Anand Akela, Senior Director of Product Marketing for Application Performance Management at CA Technologies, will discuss how a user-centric Application Performance Management solution can help inspire your users with every application transaction.
The impact of DevOps in the cloud era is potentially profound. DevOps helps businesses deliver new features continuously, reduce cycle time and achieve sustained innovation by applying agile and lean principles to assist all stakeholders in an organization that develop, operate, or benefit from the business’ lifecycle. In his session at DevOps Summit, Prashanth Chandrasekar, General Manager at Rackspace, will exam whether / how companies can work with external DevOps specialists to achieve "DevOps elasticity" and DevOps expertise at scale while internally focusing on writing code / developme...
SYS-CON Events announced today that Zentera Systems, an industry visionary delivering hybrid-cloud management solutions, will exhibit at DevOps Summit Silicon Valley, which will take place on November 4–6, 2014, at the Santa Clara Convention Center in Santa Clara, CA. Zentera Systems, Inc.™ is a Silicon Valley based private company, providing a Cloud Federation Platform (CFP) built on a virtualization architecture with patent-pending technology to address virtual network, cloud firewall, data protection and transport automation within and across cloud domains. Zentera is solving the security ...
The recent trends like cloud computing, social, mobile and Internet of Things are forcing enterprises to modernize in order to compete in the competitive globalized markets. However, enterprises are approaching newer technologies with a more silo-ed way, gaining only sub optimal benefits. The Modern Enterprise model is presented as a newer way to think of enterprise IT, which takes a more holistic approach to embracing modern technologies. This model makes use of Composable Enterprise framework put forward by Jonathan Murray of WMG.
Enthusiasm for the Internet of Things has reached an all-time high. In 2013 alone, venture capitalists spent more than $1 billion dollars investing in the IoT space. With “smart” appliances and devices, IoT covers wearable smart devices, cloud services to hardware companies. Nest, a Google company, detects temperatures inside homes and automatically adjusts it by tracking its user’s habit. These technologies are quickly developing and with it come challenges such as bridging infrastructure gaps, abiding by privacy concerns and making the concept a reality. These challenges can’t be addressed w...
Achieve continuous delivery of applications by leveraging ElasticBox and Jenkins. In his session at DevOps Summit, Monish Sharma, VP of Customer Success at ElasticBox, will demonstrate how you can achieve the following using ElasticBox and the ElasticBox Jenkins Plugin: Create consistency across dev, staging, and production environments Continuous delivery across multiple clouds to handle high loads Ensure consistent policy management across environments: tagging, admin boxes, traceability Spin up machines and environments quickly Deploy applications to any cloud Enable real-time collaboration...
Qubell, an innovator in application deployment and configuration management, empowers online companies to do what they have never been able to do before: put into consumers' hands innovative new features and services, almost as fast as they can conceive them, without sacrificing control, reliability or uptime. Qubell emerged from stealth in the summer of 2013 (see related press release) and announced that Kohl's completed its initial implementation (see press release). Founded by pioneers in enterprise cloud applications and services, Qubell has its headquarters in Menlo Park, Calif. For more ...
BlueBox bridge the chasm between development and infrastructure. Hosting providers are taking standardization and automation too far. For many app developers it does nothing but spawn mayhem and more work. They have to figure out how their creations live on a pre-fab infrastructure solution full of constraints. Operations-as-a-Service is what BlueBox does. BlueBox utilizes development tools such as OpenStack, EMC Razor, Opscode’s Chef and BlueBox's proprietary tools give the power to do the unorthodox things which most hosting providers shun.