|By Andreas Grabner||
|October 5, 2014 07:00 PM EDT||
In my role as technology evangelist I spend a lot of time helping organizations, big and small, make their IT systems better, faster and more resilient to faults in order to support their business operations and objectives. I always find it frustrating to "argue" with our competitors about what the best solution is. I honestly think that many APM tools on the market do a good job - each with advantages and disadvantages in certain use cases. There is no "one size fits all" - there is just a "this tool fits best for your APM Maturity Level" (not saying the others wouldn't do a good job).
A lot of the arguing in the APM space is about the fundamental approach to monitoring application transactions: monitor and capture ALL details vs. monitor and capture relevant details. Along with that come topics like "overhead impact", "scalability" and "data hording vs smart analytics".
Ultimately, you want to pick the right tool to solve your problems. As you have multiple tools to choose from let me - in my role as technology evangelist - highlight some of the use cases that our customers solve. As a technologist and a blogger, what I really care about is that the right technology is applied to the right problem. As such, I feel compelled to share what I have learned working with customers in the trenches. Hopefully, this will help you understand the technology and what problem it can solve in real life problems, and cut through the propaganda. Let me start with a few use cases today and follow up with some more in follow up blog posts.
Use Cases from Steven - A Performance Engineer
The first use cases are picked from Steven - whom I reached out to after I read his question on our APM Community Forum. His company decided to move from a competitor to our APM solution and I wondered why. In an email, he highlighted that he had some initial success with the tool, and had been able to solve a couple of low hanging problems. When they decided to start taking a strategic Continuous Delivery approach to software delivery, they realized that the current tool had certain shortcomings slowing their attempts to practice DevOps.
They identified the following key problems they need to solve and what they really required from an APM solution in order to get to where they are heading:
How a user got to a problem, and not just seeing the problem itself
- Every transaction, with all details they need, out-of-the-box
- Web request/response bytes, SQL bind values, exception details for every transaction
Number of transactions executed per user and tenant used for business and cost reporting
- Capture custom business context data for every transaction
- Business transactions based on "buried" context data as not every detail is in the URL
Eliminate homegrown tools which are costly to maintain
- Provide application as well as system and infrastructure monitoring
- Integrate with other tools such as JMeter, LoadRunner, Jenkins or HP Open View
Eliminate the need to make people look at other tools and data
- Foster collaboration across Architect, Dev, Test & Ops by using same data set
- Data must be shareable with a single click
Ability to extend to custom frameworks, systems and protocols
- Bring in custom metrics from external tools via Java Plugin infrastructure
- Follow transactions across any custom protocol or technologies outside Java & .NET
Full Automation to support Continuous Delivery
- Use Metrics provided by APM for every build artifact along the deployment pipeline to act as quality gateway
- Inform APM about new deployments to prevent false alerting
Replace traditional application logging
- Eliminated log files which saves I/O and storage
- Get the log messages captured in context of a transaction and the context of the user that triggered that log message
One solution for everything
- Not just performance monitoring but also business reporting as well as deep dive diagnostics
Active community forum
- Get answers right away
- Leverage extensions already provided by the community such as plugins for Jenkins, PagerDuty, ...
Let me give you some examples for Steven's use case so that you can better decide on whether that is relevant for you as well:
Every Transaction with All Details
dynaTrace was built from the ground up to support the full software lifecycle. We as Compuware APM/dynaTrace understood that we needed a technology that captures every transaction with all details for root cause diagnostics as well as proper business monitoring without falling into a sampling mode where you lose critical information for both business and root cause diagnostics. Most of our customers claim they see little to acceptable overhead in production yet capturing 100% transactions including method arguments, SQL Statements, Log Messages or Exceptions. The magic word in our case is our PurePath (see the YouTube video) & PureStack Technology which allows dynaTrace to do exactly that. One of the several visualization of the PurePath is the Transaction Flow which is a great way to understand how your transactions flow through the system - where your hotspots are (3rd party impact, custom code issues or impact of Garbage Collection) and where your architectural issues (e.g: too many web service calls, too many SQL executions):
Transaction Flow: One View that tells it all to Devs, Architects and Operations Teams
What if you don't capture all transactions but be "smart" and focus on capturing the problematic ones? While this approach allows you to find and fix the easy-to-find problems that can be analyzed by analyzing those transactions that fail or violate the average response-time based baseline, it falls short when it comes to problems that are caused by transactions that are not "outside the norm". One example here is a database deadlock we recently analyzed for a customer. The "smart" approach only highlighted the transaction that hit the deadlock but no information was captured for those transactions actually causing the deadlock with their data manipulations. Being able to see which transactions executed which UPDATE statements at the time leading up to the deadlock is required to solve this problem.
As companies - such as Steven's - are getting into a maturity level where they grow out of "smart" average response time-based analysis it is important to have the ability to look at everything and not just the average problem. As a follow up read the blog Why Averages Suck and Percentiles are great!
Capture Custom Business Context
What is Custom Business Context? The actual business function executed such as a "Create Claim", "Transfer Money," or the name of the user or tenant of your system. Why is this not as easy as it sounds? Because many applications just don't show the business function as part of the URL or provide the user name in a cookie. A great example was given in a webinar by NJM Insurance (New Jersey Manufacturing Insurance). They were using a third-party claim management software which was designed to "hide" everything behind a claimCenter.do URL. In their case they needed dynaTrace to analyze every single transaction and pick a method argument invoked in the business layer of their app to figure out which function in their system was actually executed. On top of that they also needed to know the user that executed that function because they needed to understand which insurance office and group of employees created how many claims as they needed this for their quarterly business reports. The following shows business reporting based on the user role where the user role gets captured from a method argument within the business logic of the application:
Business Reporting requires Business Context data for every Transaction
This was only possible because dynaTrace allows you to selectively capture business context in the context of every single executed transaction. Along the PurePath you will then see things like method arguments, return values, bind values, session variables, HTTP parameters or cookie values. All to be later used for your business reporting or targeted root cause diagnostics. Here is a follow up blog post that explains business transactions in more technical detail.
High-performing enterprise Software Quality Assurance (SQA) teams validate systems that are ready for use - getting most actively involved as components integrate and form complete systems. These teams catch and report on defects, making sure the customer gets the best software possible. SQA teams have leveraged automation and virtualization to execute more thorough testing in less time - bringing Dev and Ops together, ensuring production readiness. Does the emergence of DevOps mean the end of Enterprise SQA? Does the SQA function become redundant? In her session at DevOps Summit, Anne Hungat...
Nov. 25, 2014 11:30 PM EST Reads: 1,411
The term culture has had a polarizing effect among DevOps supporters. Some propose that culture change is critical for success with DevOps, but are remiss to define culture. Some talk about a DevOps culture but then reference activities that could lead to culture change and there are those that talk about culture change as a set of behaviors that need to be adopted by those in IT. There is no question that businesses successful in adopting a DevOps mindset have seen departmental culture change, the question remains, is culture the leading edge of this change? I posit that in large enterprises,...
Nov. 25, 2014 07:00 PM EST Reads: 1,306
Enthusiasm for the Internet of Things has reached an all-time high. In 2013 alone, venture capitalists spent more than $1 billion dollars investing in the IoT space. With "smart" appliances and devices, IoT covers wearable smart devices, cloud services to hardware companies. Nest, a Google company, detects temperatures inside homes and automatically adjusts it by tracking its user's habit. These technologies are quickly developing and with it come challenges such as bridging infrastructure gaps, abiding by privacy concerns and making the concept a reality. These challenges can't be addressed w...
Nov. 25, 2014 04:30 PM EST Reads: 1,578
Our expectations for buying all sorts of consumer goods has gone through a radical transformation we now take for granted. Why should we not expect this same level of service from IT businesses? We accept the status quo for how software delivery exists today but would reject it without hesitation if it were applied to pretty much any other online consumer experience. Take pizza delivery as an example. Fifteen years ago ordering a pizza meant trying to choose an item from a grease-stained menu somebody shoved under your door. You'd make a phone call and end up speaking to somebody who sounded ...
Nov. 24, 2014 11:00 PM EST Reads: 2,356
The 4th International DevOps Summit, co-located with16th International Cloud Expo – being held June 9-11, 2015, at the Javits Center in New York City, NY – announces that its Call for Papers is now open. Born out of proven success in agile development, cloud computing, and process automation, DevOps is a macro trend you cannot afford to miss. From showcase success stories from early adopters and web-scale businesses, DevOps is expanding to organizations of all sizes, including the world's largest enterprises – and delivering real results. Among the proven benefits, DevOps is correlated with 2...
Nov. 23, 2014 12:30 PM EST Reads: 1,930
Leysin American School is an exclusive, private boarding school located in Leysin, Switzerland. Leysin selected an OpenStack-powered, private cloud as a service to manage multiple applications and provide development environments for students across the institution. Seeking to meet rigid data sovereignty and data integrity requirements while offering flexible, on-demand cloud resources to users, Leysin identified OpenStack as the clear choice to round out the school's cloud strategy. Additionally, the school sought a partner to provide OpenStack infrastructure deployment and operations expert...
Nov. 23, 2014 08:00 AM EST Reads: 1,886
We are all here because we are sold on the transformative promise of The Cloud. But what good is all of this ephemeral, on-demand infrastructure if your usage doesn't actually improve the agility and speed of your business? How must Operations adapt in order to avoid stifling your Cloud initiative? In his session at DevOps Summit, Damon Edwards, co-founder and managing partner of the DTO Solutions, will highlight the successful organizational, process, and tooling patterns of high-performing companies that have reshaped their Operations to enable the business to get full value from their Clo...
Nov. 23, 2014 02:00 AM EST Reads: 1,655
Software-driven innovation is becoming a primary approach to how businesses create and deliver new value to customers. A survey of 400 business and IT executives by the IBM Institute for Business Value showed businesses that are more effective at software delivery are also more profitable than their peers nearly 70 percent of the time (1). DevOps provides a way for businesses to remain competitive, applying lean and agile principles to software development to speed the delivery of software that meets new market requirements. IBM's new DevOps Innovation Services help address the challenge of s...
Nov. 22, 2014 08:00 PM EST Reads: 2,089
Docker offers a new, lightweight approach to application portability. Applications are shipped using a common container format and managed with a high-level API. Their processes run within isolated namespaces that abstract the operating environment independently of the distribution, versions, network setup, and other details of this environment. This "containerization" has often been nicknamed "the new virtualization." But containers are more than lightweight virtual machines. Beyond their smaller footprint, shorter boot times, and higher consolidation factors, they also bring a lot of new fea...
Nov. 22, 2014 06:45 PM EST Reads: 1,831
The move in recent years to cloud computing services and architectures has added significant pace to the application development and deployment environment. When enterprise IT can spin up large computing instances in just minutes, developers can also design and deploy in small time frames that were unimaginable a few years ago. The consequent move toward lean, agile, and fast development leads to the need for the development and operations sides to work very closely together. Thus, DevOps becomes essential for any ambitious enterprise today.
Nov. 22, 2014 05:45 PM EST Reads: 1,890
ARMONK, N.Y., Nov. 20, 2014 /PRNewswire/ -- IBM (NYSE: IBM) today announced that it is bringing a greater level of control, security and flexibility to cloud-based application development and delivery with a single-tenant version of Bluemix, IBM's platform-as-a-service. The new platform enables developers to build ap...
Nov. 22, 2014 05:30 PM EST Reads: 2,100
SYS-CON Media announced today that Aruna Ravichandran, VP of Marketing, Application Performance Management and DevOps at CA Technologies, has joined DevOps Journal’s authors. DevOps Journal is focused on this critical enterprise IT topic in the world of cloud computing. DevOps Journal brings valuable information to DevOps professionals who are transforming the way enterprise IT is done. Aruna's inaugural article "Four Essential Cultural Hacks for DevOps Newbies" discusses how to demonstrate the value of DevOps behaviors, tools and processes. She states: "DevOps is not only hot it's the approa...
Nov. 21, 2014 07:00 PM EST Reads: 1,638
"Our premise is Docker is not enough. That's not a bad thing - we actually love Docker. At ActiveState all our products are based on open source technology and Docker is an up-and-coming piece of open source technology," explained Bart Copeland, President & CEO of ActiveState Software, in this SYS-CON.tv interview at DevOps Summit at Cloud Expo®, held Nov 4-6, 2014, at the Santa Clara Convention Center in Santa Clara, CA.
Nov. 20, 2014 10:30 PM EST Reads: 1,716
DevOps Summit 2015 New York, co-located with the 16th International Cloud Expo - to be held June 9-11, 2015, at the Javits Center in New York City, NY - announces that it is now accepting Keynote Proposals. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time to wait for long development cycles that produce software that is obsolete at launch. DevOps may be disruptive, but it is essential.
Nov. 20, 2014 09:00 PM EST Reads: 1,945
At first glance, it might seem that the goals of DevOps and regulatory compliance are inherently at odds. Whereas much of the buzz around DevOps advocates delivering software at dizzying rates, compliance and security are concerned with proper oversight of the change management process to ensure that the enterprise isn’t opening itself up to potential vulnerabilities. With the tedious amount of rules and regulations that enterprises are subjected to, compliance can end up being a governor on the release process, defeating the potential benefits from adopting DevOps practices.
Nov. 19, 2014 10:00 PM EST Reads: 1,678
SYS-CON Media announced today the IBM, which offers the world’s deepest portfolio of technologies and expertise that are transforming the future of work, has launched ad campaigns on SYS-CON’s numerous online magazines such as Cloud Computing Journal, DevOps Journal, Virtualization Journal, and IoT Journal. IBM's campaigns focus on application testing, improving application development processes, common challenges in testing composite applications, continuous testing as part of the DevOps lifecycle, and deploying higher quality software faster through continuous integration testing.
Nov. 19, 2014 08:15 PM EST Reads: 1,946
Considered by many as the next step beyond Agile, DevOps has proven to be effective at accelerating development cycles, improving performance, reducing bugs and overall improving the innovation and velocity of development teams. There are a couple ways to look at DevOps: first, DevOps is a development methodology based on continuous integration and continuous delivery supported by set of configuration management tools such as Chef, Puppet, Salt and Ansible. We can also think of DevOps as a simpler set of principles that guide development and deployment practices – automate everything, monitor...
Nov. 19, 2014 08:00 PM EST Reads: 1,886
Cloud computing started a technology revolution; now DevOps is driving that revolution forward. By enabling new approaches to service delivery, cloud and DevOps together are delivering even greater speed, agility, and efficiency. No wonder leading innovators are adopting DevOps and cloud together! In his session at DevOps Summit, Andi Mann, Vice President of Strategic Solutions at CA Technologies, explored the synergies in these two approaches, with practical tips, techniques, research data, war stories, case studies, and recommendations on how to: Embrace, source, consume, and expose shared...
Nov. 17, 2014 10:00 AM EST Reads: 2,053
Software is eating the world. Companies that were not previously in the technology space now find themselves competing with Google and Amazon on speed of innovation. As the innovation cycle accelerates, companies must embrace rapid and constant change to both applications and their infrastructure, and find a way to deliver speed and agility of development without sacrificing reliability or efficiency of operations. In her Day 2 Keynote DevOps Summit, Victoria Livschitz, CEO of Qubell, discussed how IT organizations can automate just-in-time assembly of application environments - each built fo...
Nov. 17, 2014 09:00 AM EST Reads: 1,942
AppDynamics is the next-generation application performance management solution that simplifies the management of complex, business-critical apps. No one can stand slow applications - not IT operations and development teams, not the Chief Information Officer, and definitely not end users. With AppDynamics, no one has to tolerate slow performing apps ever again. AppDynamics customers include TiVo, AMICA Insurance, Expedia and StubHub.
Nov. 2, 2014 10:00 PM EST Reads: 3,064
Roger: Could you explain for our audience the significance of containerization, and how it differs from virtualization? Ben: Traditional virtualization was created over a decade ago, when applications were long-lived, monolithic, and deployed to a single server. In this world, when the problem to be solved was proliferation of single purpose physical servers – e.g. one server for Microsoft exchange, one server for Mac Print, one server for a custom Unix inventory -- it made sense to turn all of those single-purpose physical servers into single purpose virtual servers. The VM was created, whi...
Nov. 2, 2014 08:00 PM EST Reads: 7,427
In today's application economy, enterprise organizations realize that it's their applications that are the heart and soul of their business. If their application users have a bad experience, their revenue and reputation are at stake. In his session at 15th Cloud Expo, Anand Akela, Senior Director of Product Marketing for Application Performance Management at CA Technologies, will discuss how a user-centric Application Performance Management solution can help inspire your users with every application transaction.
Nov. 2, 2014 07:00 PM EST Reads: 4,215
In his keynote at DevOps Summit, David Tesar, Microsoft Technical Evangelist on Microsoft Azure and DevOps, will discuss how Microsoft teams who have made huge progress with a DevOps transformation effectively utilize operations staff and how challenges were overcome. Regardless of whether you are a startup or a mature enterprise, whether you are using PaaS, Micro Services, or Containerization, walk away with some practical tips where Ops can make a significant impact working with the development teams. Operational teams and functions are increasingly more important as the industry delivers so...
Nov. 2, 2014 07:00 PM EST Reads: 2,408
Founded in 1997, ActiveState is a global leader providing software application development and management solutions. The Company's products include: Stackato, a commercially supported Platform-as-a-Service (PaaS) that harnesses open source technologies such as Cloud Foundry and Docker; dynamic language distributions ActivePerl, ActivePython and ActiveTcl; and developer tools such as the popular Komodo Edit and Komodo IDE. Headquartered in Vancouver, Canada, ActiveState is trusted by customers and partners worldwide, across many industries including telecommunications, aerospace, software, fina...
Nov. 2, 2014 07:00 PM EST Reads: 2,656
The recent trends like cloud computing, social, mobile and Internet of Things are forcing enterprises to modernize in order to compete in the competitive globalized markets. However, enterprises are approaching newer technologies with a more silo-ed way, gaining only sub optimal benefits. The Modern Enterprise model is presented as a newer way to think of enterprise IT, which takes a more holistic approach to embracing modern technologies. This model makes use of Composable Enterprise framework put forward by Jonathan Murray of WMG.
Nov. 2, 2014 06:00 PM EST Reads: 2,348