Welcome!

@DevOpsSummit Authors: Yeshim Deniz, Liz McMillan, Robin Miller, XebiaLabs Blog, Nate Vickery

Related Topics: Machine Learning , Linux Containers, Containers Expo Blog, @DevOpsSummit

Machine Learning : Article

Can Wire Data Be #APM? | @DevOpsSummit #DevOps #AI #ML #Monitoring

In general terms APM is simply 'the monitoring and management of the performance and availability of software applications'

Can Wire Data Be APM?

I recently read something – a blog, a tweet, a LinkedIn article perhaps – describing the use of wire data to analyze application performance. I remember that the author’s use of the term “APM” in this context caused one reader to comment, complaining that “you can’t call wire data APM.” This was around the same time I referred casually to Dynatrace’s wire data offering (Data Center Real User Monitoring, or DC RUM) as both “APM for IT Operations” and “probe-based APM.” So that complaint has stuck with me, prompting me to ask – and offer an answer to – the question.

It depends, of course, on answers to related questions. How do you define APM? What role does APM play in your organization? What APM insights can wire data provide? Let’s take a brief look at each of these.

What is APM to you?
In very general terms (Wikipedia is great for this), APM is simply “the monitoring and management of the performance and availability of software applications.” This is accomplished by measuring user response times, transaction volume (load), and resource consumption, with correlation to help identify system bottlenecks. But of course that description is too vague to be of much practical use. For more depth and breadth, we can turn to Gartner’s recent update to its 2010 definition of APM. Collapsed from the original five, Gartner now identifies three functional dimensions. In brief, these are:

  1. Digital experience monitoring – understanding the availability and performance of real and synthetic users.
  2. Application discovery, tracing and diagnostics – discovering application services, tracing transactions through the infrastructure, and deep insight into code execution. (This new dimension used to be three separate but interlinked dimensions.)
  3. Application analytics – automating the isolation and analysis of performance bottlenecks and fault domains.

Beyond the helpful simplification of the dimensions, there’s another interesting change to Gartner’s definition; it now explicitly calls out requirements for specific technology platforms. Included are web and mobile, HTTP, Java and .NET; by omission, all others are excluded.

“Traditional” APM architectures
To some degree, this narrowing (or perhaps clarification) of APM’s scope to focus to web stacks reflects a general market reality, if not a vendor-driven market definition; every APM vendor supports these stacks, and most don’t venture far beyond. The standardization and openness of these platforms encouraged development of compelling solutions to meet the monitoring needs of modern customer-facing web applications. Whether an offering’s pedigree can be traced to development tools or production monitoring, there’s no doubt that the performance visibility and diagnostic insights have helped many IT organizations modernize their operations, enable DevOps initiatives, and accelerate the shift from providing infrastructure and capacity to delivering business-oriented application services.

At least for mission-critical web apps.

Mission critical? Or vendor critical?
There’s a disconnect, however, that one of my colleagues helpfully calls “the myth of mission critical.” I see that myth as having two facets.

First, at a high level, it implies that some apps are not critical, and therefore don’t warrant monitoring. Maybe it’s a matter of degree – critical vs. important. But if an application isn’t at least important, why do you still have it? And if it is important, shouldn’t you be monitoring it?

The second facet is vendor-driven; APM vendors have steered the mission-critical discussion to web apps, the “apps of engagement” that are revenue-producing for many businesses. Distilling the messaging, one might conclude that “web apps are mission-critical; non-web apps are not.” This sounds like a spin on Maslow’s law of the hammer, illustrated nicely by Grace Witherell: “When all you have is a hammer, only nails are problems.”

I’d venture to say that your Oracle EBS and Forms-based apps, your SAP ERP applications, your Epic EHR and other custom thick-client applications that might be delivered via Citrix, all running core parts of your business – these and many others like them are probably critical as well.

But these “other” (i.e., non-web) apps run on a diverse set of often proprietary or closed platforms, without the instrumentation interfaces that have facilitated agent-based APM solutions for Java and .NET.

APM for IT operations
APM solutions often – always, maybe – support many user roles. Gartner’s recent Magic Quadrant for APM suggests that in a few years, only 30% of APM buyers will be in IT operations, down from 60% today; most of the other buyers will be application support and development. This doesn’t reflect a loss of interest on the part of IT operations, but rather an expanding audience of stakeholders – some of whom will have diverging interests.

One dominant example of these different interests is reflected in the need for code-level insight, a component of the second of the three APM dimensions. Application development and support teams likely need this, at least for applications built or customized and maintained in-house. But for IT operations teams, code-level insight may be less of a priority; common operational goals such as meeting service quality SLAs, reducing MTTR, and controlling costs can be easily accomplished by other APM capabilities. And the lack of code-level monitoring doesn’t preclude effective DevOps collaboration; inter-tier transaction monitoring and parameter capture help fill the gap.

Wire data
Wire data simply refers to an approach to gathering information – about performance, availability, usage, security, users, applications, networks, and much more. The type of information and the resulting value depends on the analytics applied to the data. Unlike agent technology, however, wire data is platform agnostic, unconcerned with the source application architecture. Instead, wire data relies on protocol decodes to be relevant. First tier examples include HTTP, SAPGUI, Oracle Forms, Epic, and Citrix ICA, while back end examples include SOAP, various forms of SQL, WebSphere MQ, CORBA, and more.

Analytics transforms this interpreted wire data into useful insights, and at Dynatrace, we’ve built a wire data analytics platform (DC RUM) that focuses on delivering application and user performance insights – in fact, mirroring many of the core capabilities defined by Gartner’s APM dimensions. These include:

  • EUE monitoring to help prioritize IT actions based on business impact
  • Automatic discovery to maintain visibility in increasingly dynamic environments
  • Automatic identification of users, transaction names and parameters
  • Robust analytics to automatically isolate fault domains and break out response time components
  • Detailed analysis of network performance and its impact on user experience

For an expansion of these wire data capabilities, you can read my blog “Where APM Agents Fear to Tread.”

In fact, the primary limitation of a wire data approach to APM might simply be the lack of code-level insights; in its place, however, wire data can offer enhanced network infrastructure visibility as well as enterprise-wide application coverage; for IT operations teams, that’s often a reasonable – if not desirable – tradeoff.

A positive-sum game
In the end, however, this isn’t a zero-sum game. Real benefits can be achieved through multiple perspectives of performance. An agent’s inside-out perspective – with deep code-level insights – gains new insights with the outside-in perspective of wire data, with its greater network awareness. The point is not to argue the merits of a specific collection technology, but to offer a multi-faceted solution that meets the needs of multiple stakeholders by covering your entire application portfolio.

At Dynatrace, we’re redefining monitoring to mean every user, every app, everywhere. This doesn’t mean separate products for different environments; it means a unified and modular AI-driven solution that offers comprehensive coverage for enterprise, web and newstack environments.

So, can wire data be APM? Some APM vendors want to argue that it can’t, but it might be that they’ve got only hammers to sell.

The post Can wire data be APM? appeared first on Dynatrace blog – monitoring redefined.

More Stories By Gary Kaiser

Gary Kaiser is a Subject Matter Expert in Network Performance Analytics at Dynatrace, responsible for DC RUM’s technical marketing programs. He is a co-inventor of multiple performance analysis features, and continues to champion the value of network performance analytics. He is the author of Network Application Performance Analysis (WalrusInk, 2014).

@DevOpsSummit Stories
Developers want to create better apps faster. Static clouds are giving way to scalable systems, with dynamic resource allocation and application monitoring. You won't hear that chant from users on any picket line, but helping developers to create better apps faster is the mission of Lee Atchison, principal cloud architect and advocate at New Relic Inc., based in San Francisco. His singular job is to understand and drive the industry in the areas of cloud architecture, microservices, scalability and availability. In a keynote presentation, he spoke to a standing-room-only crowd at New York's Cloud Expo about how highly available, highly scalable systems can help developers attain the goal of better apps faster.
SYS-CON Events announced today that T-Mobile will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. As America's Un-carrier, T-Mobile US, Inc., is redefining the way consumers and businesses buy wireless services through leading product and service innovation. The Company's advanced nationwide 4G LTE network delivers outstanding wireless experiences to 67.4 million customers who are unwilling to compromise on quality and value.
Everyone wants to use containers, but monitoring containers is hard. New ephemeral architecture introduces new challenges in how monitoring tools need to monitor and visualize containers, so your team can make sense of everything. In his session at @DevOpsSummit, David Gildeh, co-founder and CEO of Outlyer, will go through the challenges and show there is light at the end of the tunnel if you use the right tools and understand what you need to be monitoring to successfully use containers in your environments.
New competitors, disruptive technologies, and growing expectations are pushing every business to both adopt and deliver new digital services. This ‘Digital Transformation’ demands rapid delivery and continuous iteration of new competitive services via multiple channels, which in turn demands new service delivery techniques – including DevOps. In this power panel at @DevOpsSummit 20th Cloud Expo, moderated by DevOps Conference Co-Chair Andi Mann, panelists will examine how DevOps helps to meet the demands of Digital Transformation – including accelerating application delivery, closing feedback loops, enabling multi-channel delivery, empowering collaborative decisions, improving user experience, and ultimately meeting (and exceeding) business goals.
Grape Up is a software company, specialized in cloud native application development and professional services related to Cloud Foundry PaaS. With five expert teams that operate in various sectors of the market across the USA and Europe, we work with a variety of customers from emerging startups to Fortune 1000 companies.
@DevOpsSummit at Cloud taking place June 6-8, 2017, at Javits Center, New York City, is co-located with the 20th International Cloud Expo and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time to wait for long development cycles that produce software that is obsolete at launch. DevOps may be disruptive, but it is essential.
SYS-CON Events announced today that Interoute, owner-operator of one of Europe's largest networks and a global cloud services platform, has been named “Bronze Sponsor” of SYS-CON's 20th Cloud Expo, which will take place on June 6-8, 2017 at the Javits Center in New York, New York. Interoute is the owner-operator of one of Europe's largest networks and a global cloud services platform which encompasses 12 data centers, 14 virtual data centers and 31 colocation centers, with connections to 195 additional third-party data centers across Europe. Its full-service Unified ICT platform serves international enterprises and many of the world’s leading service providers, as well as governments and universities.
DevOps is often described as a combination of technology and culture. Without both, DevOps isn't complete. However, applying the culture to outdated technology is a recipe for disaster; as response times grow and connections between teams are delayed by technology, the culture will die. A Nutanix Enterprise Cloud has many benefits that provide the needed base for a true DevOps paradigm.
Today we can collect lots and lots of performance data. We build beautiful dashboards and even have fancy query languages to access and transform the data. Still performance data is a secret language only a couple of people understand. The more business becomes digital the more stakeholders are interested in this data including how it relates to business. Some of these people have never used a monitoring tool before. They have a question on their mind like “How is my application doing” but no idea how to get a proper answer.
SYS-CON Events announced today that Juniper Networks (NYSE: JNPR), an industry leader in automated, scalable and secure networks, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Juniper Networks challenges the status quo with products, solutions and services that transform the economics of networking. The company co-innovates with customers and partners to deliver automated, scalable and secure networks with agility, performance and value.
Cloud Expo, Inc. has announced today that Aruna Ravichandran, vice president of DevOps Product and Solutions Marketing at CA Technologies, has been named co-conference chair of DevOps at Cloud Expo 2017. The @DevOpsSummit at Cloud Expo New York will take place on June 6-8, 2017, at the Javits Center in New York City, New York, and @DevOpsSummit at Cloud Expo Silicon Valley will take place Oct. 31-Nov. 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
DevOps is often described as a combination of technology and culture. Without both, DevOps isn't complete. However, applying the culture to outdated technology is a recipe for disaster; as response times grow and connections between teams are delayed by technology, the culture will die. A Nutanix Enterprise Cloud has many benefits that provide the needed base for a true DevOps paradigm. In his Day 3 Keynote at 20th Cloud Expo, Chris Brown, a Solutions Marketing Manager at Nutanix, will explore the ways that Nutanix technologies empower teams to react faster than ever before and connect teams in ways that were either too complex or simply impossible with traditional infrastructures.
Translating agile methodology into real-world best practices within the modern software factory has driven widespread DevOps adoption, yet much work remains to expand workflows and tooling across the enterprise. As models evolve from pockets of experimentation into wholescale organizational reinvention, practitioners find themselves challenged to incorporate the culture and architecture necessary to support DevOps at scale. In his session at @DevOpsSummit at 20th Cloud Expo, Anand Akela, Senior Director of DevOps Solutions at CA Technologies, will discuss how existing adopters are employing unified agile and DevOps techniques to engage functional processes and toolchains that deliver increased software quality, faster time-to-market and measurably improved customer experience.
Join IBM November 2 at 19th Cloud Expo at the Santa Clara Convention Center in Santa Clara, CA, and learn how to go beyond multi-speed it to bring agility to traditional enterprise applications. Technology innovation is the driving force behind modern business and enterprises must respond by increasing the speed and efficiency of software delivery. The challenge is that existing enterprise applications are expensive to develop and difficult to modernize. This often results in what Gartner calls "Bimodal IT," where business struggle to apply modern tools and practices to traditional monolithic applications. But these existing assets can be modernized and made more efficient without having to be completely overhauled. By leveraging methodologies like DevOps and agile, alongside emerging technologies like cloud-native services and containerization, traditional applications and teams can be ...
Did you know that you can develop for mainframes in Java? Or that the testing and deployment can be automated across mobile to mainframe? In his session at @DevOpsSummit at 20th Cloud Expo, Vaughn Marshall, Sr. Principal Product Owner at CA Technologies, will discuss and demo how increasingly teams are developing with agile methodologies using modern development environments and automating testing and deployments, mobile to mainframe.
In recent years, containers have taken the world by storm. Companies of all sizes and industries have realized the massive benefits of containers, such as unprecedented mobility, higher hardware utilization, and increased flexibility and agility; however, many containers today are non-persistent. Containers without persistence miss out on many benefits, and in many cases simply pass the responsibility of persistence onto other infrastructure, adding additional complexity.
Most companies are adopting or evaluating container technology - Docker in particular - to speed up application deployment, drive down cost, ease management and make application delivery more flexible overall. As with most new architectures, this dream takes a lot of work to become a reality. Even when you do get your application componentized enough and packaged properly, there are still challenges for DevOps teams to making the shift to continuous delivery and achieving that reduction in cost and increase in speed.
SYS-CON Events announced today that Twistlock, the leading provider of cloud container security solutions, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Twistlock is the industry's first enterprise security suite for container security. Twistlock's technology addresses risks on the host and within the application of the container, enabling enterprises to consistently enforce security policies, monitor and audit activity and identify and isolate threats in a container or cluster of containers.
While some vendors scramble to create and sell you a fancy solution for monitoring your spanking new Amazon Lambdas, hear how you can do it on the cheap using just built-in Java APIs yourself. By exploiting a little-known fact that Lambdas aren’t exactly single threaded, you can effectively identify hot spots in your serverless code. In his session at 20th Cloud Expo, David Martin, Principal Product Owner at CA Technologies, will give a live demonstration and code walkthrough, showing how to overcome the challenges of monitoring S3 and RDS. This presentation will provide an overview of necessary Amazon Lambda concepts and discuss how to integrate the monitoring data with other tools.
SYS-CON Events announced today that Grape Up will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct. 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Grape Up is a software company specializing in cloud native application development and professional services related to Cloud Foundry PaaS. With five expert teams that operate in various sectors of the market across the U.S. and Europe, Grape Up works with a variety of customers from emerging startups to Fortune 1000 companies.
In his keynote at 19th Cloud Expo, Sheng Liang, co-founder and CEO of Rancher Labs, discussed the technological advances and new business opportunities created by the rapid adoption of containers. With the success of Amazon Web Services (AWS) and various open source technologies used to build private clouds, cloud computing has become an essential component of IT strategy. However, users continue to face challenges in implementing clouds, as older technologies evolve and newer ones like Docker containers gain prominence. He explored these challenges and how to address them, while considering how containers will influence the direction of cloud computing.
SYS-CON Events announced today that CollabNet, a global leader in enterprise software development, release automation and DevOps solutions, will be a Bronze Sponsor of SYS-CON's 20th International Cloud Expo®, taking place from June 6-8, 2017, at the Javits Center in New York City, NY. CollabNet offers a broad range of solutions with the mission of helping modern organizations deliver quality software at speed. The company’s latest innovation, the DevOps Lifecycle Manager (DLM), supports Value Stream Mapping for the development and operations tool chain by offering DevOps Tool Chain Integration and Traceability; DevOps Tool Chain Orchestration; and DevOps Insight and Intelligence. CollabNet also offers traditional application lifecycle management, ALM, for the enterprise through its TeamForge product.
SYS-CON Events announced today that CA Technologies has been named "Platinum Sponsor" of SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, New York, and 21st International Cloud Expo, which will take place in November in Silicon Valley, California.
SYS-CON Events announced today that Hitachi, the leading provider the Internet of Things and Digital Transformation, will exhibit at SYS-CON's 20th International Cloud Expo®, which will take place on June 6-8, 2017, at the Javits Center in New York City, NY. Hitachi Data Systems, a wholly owned subsidiary of Hitachi, Ltd., offers an integrated portfolio of services and solutions that enable digital transformation through enhanced data management, governance, mobility and analytics. We help global organizations open new revenue streams, increase efficiencies, improve customer experience and ensure rapid time to market in the digital age. Only Hitachi Data Systems powers the digital enterprise by integrating the best information technology and operational technology from across the Hitachi family of companies. We combine this experience with Hitachi expertise in the internet of things to d...
DevOps is being widely accepted (if not fully adopted) as essential in enterprise IT. But as Enterprise DevOps gains maturity, expands scope, and increases velocity, the need for data-driven decisions across teams becomes more acute. DevOps teams in any modern business must wrangle the ‘digital exhaust’ from the delivery toolchain, "pervasive" and "cognitive" computing, APIs and services, mobile devices and applications, the Internet of Things, and now even blockchain.