Welcome!

@DevOpsSummit Authors: Pat Romanski, Liz McMillan, Elizabeth White, Jason Bloomberg, Yeshim Deniz

Related Topics: @DevOpsSummit

@DevOpsSummit: Blog Post

How to Talk About Performance Testing | @DevOpsSummit #DevOps

Do you ever feel like your performance testing initiatives aren't getting the attention they deserve?

How to Talk About Performance Testing with CEOs and Business Managers

Do you ever feel like your performance testing initiatives aren't getting the attention they deserve? Do they get brushed off as technical exercises? Are they considered second-rate? Do adequate resources remain elusive?

If you've answered yes to any of these questions, there may be a perception problem at play. A business manager who thinks that performance testing is a "nice to have" probably doesn't understand how vital the effort is to the overall business.

You have the power to change that.

We all know things can get crazy in the board room. With all the moving parts in a tech-heavy company, test results can take a back-seat to other metrics. In order to help your manager and high-level staff understand how performance testing directly impacts the bottom line, you'll want to look at things from their perspective: profitability, revenue, and customer retention. That's why, in this post, we'll arm you with the language and examples you need to talk about performance testing with executives.

You've Been Warned: The Health Risks Of Non-Performance
A tech-driven company is only as good as its overall customer experience, and that customer experience is only good when the product is up and running.

Consumers are becoming harder and harder to please. The web is a competitive space, and you'll need more than a strong value proposition to keep customers happy and coming back. Performance matters to your users. They want zippy, interactive access over any device they choose. If they don't get that, they'll jump ship.

As a performance engineer, you know that one of the most risky times in a business's life is when it is actually growing and everything appears to be doing well. Think about it - you are acquiring new users, your brand reputation is expanding, and while everyone else is celebrating this success you know that this is when your infrastructure is being taxed the most. If you take a wrong step... BOOM! Your website could crash and users would flee. Momentum would be halted, and you'd be cooked. But when things are going well, no one wants to think about this scenario.

So, how do you best discuss this risk with your managers so they understand it?

The Business Language of Web & Mobile Performance
If you want to draw attention to the importance of load testing and online performance, use terms that are typically discussed in board rooms. These metrics underlie the business engine that keeps money flowing and servers running.

  • MRR (Monthly Recurring Revenue): This is the crown jewel of any subscription site. It is the money that comes in every month from paying users. If users stop using (because, for example, the site is slow), you'll see MRR decline which could mean the beginning of the end for your business (to put it lightly).
  • AOV (Average Order Value): This is a key metric for an ecommerce site, as it indicates how much money is in the cart when the user checks out. A frustrating experience may cause people to stop browsing (because it's too slow to see a lot of pages). This makes them buy less and subsequently AOV goes down.
  • Lost Traffic / Visitors: Virtually any online business monitors this factor closely because it's a leading indicator of success. The more traffic you have, the more people are on the site, and the more you can expect those visitors to convert to revenue in one way or another. If performance issues are impacting the user experience, you'll see visits decline - particularly those of returning visitors who got fed up and don't come back.
  • Conversion Rate: Do visitors buy? That's what the conversion rate measures. If they have a sub-par experience with your service, they won't buy and you'll observe that the conversion rate will go down. You don't want that to happen.
  • Cart Abandonment: In a real brick-and-mortar store, you would never walk up to the front counter with a full shopping cart, leave the cart, and then just wander off. But this happens all the time online. If it takes too long to check out, you'll see abandonment go up. Losing customers right before they're about to pay is painful.
  • Customer Churn: You spend a lot of money to acquire a new customer. When they start paying you, it's nice to keep them around. If people have a frustrating experience with web pages that don't load, they won't come back. That's customer churn, and it's money out the door. Wave goodbye!
  • Brand Reputation: If enough people experience problems on a site, it can develop a reputation for being hard to use. Once it's set, a reputation is a very hard thing to change. Anyone remember Friendster? That site pre-dated Facebook and could've been the world's most prominent social network, but it kept crashing, so people stopped using it.
  • PR Nightmare: When your site fails on Black Friday or Mother's Day or some other widely publicized moment, it can be a heart-attack-inducing moment. Fear of a PR nightmare alone can be a very powerful motivator.

Everyone Loves a Good Story
Another good way to speak with business people is to tell a story. There are few better ways to communicate urgency than by recounting what happened to someone else. The web is littered with real-life warnings. Here are a couple of tidbits that can help you communicate the relevance of performance testing and monitoring to executives.

Customers Have Ecommerce Anxiety
Did you know that research shows 10% of respondents would go through the trouble of switching banks based on mobile services alone? Further research by Kissmetrics says that web and app users care more about functionality and transactional speed than any other feature.

This anxiety translates into real dollars. Take the example of Best Buy on Black Friday 2014. Unexpected spikes in mobile traffic brought down the website and cost the company real revenue on what should've been one of the most lucrative days of the year.

Slow Sites = Poor Search Rankings
For those who didn't know, Google uses performance data in search rankings. That means slower sites will be dinged in the algorithm. While this has been true for some time, what's new is that Google now displays the fact that a site is slow in the results page.

When given the choice of search results, would you click on a site marked "slow" or would you scroll down the page to the next result? You don't want to push users away before they ever get to your site.

Stick to Prevention Strategies and Ward Off Big Problems
Do you need to know more about the business impact of performance testing for your website?Check out this handy write up from Neotys that will walk you through all the business benefits. Don't miss the section on Load Testing ROI for more detail on how performance testing methods address key risks in your industry. Finally, prepare yourself with our slideshare for more advice on talking about testing with non-experts.

Photo credit: Scott Schiller

More Stories By Tim Hinds

Tim Hinds is the Product Marketing Manager for NeoLoad at Neotys. He has a background in Agile software development, Scrum, Kanban, Continuous Integration, Continuous Delivery, and Continuous Testing practices.

Previously, Tim was Product Marketing Manager at AccuRev, a company acquired by Micro Focus, where he worked with software configuration management, issue tracking, Agile project management, continuous integration, workflow automation, and distributed version control systems.

@DevOpsSummit Stories
SYS-CON Events announced today that mruby Forum will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. mruby is the lightweight implementation of the Ruby language. We introduce mruby and the mruby IoT framework that enhances development productivity. For more information, visit http://forum.mruby.org/.
SYS-CON Events announced today that Mobile Create USA will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Mobile Create USA Inc. is an MVNO-based business model that uses portable communication devices and cellular-based infrastructure in the development, sales, operation and mobile communications systems incorporating GPS capability.
SYS-CON Events announced today that Daiya Industry will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Daiya Industry specializes in orthotic support systems and assistive devices with pneumatic artificial muscles in order to contribute to an extended healthy life expectancy. For more information, please visit https://www.daiyak.co.jp/en/.
SYS-CON Events announced today that Fusic will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Fusic Co. provides mocks as virtual IoT devices. You can customize mocks, and get any amount of data at any time in your test. For more information, visit https://fusic.co.jp/english/.
SYS-CON Events announced today that Keisoku Research Consultant Co. will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Keisoku Research Consultant, Co. offers research and consulting in a wide range of civil engineering-related fields from information construction to preservation of cultural properties. For more information, visit http://www.krcnet.co.jp/eng_site/e_index.htm.
SYS-CON Events announced today that MIRAI Inc. will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. MIRAI Inc. are IT consultants from the public sector whose mission is to solve social issues by technology and innovation and to create a meaningful future for people.
SYS-CON Events announced today that Interface Corporation will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Interface Corporation is a company developing, manufacturing and marketing high quality and wide variety of industrial computers and interface modules such as PCIs and PCI express. For more information, visit http://www.interface-amita.com/aboutus/interface_profile.asp.
SYS-CON Events announced today that Ryobi Systems will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Ryobi Systems Co., Ltd., as an information service company, specialized in business support for local governments and medical industry. We are challenging to achive the precision farming with AI. For more information, visit http://www.ryobi-sol.co.jp/en/.
SYS-CON Events announced today that Enroute Lab will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Enroute Lab is an industrial design, research and development company of unmanned robotic vehicle system. For more information, please visit http://elab.co.jp/.
SYS-CON Events announced today that Nihon Micron will exhibit at the Japan External Trade Organization (JETRO) Pavilion at SYS-CON's 21st International Cloud Expo®, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Nihon Micron Co., Ltd. strives for technological innovation to establish high-density, high-precision processing technology for providing printed circuit board and metal mount RFID tags used for communication devices. For more information, visit http://www.nihon-micron.co.jp/.
DevOps at Cloud Expo – being held October 31 - November 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA – announces that its Call for Papers is open. Born out of proven success in agile development, cloud computing, and process automation, DevOps is a macro trend you cannot afford to miss. From showcase success stories from early adopters and web-scale businesses, DevOps is expanding to organizations of all sizes, including the world's largest enterprises – and delivering real results. Among the proven benefits, DevOps is correlated with 20% faster time-to-market, 22% improvement in quality, and 18% reduction in dev and ops costs, according to research firm Vanson-Bourne. It is changing the way IT works, how businesses interact with customers, and how organizations are buying, building, and delivering software.
With major technology companies and startups seriously embracing Cloud strategies, now is the perfect time to attend 21st Cloud Expo October 31 - November 2, 2017, at the Santa Clara Convention Center, CA, and June 12-14, 2018, at the Javits Center in New York City, NY, and learn what is going on, contribute to the discussions, and ensure that your enterprise is on the right path to Digital Transformation.
Your clients expect transactions to never fail, cloud access to be fast and always on, and their data to be protected - no exceptions. Hear about how Secure Service Container (SSC), an IBM-exclusive open technology, enables secure building and hosting of next-generation applications, both cloud and on-premises. SSC protects the full stack from external and insider threats, allows automatic encryption of data in-flight and at-rest, and is tamper-resistant during installation and runtime – with no changes to applications required.
While some developers care passionately about how data centers and clouds are architected, for most, it is only the end result that matters. To the majority of companies, technology exists to solve a business problem, and only delivers value when it is solving that problem. 2017 brings the mainstream adoption of containers for production workloads. In his session at 21st Cloud Expo, Ben McCormack, VP of Operations at Evernote, will discuss how data centers of the future will be managed, how the public cloud best suits your organization, and what the future holds for operations and infrastructure engineers in a post-container world. Is a serverless world inevitable?
SYS-CON Events announced today that Massive Networks, that helps your business operate seamlessly with fast, reliable, and secure internet and network solutions, has been named "Exhibitor" of SYS-CON's 21st International Cloud Expo ®, which will take place on Oct 31 - Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. As a premier telecommunications provider, Massive Networks is headquartered out of Louisville, Colorado. With years of experience under their belt, their team of engineers can navigate the Carrier Ecosystem for your IT team acting as an extension of your business, producing a hassle-free experience.
With the rise of DevOps, containers are at the brink of becoming a pervasive technology in Enterprise IT to accelerate application delivery for the business. When it comes to adopting containers in the enterprise, security is the highest adoption barrier. Is your organization ready to address the security risks with containers for your DevOps environment? In his session at @DevOpsSummit at 21st Cloud Expo, Chris Van Tuin, Chief Technologist, NA West at Red Hat, will discuss: The top security risks with containers and how to manage these risks at scale including Images, Builds, Registry, Deployment, Hosts, Network, Storage, APIs, Monitoring/Logging, and Federation.
SYS-CON Events announced today that TMC has been named “Media Sponsor” of SYS-CON's 21st International Cloud Expo and Big Data at Cloud Expo, which will take place on Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. Global buyers rely on TMC’s content-driven marketplaces to make purchase decisions and navigate markets. Learn how we can help you reach your marketing goals.
New competitors, disruptive technologies, and growing expectations are pushing every business to both adopt and deliver new digital services. This ‘Digital Transformation’ demands rapid delivery and continuous iteration of new competitive services via multiple channels, which in turn demands new service delivery techniques – including DevOps. In this power panel at @DevOpsSummit 20th Cloud Expo, moderated by DevOps Conference Co-Chair Andi Mann, panelists examined how DevOps helps to meet the demands of Digital Transformation – including accelerating application delivery, closing feedback loops, enabling multi-channel delivery, empowering collaborative decisions, improving user experience, and ultimately meeting (and exceeding) business goals.
Cloud Expo, Inc. has announced today that Andi Mann and Aruna Ravichandran have been named Co-Chairs of @DevOpsSummit at Cloud Expo Silicon Valley which will take place Oct. 31-Nov. 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. "DevOps is at the intersection of technology and business-optimizing tools, organizations and processes to bring measurable improvements in productivity and profitability," said Aruna Ravichandran, vice president, DevOps product and solutions marketing, CA Technologies. "It's this results-driven combination of technology and business that makes me so passionate about DevOps and its future in the industry. I am truly honored to take on this co-chair role, and look forward to working with the DevOps Summit team at Cloud Expo and attendees to advance DevOps."
Many organizations adopt DevOps to reduce cycle times and deliver software faster; some take on DevOps to drive higher quality and better end-user experience; others look to DevOps for a clearer line-of-sight to customers to drive better business impacts. In truth, these three foundations go together. In this power panel at @DevOpsSummit 21st Cloud Expo, moderated by DevOps Conference Co-Chair Andi Mann, industry experts will discuss how leading organizations build application success from all three of these foundations of DevOps - speed, quality, and impact.
Most of the time there is a lot of work involved to move to the cloud, and most of that isn't really related to AWS or Azure or Google Cloud. Before we talk about public cloud vendors and DevOps tools, there are usually several technical and non-technical challenges that are connected to it and that every company needs to solve to move to the cloud. In his session at 21st Cloud Expo, Stefano Bellasio, CEO and founder of Cloud Academy Inc., will discuss what the tools, disciplines, and cultural aspects are that enterprise companies are considering to get to the cloud and eventually transform the way they build software and services.
As DevOps methodologies expand their reach across the enterprise, organizations face the daunting challenge of adapting related cloud strategies to ensure optimal alignment, from managing complexity to ensuring proper governance. How can culture, automation, legacy apps and even budget be reexamined to enable this ongoing shift within the modern software factory?
DevOps is being widely accepted (if not fully adopted) as essential in enterprise IT. But as Enterprise DevOps gains maturity, expands scope, and increases velocity, the need for data-driven decisions across teams becomes more acute. DevOps teams in any modern business must wrangle the ‘digital exhaust’ from the delivery toolchain, "pervasive" and "cognitive" computing, APIs and services, mobile devices and applications, the Internet of Things, and now even blockchain.
Agile has finally jumped the technology shark, expanding outside the software world. Enterprises are now increasingly adopting Agile practices across their organizations in order to successfully navigate the disruptive waters that threaten to drown them. In our quest for establishing change as a core competency in our organizations, this business-centric notion of Agile is an essential component of Agile Digital Transformation. In the years since the publication of the Agile Manifesto, the connection between building better software and business agility has been a tenuous one at best. But now that Agile is maturing and Digital Transformation is driving change across enterprises large and small, companies are realizing that their best bet for achieving business agility is to take the best of Agile and apply it across the entire organization.
SYS-CON Events announced today that App2Cloud will exhibit at SYS-CON's 21st International Cloud Expo®, which will take place on Oct. 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA. App2Cloud is an online Platform, specializing in migrating legacy applications to any Cloud Providers (AWS, Azure, Google Cloud).