@DevOpsSummit Authors: Ram Sonagara, Yeshim Deniz, Elizabeth White, Roger Strukhoff, Liz McMillan

Related Topics: @DevOpsSummit, Linux Containers, Containers Expo Blog

@DevOpsSummit: Article

Continuous Performance Validation | @CloudExpo #DevOps #IoT #BigData

Today, user experience is crucial to an application's success

Achieving Continuous Performance Validation with Synthetic Users

In most modern development groups, there's a big focus on creating code that works. The idea goes that in more traditional, waterfall approaches, errors in code that aren't fixed until late in the cycle are much more expensive to resolve due to unintended consequences. Today, some groups practice test-driven development as a way to ensure that code is always functional while others work with short agile sprints where all code produced must be usable in the field by the end of each sprint.

There's a common understanding about what it means for a coding task to be "done." Yet, often this "doneness" is only a measure of functionality - not necessarily usability.

Today, user experience is crucial to an application's success, and that goes well beyond what color your button is or how prominently a call-to-action is placed. Users leave your site if pages don't load fast enough or if the site simply feels sluggish when compared with your competitors' sites.

The question is: is there a process for validating that your app is performing up to your customers' expectations and have that process treat performance testing with the same amount of rigor as functional testing?

Yes. It's called Continuous Performance Validation.

Defining Continuous Performance Validation
Continuous Performance Validation is the process of continuously testing, monitoring and improving performance at every stage of the application development lifecycle, from development to production, utilizing automated and collaborative tooling.

The heart of this process is a set of performance test scenarios that sit alongside more traditional functional unit tests. These tests, in conjunction with performance SLAs that can be placed on the task board as inputs to the development process itself, give you a library of performance unit tests that you can mix and match in a number of interesting ways.

As the building blocks of Continuous Performance Validation, these elements let you form long chains of specific behavior to use as test scenarios, mimicking how users actually use your app. These transactions should be used in every phase of the application development lifecycle - development, pre-production, and production - to proactively test and monitor performance across environments.

Before code is released, these scenarios can be pieced together to analyze how load and stress impact your app in different ways. That information can be fed to Operations for infrastructure and capacity planning purposes. Meanwhile, in production, those same scenarios can be executed to see if performance trends in reality match what was predicted in test. Problems can be identified, sent back to dev & test so that issues can be patched quickly.

Synthetic Users: The Key to Continuity
Continuous Performance Validation depends on the idea of creating synthetic users that can move through the app just like real users. Throughout the synthetic users' journeys, they gather metrics on performance and the end user experience.

As part of your continuous integration process, you can continually validate the overall performance of the system by building automated scenarios and plugging them into various stages of the dev process, executed by synthetic users.

In development, synthetic users execute transactions through unit tests that can validate localized performance requirements of key systems. They can fulfill a pre-planned test requirement, similar to TDD, and they can protect against regressions as code changes.

In pre-production, synthetic users can be thrown out from the cloud for truly distributed and georealistic load testing across the entire system. These users traverse various paths through the application and help to define the scalability and geographical contours of your application's distributed user base.

In production, synthetic users operate alongside real users. They navigate the system and exercise the same user paths you have already tested. But instead of sending out millions of them, you only need to send out a few, which gather metrics about performance and availability at particular moments in time and tell you if there is going to be a problem. Synthetic users validate that your performance is up to par and give you insight as to where it is not.

Continuous Performance Validation is a powerful process for apps that need to perform. By putting it in place, you won't be treating performance as an afterthought. Performance testing will be directly integrated into your standard process of development, making performance part of everyone's job description and everyone's responsibility.

Want to learn more? Check out our webinar on Continuous Performance Validation in Agile Development.

Photo Credit: Bicentennial Man (1999) from 1492 Pictures, Columbia Pictures Corporation, Laurence Mark Productions

More Stories By Tim Hinds

Tim Hinds is the Product Marketing Manager for NeoLoad at Neotys. He has a background in Agile software development, Scrum, Kanban, Continuous Integration, Continuous Delivery, and Continuous Testing practices.

Previously, Tim was Product Marketing Manager at AccuRev, a company acquired by Micro Focus, where he worked with software configuration management, issue tracking, Agile project management, continuous integration, workflow automation, and distributed version control systems.

@DevOpsSummit Stories
Andrew Keys is Co-Founder of ConsenSys Enterprise. He comes to ConsenSys Enterprise with capital markets, technology and entrepreneurial experience. Previously, he worked for UBS investment bank in equities analysis. Later, he was responsible for the creation and distribution of life settlement products to hedge funds and investment banks. After, he co-founded a revenue cycle management company where he learned about Bitcoin and eventually Ethereal. Andrew's role at ConsenSys Enterprise is a multi-faceted approach of strategy and enterprise business development. Andrew graduated from Loyola University in Maryland and University of Auckland with degrees in economics and international finance.
The best way to leverage your Cloud Expo presence as a sponsor and exhibitor is to plan your news announcements around our events. The press covering Cloud Expo and @ThingsExpo will have access to these releases and will amplify your news announcements. More than two dozen Cloud companies either set deals at our shows or have announced their mergers and acquisitions at Cloud Expo. Product announcements during our show provide your company with the most reach through our targeted audiences.
DevOpsSummit New York 2018, colocated with CloudEXPO | DXWorldEXPO New York 2018 will be held November 11-13, 2018, in New York City. Digital Transformation (DX) is a major focus with the introduction of DXWorldEXPO within the program. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term. A total of 88% of Fortune 500 companies from a generation ago are now out of business. Only 12% still survive. Similar percentages are found throughout enterprises of all sizes.
With 10 simultaneous tracks, keynotes, general sessions and targeted breakout classes, @CloudEXPO and DXWorldEXPO are two of the most important technology events of the year. Since its launch over eight years ago, @CloudEXPO and DXWorldEXPO have presented a rock star faculty as well as showcased hundreds of sponsors and exhibitors! In this blog post, we provide 7 tips on how, as part of our world-class faculty, you can deliver one of the most popular sessions at our events. But before reading these essential tips, please take a moment and watch this brief video from Sandy Carter.
DXWorldEXPO LLC announced today that "Miami Blockchain Event by FinTechEXPO" has announced that its Call for Papers is now open. The two-day event will present 20 top Blockchain experts. All speaking inquiries which covers the following information can be submitted by email to [email protected] Financial enterprises in New York City, London, Singapore, and other world financial capitals are embracing a new generation of smart, automated FinTech that eliminates many cumbersome, slow, and expensive intermediate processes from their businesses. FinTech brings efficiency as well as the ability to deliver new services and a much improved customer experience throughout the global financial services industry. FinTech is a natural fit with cloud computing, as new services are quickly developed, deployed, and scaled on public, private, and hybrid clouds.