Welcome!

@DevOpsSummit Authors: Dalibor Siroky, Pat Romanski, Elizabeth White, Liz McMillan, Stackify Blog

Related Topics: FinTech Journal, Java IoT, @CloudExpo, Cloud Security, @DevOpsSummit

FinTech Journal: Article

Load Testing Tool Must-Haves | @CloudExpo #DevOps #Monitoring

Here's what you should look for in a cloud-based load testing solution

The cloud provides two major advantages to load and performance procedures that help testing teams better model realistic behavior: instant infrastructure and geographic location. Cloud-based load testing also lowers the total cost of ownership, increases flexibility and allows testers to understand the impact of third-party components. So you're sold on using the cloud; here's what you should look for in a cloud-based load testing solution.

One of the most dangerous moves software developers and testers can make is being lulled into a false sense of security. For example, when application features and performance levels meet expectations during pre-production, only to crash and burn when presented to real users in production.

In that same vein, if your organization has any kind of performance testing strategy, chances are you're conducting load testing. However, you may not be truly emulating the real world behavior of your end users in your load tests.

Realism in load tests, when overlooked, can cause a myriad of performance problems in production, and end users won't wait around. If you're not performing accurate and realistic load testing, you risk revenue loss, brand damage and diminished employee productivity. The solution: cloud-based load testing.

Right off the bat, the cloud provides two major advantages to load and performance procedures that help testing teams better model realistic behavior: instant infrastructure and geographic location. Cloud-based load testing also lowers the total cost of ownership, increases flexibility and allows testers to understand the impact of third-party components. So you're sold on using the cloud; here's what you should look for in a cloud-based load testing solution.

Selecting a Cloud-Based Load Testing Tool
Make sure it includes the following must-haves:

1. Integration With Cloud Platforms
Take into account the extent to which the tool integrates with the cloud. If you select a solution that isn't integrated with one or more cloud platforms, you'll need to handle several steps manually-like learning how each platform you'll be using works (including its limitations & constraints) and building, maintaining and accessing the machines you want to use.

Your cloud-based load testing solution should offer integration with the cloud to simplify and accelerate the steps needed to use the cloud infrastructure. Your chosen tool should also enable fast provisioning, simplified security, improved scalability, a unified interface, advanced test launching and advanced results reporting.

2. Ability to Conduct Realistic Tests
While cloud-based load testing is, in many cases, more realistic than testing in the lab, simply moving to the cloud isn't enough to ensure the most realistic tests. To truly add a layer of realism to your cloud-based tests, you must account for bandwidth constraints experienced by real users.

Your cloud-based load testing tool should:

  • Virtualize Networks: You'll need to incorporate limited bandwidth, latency and packet loss in order to validate the rate at which virtual users download the application content. Testing under various network conditions is especially important for mobile users because unlike desktop and laptop users, they'll suffer from limited bandwidth, latency and packet loss at a much greater rate.
  • Parallelize Requests: Modern browsers have the ability to parallelize HTTP requests as they retrieve static resources. These parallel requests require more connections with the server and can lengthen response times. A cloud-based load testing tool that doesn't parallelize requests is incapable of producing the most realistic performance tests.

3. Unified Lab Testing & Cloud Testing
Cloud-based load testing is advantageous, to say the least. However, it shouldn't take the place of lab testing. Rather, the two should be used in tandem to test from inside/outside of the firewall and to validate application performance for any user regardless of device, location, network and/or bandwidth constraints.

Look for a single solution that will allow your organization to reuse scripts for both lab testing and cloud testing. This type of tool will save you time and effort, while enabling you to pinpoint performance problems that show up in cloud load testing but not in internal tests. Ultimately, a unified solution will lower licensing/training costs and allow test engineers to use their existing skills for both types of load testing.

4. Analysis, Monitoring, Reporting
Creating and running your cloud-based load tests is only one part of the equation. After the tests have run, you'll need to understand how your application behaved under load. Analysis tools are required to ensure the app met predetermined performance requirements.

With your cloud-based load testing tool, you should be able to easily analyze the collected data and create actionable reports that describe the performance of the tested application. Make sure your tool also includes a comprehensive monitoring system-understanding what was happening within the infrastructure of your application during the load test is critical in identifying any root issues.

5. Support for Web Technologies
This one is pretty straightforward. Whether you're using Adobe Flex, Microsoft Silverlight, Real-Time Messaging Protocol (RTMP), AJAX push technologies, WebSockets, etc., you'll need a tool with built-in support for tech you're using. Without this support, effectively testing your application's performance is near impossible.

The cloud provides organizations with the opportunity to improve the scale and realism of load testing, while saving time and lowering overall costs. To meet user expectations in terms of application performance, cloud-based load testing is a must! Just remember, simply moving to the cloud isn't enough-a primary driver of your success will be the tool you use and how well it uses cloud technology. Choose wisely!

More Stories By Tim Hinds

Tim Hinds is the Product Marketing Manager for NeoLoad at Neotys. He has a background in Agile software development, Scrum, Kanban, Continuous Integration, Continuous Delivery, and Continuous Testing practices.

Previously, Tim was Product Marketing Manager at AccuRev, a company acquired by Micro Focus, where he worked with software configuration management, issue tracking, Agile project management, continuous integration, workflow automation, and distributed version control systems.

@DevOpsSummit Stories
ChatOps is an emerging topic that has led to the wide availability of integrations between group chat and various other tools/platforms. Currently, HipChat is an extremely powerful collaboration platform due to the various ChatOps integrations that are available. However, DevOps automation can involve orchestration and complex workflows. In his session at @DevOpsSummit at 20th Cloud Expo, Himanshu Chhetri, CTO at Addteq, will cover practical examples and use cases such as self-provisioning infrastructure/applications, self-remediation workflows, integrating monitoring and complimenting integrations between Atlassian tools and other top tools in the industry.
"Storpool does only block-level storage so we do one thing extremely well. The growth in data is what drives the move to software-defined technologies in general and software-defined storage," explained Boyan Ivanov, CEO and co-founder at StorPool, in this SYS-CON.tv interview at 16th Cloud Expo, held June 9-11, 2015, at the Javits Center in New York City.
Is advanced scheduling in Kubernetes achievable?Yes, however, how do you properly accommodate every real-life scenario that a Kubernetes user might encounter? How do you leverage advanced scheduling techniques to shape and describe each scenario in easy-to-use rules and configurations? In his session at @DevOpsSummit at 21st Cloud Expo, Oleg Chunikhin, CTO at Kublr, answered these questions and demonstrated techniques for implementing advanced scheduling. For example, using spot instances and cost-effective resources on AWS, coupled with the ability to deliver a minimum set of functionalities that cover the majority of needs – without configuration complexity.
As Marc Andreessen says software is eating the world. Everything is rapidly moving toward being software-defined – from our phones and cars through our washing machines to the datacenter. However, there are larger challenges when implementing software defined on a larger scale - when building software defined infrastructure. In his session at 16th Cloud Expo, Boyan Ivanov, CEO of StorPool, provided some practical insights on what, how and why when implementing "software-defined" in the datacenter.
A strange thing is happening along the way to the Internet of Things, namely far too many devices to work with and manage. It has become clear that we'll need much higher efficiency user experiences that can allow us to more easily and scalably work with the thousands of devices that will soon be in each of our lives. Enter the conversational interface revolution, combining bots we can literally talk with, gesture to, and even direct with our thoughts, with embedded artificial intelligence, which can process our conversational commands and orchestrate the outcomes we request across our personal and professional realm of connected devices.