Welcome!

@DevOpsSummit Authors: Liz McMillan, Pat Romanski, Yeshim Deniz, Elizabeth White, SmartBear Blog

Related Topics: Microservices Expo

Microservices Expo: Article

Load Testing Web Services

Software testing is crucial to SDLC and load testing is integral to any efficient testing scheme

The quality of any application is determined by the robustness and scalability of the system. It's mandatory to simulate the actual environment and test the application for preparedness. Web Services-savvy applications need a different methodology for testing in a real-world scenario. The UI-less nature of Web Services presents a significant challenge in testing such applications. The whole persona of consumer stubs with different payloads dictates the planning of Web Services load-testing schemes. This paper talks about the different aspects of load testing and areas of contention that need special attention. This will be helpful in not only building a better application but also compiling a robust, high-quality enterprise architecture.

Web Services are the natural delivery mechanism to achieve SOA. While having the potential to free enterprises from the endless cycle of vendor-specific hardware/software upgrades by ensuring interoperability, they bring in integration complexities and the overhead of maintaining compatibility with the underlying EIS applications/systems. This brings in an absolutely different perspective to testing Web Services.

Web Services applications generally use a lot of data transformation, wraparounds (wrappers), translation, and abstraction to bring about the promised interoperability and portability. Their dependence on bandwidth-heavy protocols like SOAP doesn't ensure many performance benefits when compared to legacy applications (which tend to be very tightly coupled). Parameters like response time, throughput, and CPU utilization for transactions determine the viability of a real-world business application. Extensive testing of Web Services based on these parameters brings to the fore the most common performance constraints associated with them. The test results not only indicate whether the associated benchmarks are attained, but also if the service can scale to meet demands imposed by concurrent access from multiple users, simulated or otherwise.

Web Service endpoints generally also have very high visibility. They have to service multiple clients over the network simultaneously, maintaining robustness and availability at the same time. In such a situation, performance becomes even more crucial. Thus, the significance of proper performance testing for Web Services can't be overemphasized.

A Web Service, like any other application, can be subject to a wide range of test conditions and testing strategies. Some of them being functional testing, regression testing, performance testing, stress testing, and load testing. This paper will focus only on the load testing of Web Services. The expected behavior of a Web Service will be evaluated against various performance criteria when concurrent access by multiple clients is simulated. It becomes crucial to ensure that apart from optimizing design and implementation, Web Services have to be tested for throughput, efficiency, and response simulating real-world conditions as closely as possible. This is where load testing plays a major role. A properly designed load-testing strategy can simulate real-world load and performance scenarios with minimal hassle and cost. User loads and network conditions of varying nature can be effortlessly created and replicated. Testing can be undertaken till the output charts show a performance range considered acceptable for an application of its nature. Load-testing results can hence be taken as a strong indicator of application performance in actual business environments.

To ensure optimal testing of Web Services, the test cases have been designed keeping the following parameters in mind:

  • Size of payload: This tests the amount or size of the incoming requests. This parameter is vital in determining the threshold of data beyond which the service behaves in unexpected ways.
  • Concurrency: The test cases have to simulate simultaneous access of the service by multiple clients to replicate real-world conditions.
  • Latency: It can be defined as the time from issuing a request to the service from the client and the receipt of the first response. This parameter encapsulates the performance of the service, bandwidth in the network, and other communication overheads. It's important that latency be minimized as far as possible, at least up to a tolerable point.
  • System utilization: The net CPU and memory resources consumed by the service under varying loads in normal enterprise environments should be captured by the load-testing scheme. This helps in identifying potential bottlenecks and points out areas of improvement.
These parameters shall be discussed in detail later:

Load Testing with Reference to Web Services
Load testing of Web Services is significantly different from testing of other applications since their performance is not just attributed to how robust the underlying architecture is but also to the network overheads, underlying processing involved, and the performance of the Web server that hosts the service. The behavior of the SOAP engine also invariably adds to the architecture of service provider systems. Certain major areas of contention when evaluating the Web Service performance that will be discussed here are:

  • The impact of an incremental XML payload size wrapped inside a SOAP message
  • The impact of a chosen style/use during the design of a Web Service
  • The serialization/de-serialization involved in processing the SOAP message
  • The underlying parsing model and validation schemes chosen to process the XML payload
The results of the load testing such as response time graphs will further depict the vitality of the load testing of Web Services to ascertain their conformity prior to actual deployment to enable their wide-scale adoption without compromising their performance and scalability characteristics, enabling the enforcement of stringent operational, behavioral, and non-functional requirements that are inherent in the successful realization of any business process.

Load Testing Metrics and Parameters
The results obtained by load testing Web Services can potentially be reflected in terms of the following parameters.

  • Response time: It's the most important parameter to reflect the quality of a Web Service. Response time is the total time it takes after the client sends a request till it gets a response. This includes the time the message remains in transit on the network, which can't be measured exclusively by any load-testing tool. So we're restricted to testing Web Services deployed on a local machine. The result will be a graph measuring the average response time against the number of virtual users.
  • Number of transactions passed/failed: This parameter simply shows the total number of transactions passed or failed.
  • Throughput: It's measured in bytes and represents the amount of data that the virtual users receive from the server at any given second. We can compare this graph to the response-time graph to see how the throughput affects transaction performance.
  • Load size: The number of concurrent virtual users trying to access the Web Service at any particular instance in an interval of time.
  • CPU utilization: The amount of CPU time used by the Web Service while processing the request.
  • Memory utilization: The amount of memory used by the Web Service while processing the request.
  • Wait Time (Average Latency): The time it takes from when a request is sent until the first byte is received.
Performance Bottlenecks & Areas of Contention
Web Services are simply components deployed on a server. Most of the Web Services today are exposed out of existing components such as Enterprise Java Beans. Hence, in theory, we should be able to use the existing testing mechanisms and performance-enhancing strategies. But as already discussed load testing Web Services is quite different. The performance of Web Services is influenced by a lot of factors like bottlenecks in the network, processing at intermediate nodes if any, pre-processing of the SOAP message at the SOAP engine before it's dispatched to the service, etc. To identify the areas of contention, we'll look first at the architecture of the SOAP message processing on the service side. (see Figure 1)

A client application creates a SOAP message containing the XML payload, which can be either a SOAP-RPC-encoded request or a document-style message. The client sends this message along with the service endpoint URL to the SOAP client runtime, which in turn sends it over the network. Once the SOAP message is delivered to the SOAP runtime at the service, it passes through handlers (if any) that handle the processing of any additional tags for WS-Security, WS-Addressing, etc. Then the SOAP runtime converts the XML message into programming language-specific objects if required by the application. The Web Service processes the request message and formulates a response. The SOAP runtime on the service side takes care of creating a SOAP message and dispatching it back to the client.

So, apart from the actual processing of the Web Service, there's some additional processing involved before and after the Web Service builds a response. Let's identify the bottlenecks involved in invoking a Web Service:

  • XML processing and overheads: Since XML data is verbose and contains lot of metadata information, processing of XML is a major performance bottleneck. Processing XML involves parsing, validating the data against schema, and marshalling/unmarshalling. It's quite memory- and CPU-intensive and the response time takes a hit if the proper strategies aren't followed.
  • Parsing of XML: The larger the SOAP message, the longer it takes to parse it. Parsing SOAP messages is a major contributor to performance issues with Web Services. A memory-efficient parser like StAX (a pull parser) should be used in place of a memory-intensive parser like DOM.
  • Serialization/de-serialization: When the SOAP engine on the service side receives a SOAP request, it de-serializes the XML data according to the encoding format mentioned, i.e., extracts the payload out of it, and creates objects that are used by the Web Service. After the Web Service executes the business logic, the SOAP engine takes care of serializing the response back to XML and sends the data to the client. For huge XML documents, the serialization/de-serialization takes a performance hit if a proper mechanism isn't selected.
  • Select a proper style for your Web Service: The two most pre-dominantly used styles of Web Services are document/literal and RPC/encoded. The SOAP message of the RPC-encoded style of Web Service contains the type-encoding information, which is an overhead on the SOAP engine and degrades the throughput performance whereas the document/literal SOAP message contains no such type-encoding information. The XML payload can be easily validated against the schema included in the WSDL. Also, the data binding specific to a SOAP engine can be switched off in the case of a document/literal-style Web Service. This enables one to use a custom binding framework like XMLBeans, castor, JAXB, etc. This is especially useful when the application uses a large number of complex custom data types.
  • Processing by handlers: The SOAP engine first dispatches the SOAP message to the handlers. The handlers may be responsible for performing additional processing like authentication, encryption/decryption of the XML payload, parsing the SOAP message for any information like WS-Security, WS-Addressing, etc.

More Stories By Bijoy Majumdar

Bijoy Majumdar is a member of the Web Services COE (Center of Excellence) for Infosys Technologies, a global IT consulting firm, and has substantial experience in publishing papers, presenting papers at conferences, and defining standards for SOA and Web services. Prior to Infosys, Bijoy Majumdar worked as an IT Analyst, and had been a member of the GE Center of Excellence (e-center) under the E-Business Practice of Tata Consultancy Services.

More Stories By Ujval Mysore

Ujval Mysore is a member of the Web Services COE (Center of Excellence) for Infosys Tehcnologies, a global IT consulting firm, and have substantial experience in publishing papers, presenting papers at conferences, and defining standards for SOA and Web services. The Web Services COE specializes in SOA, Web services, and other related technologies. Dr. Srinivas Padmanabhuni heads the Web Services COE.

More Stories By Lipika Sahoo

Lipika Sahoo currently works with the Web Services Centre of Excellence in SETLabs, the technology research division at Infosys Technologies, India. Her core area of work involves dynamic adaptability and management of Web services. She is currently involved in various research activities within the group relating to MDA and AOSD.

More Stories By Sunny Saxena

Sunny Saxena currently works with the Web Services Centre of Excellence in SETLabs, the technology research division at Infosys Technologies, India. His interests range from Web service security platforms to aspect-oriented development models.

Comments (1) View Comments

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.


Most Recent Comments
JDJ News Desk 10/19/06 12:12:22 PM EDT

The quality of any application is determined by the robustness and scalability of the system. It's mandatory to simulate the actual environment and test the application for preparedness. Web Services-savvy applications need a different methodology for testing in a real-world scenario. The UI-less nature of Web Services presents a significant challenge in testing such applications. The whole persona of consumer stubs with different payloads dictates the planning of Web Services load-testing schemes. This paper talks about the different aspects of load testing and areas of contention that need special attention. This will be helpful in not only building a better application but also compiling a robust, high-quality enterprise architecture.

@DevOpsSummit Stories
Without lifecycle traceability and visibility across the tool chain, stakeholders from Planning-to-Ops have limited insight and answers to who, what, when, why and how across the DevOps lifecycle. This impacts the ability to deliver high quality software at the needed velocity to drive positive business outcomes. In his general session at @DevOpsSummit at 19th Cloud Expo, Eric Robertson, General Manager at CollabNet, will discuss how customers are able to achieve a level of transparency that enables everyone from Planning-to-Ops to make informed decisions based on business priority and leverage automation to accelerate identifying issues and fast fix to drive continuous feedback and KPI insight.
More and more brands have jumped on the IoT bandwagon. We have an excess of wearables – activity trackers, smartwatches, smart glasses and sneakers, and more that track seemingly endless datapoints. However, most consumers have no idea what “IoT” means. Creating more wearables that track data shouldn't be the aim of brands; delivering meaningful, tangible relevance to their users should be. We're in a period in which the IoT pendulum is still swinging. Initially, it swung toward "smart for smart's sake," and many brands remain in that corner. But many brands are also gradually opting for more strategic approaches. They're taking a breath and stepping back to examine both existing and potential IoT experiences, asking themselves whether their products lend real value. Once we reach this goal, the implications for personalization are staggering. Consumers will expect devices they use and i...
We all know that end users experience the internet primarily with mobile devices. From an app development perspective, we know that successfully responding to the needs of mobile customers depends on rapid DevOps – failing fast, in short, until the right solution evolves in your customers' relationship to your business. Whether you’re decomposing an SOA monolith, or developing a new application cloud natively, it’s not a question of using microservices - not doing so will be a path to eventual business failure. The real and more difficult question, in developing microservices-based applications, is this: What's the best combination of cloud services and tools to use to get the right results in the specific business situation in which you need to deliver what your end users’ want. Considering that new streams of IoT data are already raising the stakes on what end users expect in their mo...
We all know that end users experience the internet primarily with mobile devices. From an app development perspective, we know that successfully responding to the needs of mobile customers depends on rapid DevOps – failing fast, in short, until the right solution evolves in your customers' relationship to your business. Whether you’re decomposing an SOA monolith, or developing a new application cloud natively, it’s not a question of using microservices - not doing so will be a path to eventual business failure. The real and more difficult question, in developing microservices-based applications, is this: What's the best combination of cloud services and tools to use to get the right results in the specific business situation in which you need to deliver what your end users’ want. Considering that new streams of IoT data are already raising the stakes on what end users expect in their mo...
DXWorldEXPO LLC announced today that ICC-USA, a computer systems integrator and server manufacturing company focused on developing products and product appliances, will exhibit at the 22nd International CloudEXPO | DXWorldEXPO. DXWordEXPO New York 2018, colocated with CloudEXPO New York 2018 will be held November 11-13, 2018, in New York City. ICC is a computer systems integrator and server manufacturing company focused on developing products and product appliances to meet a wide range of computational needs for many industries. Their solutions provide benefits across many environments, such as datacenter deployment, HPC, workstations, storage networks and standalone server installations. ICC has been in business for over 23 years and their phenomenal range of clients include multinational corporations, universities, and small businesses.