Web service Performance Testing using Load Runner

Web service performance testing can do two ways using the Loadrunner.

1.If client provides a Wsdl URL we will use web service protocol and record.

Select the web service protocol from the protocol list and click on create.(refer the screen shot)


Click on Manage service in the lodrunner and click on Import


Click on Import on Manage service window enter URL (wsdl) and click on Import.



Click on Apply and OK on Manage service window.



Click on Add Service Call on Load runner window, and select the methods from the Operations list


Select the objects which we are including in the web service call and give the value



And the sample code below.


Action()
{

char name[10];
int temparature;

lr_start_transaction("FirstWebservice");


web_service_call( "StepName=GetCitiesByCountry_101",
"SOAPMethod=GlobalWeather|GlobalWeatherSoap|GetCitiesByCountry",
"ResponseParam=response",
"Service=GlobalWeather",
"ExpectedResponse=SoapResult",
"Snapshot=t1291533619.inf",
BEGIN_ARGUMENTS,
"CountryName=INDIA",
END_ARGUMENTS,
BEGIN_RESULT,
END_RESULT,
LAST);

lr_xml_get_values("XML={response}",
"ValueParam=OutputParam",
"Query=//Body/GetCitiesByCountryResponse/GetCitiesByCountryResult",
LAST);

lr_xml_get_values("XML={OutputParam}",
"ValueParam=OutputParam1",
"Query=//NewDataSet/Table/City",
"SelectAll=yes",
LAST);

sprintf(name,"{OutputParam1_%d}",5);

lr_save_string(lr_eval_string(name),"name1");

lr_output_message ("The new file name is %s", lr_eval_string("{name1}"));


lr_end_transaction("FirstWebservice",LR_AUTO);



lr_start_transaction("Cityname");

web_service_call( "StepName=GetWeather_101",
"SOAPMethod=GlobalWeather|GlobalWeatherSoap|GetWeather",
"ResponseParam=response1",
"Service=GlobalWeather",
"ExpectedResponse=SoapResult",
"Snapshot=t1291533100.inf",
BEGIN_ARGUMENTS,
"CityName={name1}",
"CountryName=INDIA",
END_ARGUMENTS,
BEGIN_RESULT,
END_RESULT,
LAST);

lr_xml_get_values("XML={response1}",
"ValueParam=OutputParam11",
"Query=//Body/GetWeatherResponse/GetWeatherResult",
LAST);

lr_xml_get_values("XML={OutputParam11}",
"ValueParam=OutputParam111",
"Query=//CurrentWeather/Temperature",
LAST);

lr_output_message("City temparature %s",lr_eval_string("{OutputParam111}"));

temparature=lr_xml_find("XML={OutputParam11}",
"Query=//CurrentWeather/Temperature",LAST);

if (temparature>0) {

lr_end_transaction("Cityname",LR_PASS);
}
else
{
lr_end_transaction("Cityname",LR_FAIL);

return 0;
}

Soak Tests (Also Known as Endurance Testing)

Soak Test

Soak testing is running a system at high levels of load for prolonged periods of time. A soak test would normally execute several times more transactions in an entire day (or night) than would be expected in a busy day, to identify any performance problems that appear after a large number of transactions have been executed.
Also, it is possible that a system may ‘stop’ working after a certain number of transactions have been processed due to memory leaks or other defects. Soak tests provide an opportunity to identify such defects, whereas load tests and stress tests may not find such problems due to their relatively short duration.

The above diagram shows activity for a certain type of site. Each login results in an average session of 12 minutes duration with and average eight business transactions per session.
A soak test would run for as long as possible, given the limitations of the testing situation. For example, weekends are often an opportune time for a soak test. Soak testing for this application would be at a level of 550 logins per hour, using typical activity for each login.
The average number of logins per day in this example is 4,384 per day, but it would only take 8 hours at 550 per hour to run an entire days activity through the system.
By Starting a 60 hour soak test on Friday evening at 6 pm (to finish at 6am Monday morning), 33,000 logins would be put through the system, representing 7½ days of activity. Only with such a test, will it be possible to observe any degradation of performance under controlled conditions.
Some typical problems identified during soak tests are listed below:
Serious memory leaks that would eventually result in a memory crisis,
Failure to close connections between tiers of a multi-tiered system under some circumstances which could stall some or all modules of the system.
Failure to close database cursors under some conditions which would eventually result in the entire system stalling.
Gradual degradation of response time of some functions as internal data-structures become less efficient during a long test.
Apart from monitoring response time, it is also important to measure CPU usage and available memory. If a server process needs to be available for the application to operate, it is often worthwhile to record it's memory usage at the start and end of a soak test. It is also important to monitor internal memory usages of facilities such as Java Virtual Machines, if applicable.

Long Session Soak Testing

When an application is used for long periods of time each day, the above approach should be modified, because the soak test driver is not Logins and transactions per day, but transactions per active user for each user each day.
This type of situation occurs in internal systems, such as ERP and CRM systems, where users login and stay logged in for many hours, executing a number of business transactions during that time. A soak test for such a system should emulate multiple days of activity in a compacted time-frame rather than just pump multiple days worth of transactions through the system.
Long session soak tests should run with realistic user concurrency, but the focus should be on the number of transactions processed. VUGen scripts used in long session soak testing may need to be more sophisticated than short session scripts, as they must be capable of running a long series of business transactions over a prolonged period of time.

Test Duration

The duration of most soak tests is often determined by the available time in the test lab. There are many applications, however, that require extremely long soak tests. Any application that must run, uninterrupted for extended periods of time, may need a soak test to cover all of the activity for a period of time that is agreed to by the stakeholders, such as a month. Most systems have a regular maintenance window, and the time between such windows is usually a key driver for determining the scope of a soak test.
A classic example of a system that requires extensive soak testing is an air traffic control system. A soak test for such a system may have a multi-week or even multi-month duration.

What is Failover Tests?

Failover Tests verify of redundancy mechanisms while the system is under load. This is in contrast to Load tests which are conducted under anticipated load with no component failure during the course of a test.

For example, in a web environment, failover testing determines what will happen if multiple web servers are being used under peak anticipated load, and one of them dies.

Does the load balancer react quickly enough?

Can the other web servers handle the sudden dumping of extra load?

Failover testing allows technicians to address problems in advance, in the comfort of a testing situation, rather than in the heat of a production outage. It also provides a baseline of failover capability so that a 'sick' server can be shutdown with confidence, in the knowledge that the remaining infrastructure will cope with the surge of failover load.

Explanatory Diagrams:

The following is a configuration where fail over testing would be required.

This is just one of many failover configurations. Some failover configurations can be quite complex, especially when there are redundant sites as well as redundant equipment and communications lines.

In this type of configuration, when one of the application servers goes down, then the two web servers that were configured to communicate with the failed application server can not take load from the load balancer, and all of the load must be p

assed to the remaining two web servers. See diagram below:

When such a failover event occurs, the web servers are under substantial stress, as they need to quickly accommodate the failed over load, which probably will result in doubling the number of HTTP connections as well as application server connections in a very short amount of time. The remaining application server will also be subjected to severe increase in load and the overheads associated with catering for the increased load.

It is crucial to the design of any meaningful failover testing that the failover design is understood, so that the implications of a failover event, while under load can, be scrutinized.

Fail-back Testing:

After verifying that a system can sustain a component outage, it is also important to verify that when the component is back up, that it is available to take load again, and that it can sustain the influx of activity when it comes back online.

Deatails On Load Test

Load Tests

Load Tests are end to end performance tests under anticipated production load. The objective such tests are to determine the response times for various time critical transactions and business processes and ensure that they are within documented expectations (or Service Level Agreements - SLAs). Load tests also measures the capability of an application to function correctly under load, by measuring transaction pass/fail/error rates. An important variation of the load test is the Network Sensitivity Test, which incorporates WAN segments into a load test as most applications are deployed beyond a single LAN.
Load Tests are major tests, requiring substantial input from the business, so that anticipated activity can be accurately simulated in a test environment. If the project has a pilot in production then logs from the pilot can be used to generate ‘usage profiles’ that can be used as part of the testing process, and can even be used to ‘drive’ large portions of the Load Test.
Load testing must be executed on “today’s” production size database, and optionally with a “projected” database. If some database tables will be much larger in some months time, then Load testing should also be conducted against a projected database. It is important that such tests are repeatable, and give the same results for identical runs. They may need to be executed several times in the first year of wide scale deployment, to ensure that new releases and changes in database size do not push response times beyond prescribed SLAs.

What is the purpose of a Load Test?
The purpose of any load test should be clearly understood and documented. A load test usually fits into one of the following categories:

1. Quantification of risk. - Determine, through formal testing, the likelihood that system performance will meet the formal stated performance expectations of stakeholders, such as response time requirements under given levels of load. This is a traditional Quality Assurance (QA) type test. Note that load testing does not mitigate risk directly, but through identification and quantification of risk, presents tuning opportunities and an impetus for remediation that will mitigate risk.

2. Determination of minimum configuration. - Determine, through formal testing, the minimum configuration that will allow the system to meet the formal stated performance expectations of stakeholders - so that extraneous hardware, software and the associated cost of ownership can be minimized. This is a Business Technology Optimization (BTO) type test.

What functions or business processes should be tested?
The following table describes the criteria for determining the business functions or processes to be included in a test.
Example of Load Test Configuration for a web system



The following diagram shows how a thorough load test could be set up using LoadRunner.

The important thing to understand in executing such a load test is that the load is generated at a protocol level, by the load generators, that are running scripts developed with the VUGen tool. Transaction times derived from the VUGen scripts do not include processing time on the client PC, such as rendering (drawing parts of the screen) or execution of client side scripts such as JavaScript. The WinRunner PC(s) is utilized to measure end user experience response times. Most load tests would not employ a WinRunner PC to measure actual response times from the client perspective, but is highly recommended where complex and variable processing is performed on the desktop after data has been delivered to the client.
The LoadRunner controller is capable of displaying real-time graphs of response times as well as other measures such as CPU utilization on each of the components behind the firewall. Internal measures from products such as Oracle, WebSphere are also available for monitoring during test execution.
After completion of a test, the Analysis engine can generate a number of graphs and correlations to help locate any performance bottlenecks.

Simplified Load Test Configuration for a web system




In this simplified load test, the controller communicates directly to a load generator that can communicate directly to the load balancer. No WinRunner PC is utilized to measure actual user experience. The collection of statistics from various components is simplified as there is no firewall between the controller and the web components being measured.

Reporting on Response Time at various levels of load.
Expected output from a load test often includes a series of response time measures at various levels of load, eg 500 users, 750 users and 1,000 users. It is important when determining the response time at any particular level of load, that the system has run in a stable manner for a significant amount of time before taking measurements.
For example, a ramp-up to 500 users may take ten minutes, but another ten minutes may be required to let the system activity stabilize. Taking measurements over the next ten minutes would then give a meaningful result. The next measurement can be taken after ramping up to the next level and waiting a further ten minutes for stabilization and ten minutes for the measurement period and so on for each level of load requiring detailed response time measures.

What is Load Tests

Load Tests are end to end performance tests under anticipated production load. The primary objective of this test is to determine the response times for various time critical transactions and business processes and that they are within documented expectations (or Service Level Agreements - SLAs). The test also measures the capability of the application to function correctly under load, by measuring transaction pass/fail/error rates. This test is one of the most fundamental load and performance tests and needs to be well understood.
This is a major test, requiring substantial input from the business, so that anticipated activity can be accurately simulated in a test situation. If the project has a pilot in production then logs from the pilot can be used to generate ‘usage profiles’ that can be used as part of the testing process, and can even be used to ‘drive’ large portions of the Load Test.
Load testing must be executed on “today’s” production size database, and optionally with a “projected” database. If some database tables will be much larger in some months time, then Load testing should also be conducted against a projected database. It is important that such tests are repeatable as they may need to be executed several times in the first year of wide scale deployment, to ensure that new releases and changes in database size do not push response times beyond prescribed SLAs.

Web service testing usng Load runner

What is web Service Testing?

Any service that provided over the web is called as a web service.

A Web service is a set of related application functions that can be programmatically invoked over the Internet.

Businesses can dynamically mix and match Web services to perform complex transactions with minimal programming.

Web services allow buyers and sellers all over the world to discover each other, connect dynamically, and execute transactions in real time with minimal human interaction.

OR

A service-oriented architecture (SOA) is the underlying structure supporting communications between services. SOA defines how two computing entities, such as programs, interact in such a way as to enable one entity to perform a unit of work on behalf of another entity.

Web Service components

Requirement Gathering

Table Of Content

1. Introduction

1.1 Technical Details

1.2 Load Expectations/User Details

1.3 General Details

1.4 Details of performance Testing.

2 Point of Contact /

1. Introduction

The Quality of Service attributes like performance, availability, reliability and supportability are becoming an increasingly important measure of project success, gathering performance

Requirements are very important.

The following points are considered while Gathering the Performance Requirements.

1.1 Technical Details

1. Application / System Accessibility Method (Intranet Based / Internet Based).

2. What is the GUI used in the system, please provide details about the GUI.

3. Middleware used in building the Application / System (Ex: CPM, DCOM, EJB etc.).

4. What is the Application Server used to run the Business logic?

5. What is the Web Server used to run the Application / System?

6. What are the Transaction Servers? (If any)

7. What are the Proxy Servers the system is using?

8. Provide details about streaming media Servers? (If any)

9. What is the network band width allocated for the application to work?

10. What are the ports Application will be using? Provide details.

11. Provide details about the database used by the application.

12. Is the Application / System is using any Load Balances? If yes, please provide details.

13. What type of data will be transferred from client to server and server to client please provide details?

14. Any encryption method used in the application / System? please provide details.

15. Details of Software used by the Application / System.

16. Details of Hardware on which the Application / System is running.

1.2 Load Expectations/User Details

1. What is the present load on the Application / System, total No. of users using the Application / System?

2. How many concurrent users will be using the Application / System?

3. How many peak users will be using Application / System?

4. How often the peaks are occurring and when it is occurring? Provide details about the peak loads.

5. Network connections and methods used by the users?

6. Provide details about the different types of routers used in the network setup?

7. Number of concurrent connection to test the Application?

8. List the business functions supported by the Application / System?

9. Provide the frequency of the business functions, what are the business functions used frequently by the users?

10. What are the dependencies on other systems?

11. What are the dependencies on system resources?

12. System resources are sharing with any other Application / System?

13. Provide details about the Security requirements which may effect the performance of the Application / System?

14. Any other technical information about the Application which may effect the performance of the Application / System?

1.3 General Details

Details of Performance Testing required

1. What type of performance testing is required?

ü Load Testing

The application is tested against heavy loads such as testing of web sites in order to find out at what point the web-site/application fails or at what point its performance degrades.

ü Stress Testing

System is stressed beyond its specifications to check how and when it fails. Performed under heavy load like putting large number beyond storage capacity, complex database queries, continuous input to system or database load.

ü Volume Testing

Volume testing is done against the efficiency of the application. Huge amount of data is processed through the application in order to check the extreme limitations of the system.

ü Endurance test

To load the application continuously for certain period to determine resource problems

ü Others (Please specify)

2. Expected duration for test to complete?

3. What are deliverables expected?

4. Is Performance testing currently being done for this system?

ü Yes

ü NO

If yes, what type of testing is done? Used any automation tools? Does the test plan, Test Scripts exists? Please provide details about the acceptance, Suspension and resumption criteria.

ü Is test data available? YES NO

ü Test Script / Scenario available? YES NO

ü Please attach respective documents

ü Baseline Performance measurements available? YES NO

5. What tools are available to perform the testing? If available, how many licenses are available?

6. Documentation is available for the project?

ü Non Functional Requirement

ü Functional Specification

ü System Design and Architecture

ü Others documents (Please Specify)

7. What type of testing currently completed for the Application / System?

8. Is the Application / System functionality is stable? Please provide details

Details of Application / System

1. What is name of Application / System?

2. Please provide brief description about Application / System?

3. What is the architecture of the application / System?

4. What type of Application / System it is? (Ex: Business tool, utility tool etc.)

5. On what operating system this Application / System will run?

6. What is the present status of the Application / System?

ü Initial Stage (Documentation)

ü Designing Stage

ü Development

ü Production

7. This Application / System functioning mode (Front office, Middle Office, Back Office etc.)

8. List of Interfacing Application /System?

9. What is the programming language or Scripting language used to develop the Application / System?

10. Any multimedia objects used (3D Max, Flash, etc.) in the Application?

11. Streaming Media Used (if any) Ex: Windows Media, Real Media etc.

12. Databases (If any please specify)

1.4 Details of performance Testing

1. What are the performance testing goals and Objectives?

2. Is Test Data available for performance testing?

3. Performance Test Scripts available for executing Test runs?

4. Performance test plan and test strategy available?

5. Performance test Scenario available?