IP Spoofing in Performance Testing

Do I need IP spoofing? - Yes, if your system uses IP-dependent load balancing

Scenario without IP Spoofing 

                                    The above picture shows how the System Under Test (SUT) is being loaded if we test the application without IP Spoofing. Reason being that all the network elements and the servers have cache and they consider clients IP as the means to route the requests to different tiers, so if we don't have IP Spoofing a LoadGenerator can have only 1 IP and so the path traced by 1 Vuser and the other is the same as seen above as Routers and the Load Balancers will treat that the same user is coming with  a new request.

                                           Scenario with IP Spoofing 

      Performance testing targets are only met if we do the testing the same way as the system is loaded in the real time (same way as shown in the above snapshot) . Every performance test should aim at loading the SUT as shown above and thus highlight the genuine performance bottlenecks.

How do I implement IP spoofing?

1. Run the IP Wizard on each Vuser host machine (Load Generator)

2. Enable the IP Spoofer from the Controller's main menu.

Rendezvous in LoadRunner

                          You all know the purpose of doing performance testing, but we have some features available with LoadRunner like Rendezvous and IP Spoofing which takes us more close to a real world situations and thus makes our test more valid than ever.
                          In the current topic we shall discuss about how Rendezvous helps us in improving and taking us close to those real world conditions. Lets take an instance of a shopping website which obviously includes some 3rd party sites during checkout eg: Paypal or Google checkout or Credit/Debit card payment etc. During the festival season we expect huge requests from the customers where the site might experience the problems and more importantly issues during the checkout which is major drawback for the shopping site as it decreases its revenue and the customer confidence.
                          So our next step would be to replicate the real world situation for the checkout functionality and to do that, LoadRunner has provided a function called lr_rendezvous() which can do the job for us

In the following example taken from LoadRunner tool doc's, the lr_rendezvous function sets the Meeting rendezvous point. When all users that belong to the Meeting rendezvous arrive at the rendezvous point, they perform do_transaction simultaneously. 

lr_rendezvous("Meeting");
    do_transaction(); /* application dependent transaction */  

                        The lr_rendezvous function creates a rendezvous point in a Vuser script. When this statement is executed, the Vuser program stops and waits for permission from LoadRunner to continue.

This mean that when we are doing the Load test every Vuser will be at different transaction (Vuser 1 is at 3rd transaction and Vuser 2 at 5th transaction etc) and to test business critical transactions (say 6th transaction) then we have to place Rendezvous after 5th transactions so that Vusers wait for all others Vusers to come to that complete 5th transaction and start 6th transaction simultaneously. Thus we can bring out the bottlenecks of that business transaction and could less the performance issues in the production environment. 

Note:- This function can only be used in action section, and not in vuser_init or vuser_end. 

Recording a script in VuGen using SAFARI web browser

          We all know that Internet Explorer and Mozilla Firefox are compatible with VuGen for recording Web applications, but only few knew how to record web scripts through Safari browser. Lets see how it can done.
                   
         Similar to the settings we do in VuGen to record web scripts using FireFox we also need similar recording settings for Safari. As seen below we select Application type as Win32 Applications and give the exe file path of Safari in the Program to record entry.

         Once these basic settings are done, you are good to go. Enjoy recording

         If you have any constraint of using specific User Agent (iPad, iPhone etc) for recording which are listed below as Safari can support all these user agents and there might be some applications which are compatible(build for) with specific user agents, VuGen still supports it.

       

      In the recording settings of VuGen, you update the Headers settings under Advanced as shown below

    Once this is done, you can start recording which opens Safari browser where you select the user agent and continue recording the functionality of any application that runs on any of the said user agents.

Agent less performance monitoring using SNMP

                We have many performance monitoring tools in the market from freeware to licensed. All those tools can be divided as Agent less and Agent based tools.
               Agent based tools are very intuitive in terms of performance monitoring but costly to implement. CA Wily Introscope is the best example for Agent based monitoring. Agents/Probes play a major part in collecting the performance metrics from the systems under test, we have to manually build them but are very intuitive as they collect the metrics from the lowest level possible and it depends on how its configured. This type of monitoring is very useful as you can track each and every request from a module/function/component where the agents are placed. The negative part of it is these configured agents will be an overhead to the system, more are the agents more are the metrics configured and more is the overhead on the system.
               Most of the agent less monitoring tools used SNMP(Simple Network Management Protocol) to track all the performance metrics for the applications. All SNMP compliance systems can be monitored using SNMP. To monitor a system, we have to activate the SNMP service on that system and set all performance metrics to be monitored and the frequency at which it has to monitor. Once these basic settings are done you are ready to go. HP SiteScope is an example of agent less successful monitoring tool.

In brief:-
Agent based monitoring
Pros:-  more intuitive, monitoring at module/function/component level, more metrics can be configured.
Cons:- skill set to configure all agents, overhead on the system.

Agent less monitoring
Pros:- fast to implement, no skill set required, all high level metrics can be monitored.
Cons:- used more network, cannot be customized. metrics are limited

4 Risks in Cloud Implementaion - Gartner

Although cloud offerings are rapidly maturing, the immaturity of cloud service contracting means that many contracts have structural deficits, according to Gartner Inc. Gartner has identified four risky issues that CIOs and sourcing executives should be aware of when contracting for cloud services.

"Cloud service providers will need to address these structural shortcomings to achieve wider acceptance of their standard contracts and to benefit from the economies of scale that come with that acceptance," said Frank Ridder, research vice president at Gartner. "CIOs and sourcing executives have a duty to understand key areas of risk for their organizations.

"It's essential that organizations planning to contract for cloud services do a deep risk analysis on the impact and probability of their risks, and they should also plan mitigation for the most critical issues," said Alexa Bona, research vice president at Gartner. "This might cost additional money, but it is worth the effort. Risk should be continuously evaluated, because contracts can change -- sometimes without notification."

The four risky issues for CIOs, when contracting for cloud services include:

Cloud sourcing contracts are not mature for all markets:-

When analyzing cloud sourcing contracts, it is often obvious whether the cloud service provider wrote the contract with larger, more mature corporations, or the consumer side of the market, in mind.

For example, there are cloud service contracts from traditional service providers for their private cloud offerings; these tend to include more generally acceptable terms and conditions.

Gartner also sees many cloud-sourcing contracts that lack descriptions of cloud service providers' responsibilities and do not meet the general legal, regulatory and commercial contracting requirements of most enterprise organisations.

Organizations are advised to carefully assess the risks associated with cloud sourcing contracts. Areas such as data-handling policies and procedures can have a negative impact on the business case (for example, additional backup procedures or a fee for data access after cancellation) potentially creating compliancy issues and cost increases, and indicating specific risk mitigation activities.

Contract terms generally favor the vendor:-

Organisations that successfully outsource, evolve more partnership-style relationships with their vendors. Cloud service contracts do not lend themselves to such partnerships — mainly because of the high degree of contract standardisation — where terms are consistent for every customer, and service is typically delivered remotely rather than locally.

An organization needs to understand that it is one of many customers and that customization breaks the model of industrialized service delivery. Cloud service contracts are currently written in very standardized terms, and buying organizations need to be clear about what they can accept and what is negotiable. To manage cloud services contracts successfully, organizations need to manage user expectations.

Contracts are opaque and easily changed:-

Contracts from cloud service providers are not long documents. Certain clauses are not very detailed, as URL links to Web pages detail additional terms and conditions.

These details are often critical to the quality of service and the price (such as SLAs) for uptime or performance, service and support terms, and even the description of the core functionality of the offering.

Clauses that are only fully documented on these Web pages can change over time; often without any prior notice.

Organizations need to ensure that they understand the complete structure of their cloud sourcing contract, including the terms that are detailed outside of the main contract. They need to be sure that these terms cannot change for the period of the contract and, ideally, for at least the first renewal term without forewarning.

It is also critical to understand what parts of the contracts can be changed and when the change will take place.

Contracts do not have clear service commitments:-

As the cloud services market matures, increasing numbers of cloud service providers include SLAs in URL documents referenced in their contracts and, in fewer cases, in the contract itself. Usually, the cloud service providers limit their area of responsibility to what is in their own network as they cannot control the public network.

Things are improving, but service commitments remain vague.

When deciding whether to invest in cloud offerings, buyers should understand what they can do, if the service fails or performs badly. They should understand whether the SLAs are acceptable and if the credit mechanisms will lead to a change in the providers' behavior; if not, they should negotiate terms that meet their requirements — or not engage.

Wasted Time in LoadRunner

Wasted Time:-
If your guess is Replay log, Yes… We get this in the Replay Log of VuGen Script as shown below.
Action.c(21): Notify: Transaction "Home" ended with "Pass" status (Duration: 20.1061 Wasted Time: 0.0010)


Here is what HP says about Wasted Time.


Wasted time is time spent on activities whose purpose is to support test analysis, but would never be performed by a browser user, for example, time spent keeping transaction statistics for later reporting. Wasted time is calculated internally by LoadRunner. Your script can also add wasted time with lr_wasted_time.


Sometimes, you may enter activities in a script that your do not want reported as part of the transaction statistics. Generally, these are activities related to record keeping, logging, or custom analysis. If you enhance the script with steps whose durations should not be included in the test statistics, you can track the time used by these steps with lr_start_timer and lr_end_timer. Then, the function lr_wasted_time is used for adding this user-determined time to the internally generated wasted time.


You can retrieve the total wasted time (both that generated by LoadRunner automatically and that added with lr_wasted_time) with the function lr_get_transaction_wasted_time, or with lr_get_trans_instance_wasted_time, as appropriate.

When VuGen creates the log file, output.txt, and when on line testing generates the Vuser log files, no action is taken with the wasted time. The actual elapsed transaction times are reported. The lr_get_transaction_duration function also returns the actual elapsed transaction time. This information may be useful in analyzing and developing test scripts.


However, in the on-line graphs in the LoadRunner Controller and the transaction response time graphs in the LoadRunner Analysis, the transaction times are reported after subtracting the wasted time. This is the time most pertinent to understanding the system being tested.

HP LoadRunner 11.0 Features

HP LoadRunner for the Windows operating system Software version: 11.00
Publication date: October 2010
What's New
Protocols
Ajax TruClient - An advanced protocol for modern JavaScript based applications (including Ajax) emulating user activity within a web browser. Scripts are developed interactively in Mozilla Firefox.
Silverlight - A new protocol for Silverlight based applications emulating user activity at the transport level. Allows generating high level scripts by automatically importing and configuring WSDL files used by the application.
Java over HTTP - A new protocol designed to record java-based applications and applets. It produces a Java language script using web functions. This protocol is distinguished from other Java protocols in that it can record and replay Java remote calls over HTTP.
Citrix
The Citrix Protocol now supports Citrix Online Plugin versions 11.2 and 12.0.
Added support for Citrix XenApp Server 5.0
Oracle NCA - NCA Java object property support now provides automated creation and registration within a script of a query-answer table of communication between client-side Java objects and the Oracle NCA server.
SAPGUI - Added support for SAPGUI for Windows Client version 7.20..
Service Test - The LoadRunner Controller can run scripts created in HP Service Test 11.00, HP's solution for creating and running automated tests for SOA and headless technologies. Refer to the Service Test documentation for details of creating Service Test scripts for a load testing scenario.

Features
Data Format Extension (DFE) - Enhanced data format capabilities for the Web (HTTP/HTML) protocol family. Allows converting raw HTTP traffic into a maintainable and structured XML format and enables correlations by XPATH.
Correlation Studio - Web (HTTP/HTML) automatic correlation mechanism has been enhanced to search for possible correlations in the larger scope of snapshot data created during code generation including data formatted by DFE.
Snapshot View - New snapshot view for Web (HTTP/HTML) protocol steps allows viewing complete HTTP traffic in both raw and DFE generated formats.
VuGen - HP ALM Integration - Enhanced integration with HP Application Lifecycle Management platform that serves also Quality Center and Performance Center editions..
Windows Support - Added support for Windows 7 and Windows Server 2008. See below for limitations. Analysis Reports - Enhanced Analysis reports are more customizable. Analysis data can be exported to a variety of formats, including Word, Excel, PDF, and HTML. New report templates allow saving report definitions and generating reports based on a template.

LIMITATIONS:-
VuGen
SAP (Click and Script) recording. During recording, if you use a keyboard option instead of a UI element (for example, pressing Enter instead of clicking the log on button), the step may not be recorded. In general, when recording your script, it is recommended to use UI elements rather than keyboard options.
Citrix snapshots. Black snapshots may appear during record or replay when using Citrix Presentation Server 4.0 and 4.5 (before Rollup Pack 3).
Possible workaround: On the Citrix server select Start Menu > Settings > Control Panel > Administrative Tools > Terminal Services Configuration > Server Settings > Licensing and change the setting Per User or Per Device to the alternative setting (ie If it is set to Per User , change it to Per Device and vice versa.)

Recording Window Size and XenApp Plugin for Hosted Applications 11. The recording window size options does not work properly with the XenApp Plugin for Hosted Applications 11. The size of the client window is installed, but the server screen resolution is not. This is a Citrix Client bug and will be fixed in future Citrix Client versions.

Workaround: When recording, set the window size equal to the local screen resolution. When replaying/load testing, set the VuGen or Load Generator's screen resolution to equal the resolution used when the script was recorded.
-->For the following protocols, replay is supported but recording is no longer supported: Siebel DB2, Siebel MSSQL, Siebel Oracle, and Voice XML.
-->In Test Result reports , you can only use the Export to HTML file utility for scripts replayed in version 9.50 and later. To generate an HTML report for scripts created with earlier versions, run the script again in the 9.50 version (or later) of the product.
-->If you terminated the Protocol Advisor immediately after the detection process started, the Protocol Advisor may leave a zombie process of the detected application, causing all recordings and detection sessions that follow to fail. Workaround: Manually terminate the zombie process.
-->The new Web snapshot model is backward compatible with previous versions of LoadRunner, however some snapshot data may be missing. If this occurs, regenerate the script.
-->The JSON Data Format Extension does not work in UNIX environments.
-->Recording a network based application on Windows 7. To record a network-based application (including Web), IPv6 communication must be disabled. Follow the instructions in http://www.addictivetips.com/windows-tips/how-to-disable-ipv6-in-windows-7/ to disable IPv6 in Windows 7.
-->For protocols using XML, replay fails to create a request when a parameterized input argument contains the ampersand (&) character.

Correlation Studio
In VuGen, when correlating large amount of snapshot data, in some cases the parameter is not created and data is not correlated.
Scan for correlation will work on a script that was regenerated and replayed with the early access version.

Note:- Please share your experiences on LR 11.0

Source:- Vietnamesetestingboard