Hybrid performance testing using Noise Generation in LoadRunner 12.02

In Real world scenario, there will be 20-30 % of idle users(may be higher/lower for some apps) and we also include them while taking the volumetrics but those idle users majorly access the website and then remain idle or closes the session by doing nothing and we call those as NOISE.

With LoadRunner 12.0 HP has provided us the option to generate NOISE along side normal testing and we shall see the purpose of that noise and how we can build a hybrid test scenario using Noise Generation option in LoadRunner. With that we have 2 approaches to web performance testing...

1. Create relatively complex tests with transactions, complicated flows, parameterization, checkpoints etc. Such tests, of course, require a complex script and can produce large amounts of interesting data.
2. Create a heavy load on the server by having the same URL accessed simultaneously by a large number of real or virtual users utilizing the power of load testing. The test typically consists of very few lines of code, and won’t be able to produce any sophisticated data. It can only determine the response times and whether the server has crashed or not. We call this “noise generation”, not out of disrespect, but to illustrate that such a test would create a massive and not quite coherent load on the server application.

Traditionally HP LoadRunner has been providing the means for enabling the first approach, so that users can test complex business flows and perform a detailed analysis of the resulting data. However, in some cases it’s enough to execute such flows for a relatively small fraction of vusers, while the rest can just generate load on the server in order to simulate stress conditions for the more “sophisticated” vusers.

HP introduced the “Noise Generation” feature in LoadRunner for precisely this reason. It lets you implement a hybrid testing model:  run large amounts of reasonably-priced vusers, which will generate a ‘primitive’ load by accessing a single URL, and alongside them, a smaller number of ‘proper’ vusers, which will gather the ‘interesting’ data.

Here are the main characteristics of the ‘noise’ vusers:

1. They’re 10 times cheaper than the regular ones! If you purchase a license for, let’s say, 100 Web vusers, you’ll be able to run 100 regular vusers (as before), or 50 regular and 500 noise ones (which you’re getting for the price of 50 regular), or 10 regular and 900 noise – any combination, as long as the amount of full-scale vusers plus the amount of the noise vusers divided by 10 doesn’t exceed the number of purchased licenses. (Note:- With trail version HP is giving license to 50 Vusers and that means 500 noise user load by default)
2. No VuGen scripting is needed in order to build a noise vuser script. Just input the number of vusers and the URL you’d like to put under stress, and LoadRunner will take care of everything else behind the scenes.
3. A noise vuser works by simply sending an HTTP GET request to the specified URL, once per test iteration.
4. Although noise vusers can run alongside a script based on any protocol, the main goal of this feature is to support Web-based protocols, such as Web HTTP, Mobile HTTP, Flex, SAP Web etc. Nevertheless, if your application under test has a Web interface of any sort, you can apply a load to it using the Noise Generation feature, even if the main testing is being done using a different technology.

 So, how does this feature work? Very simply! As mentioned above, no recording or VuGen scripting is needed. When you’re defining your testing scenario in the Controller, just press the familiar “Add group” button, and you’ll see the new “Noise Generator” option:

Select the ‘Use a Noise Generator’ checkbox, and type the URL, which will be the target of the noise generation.  Select the Load Generator machine as usual (unless you’re working in the percentage mode), the number of vusers (remember that you can use 10 of these vusers for the same price as a regular vuser) and the group name (although you don’t have to do the latter, the Controller will auto-generate it based on the URL’s domain):

Note that the “noise_” prefix is automatically added to the noise group names, in order to distinguish them.

The ‘Details’ dialog will reflect the fact that it’s a noise group we’re talking about in the script type and the ’View Script’ button will be disabled:

Since the “noise” script is rather primitive, there’s not much you can do in the way of configuring it. However, one thing is particularly important. If you want to get meaningful data from the script execution, make sure that either the “action as a transaction” or the “step as a transaction” option (which is the same thing in this case, since the script’s main action contains exactly one step) is turned on in the Run-time settings. That way you’ll be able to obtain accurate measurements at the end of the script run:

And that’s it! You’re ready to run your hybrid Web+noise test, and the ‘noise’ groups will run just like any other. 

How to calculate Virtual User "footprint"

Originally authored by Mark Tomlinson in HP.com and i felt its very useful for everyone.

One of the most common questions we get about LoadRunner Virtual Users relates to the resources required to execute the scripts on the Load Generator.  One advantage of the maturity of LoadRunner is that we have supported so many different drivers and protocols and environments over the past 2 decades.  We've learned so much about how to give a more detailed response to really advise users on how much Load Generator resources will be required to be successful.  You might imagine that the answer isn't black & white or even close to a 1 sentence answer.  Here are some simple ideas that can help you determine how to configure your Load Generators.

For Memory: each protocol has different parts that affect how much memory is required, so there is no single answer across all virtual users - Web is different RDP, which is different from SAP Click & Script, which is different from RMI-Java.  Some vuser types have external drivers (like ICA, RDP or SAP) so the guidelines didn’t include the footprint for the external executable driver. The Click & Script vuser types really can confuse you, because these seem like new versions of old protocols...but that's not actually true - the C&S protocols are completely new architecture, but more than anything, every vuser’s memory foot print is GREATLY impacted by the following factors:

• the size of the driver library (this is fairly static)
• the # of lines of code included in the recording (varies greatly by customer)
• the # and size of parameters included in the recording (varies greatly by customer and script type)

For CPU: of course, each driver has slight differences in CPU overhead, but for the most part they are all very efficient (and - yes, we will continue to improve Click & Script to be better!!) the amount of CPU used on a LoadRunner load generator will vary by the following factors:

• iteration and pacing, which controls how “active” the vuser is (varies greatly by customer)
• stress testing scenarios usually use more CPU, as opposed to real-world testing which has slower vusers (but more of them)
• customized code or extra script processing (like string manipulation, or calculations) will chew up more CPU

For Disk: the main thing here is logging, the more customers increase the details and amount of logging the more disk will be consumed external parameter files (writing or reading from individual vuser threads) will really hammer local disk some vusers with external driver executables will have additional logging of their own, or caching of content.

For Network: the result of multiple virtual users running on single load generator is a concentration of all those vusers network traffic on single NIC the payload of network api calls varies greatly for each and every different application stress testing (e.g. fast iteration pacing, no think-times) could easily result in over-utilization of NIC bandwidth

When it comes to calculating your virtual user footprint, it's actually quite easy.  But first, let me tell you that not everyone should need to do extensive calculations of the virtual user resource utilization.  This is important *only* when you have a very high number of virtual users or a very limited number of Load Generators.  The basic approach is to run a preliminary test with just 1 script, while you measure the resource utilization on the Load Generator directly.  You are specifically interested in the mmdrv.exe process on the Load Generator, which is LoadRunner's primary multi-threaded driver.  Measuring the private bytes reserved by this process for 1, 2, 3, 4 then 5 then 10 virtual users will give you some clue as to how much memory is required by each additional virtual user added.  Simultaneously you should monitor CPU, Network and Disk just to determine if there are any excessive utilizations.

It is important to note that you should be gathering information about the performance of your script running on the Load Generator - using the same run-time settings that you will use during the full scenario run.  If you are stress testing with very little think time or delay, then you'll want to use those same settings.

Source:- http://h30499.www3.hp.com/t5/HP-LoadRunner-and-Performance/How-To-Understand-and-calculate-Virtual-User-quot-footprint-quot/ba-p/2407591#.VNPsedLF_ng

How to correlate a value with dynamic boundaries in VuGen

If you are checking this post, seems the left and right boundaries are killing your time. Lets check how to deals with those dynamic boundaries using the text flags of VuGen.


You can use one of the following flags after the text, preceded by a forward slash:
/IC to ignore the case.
/DIG to interpret the pound sign (#) as a wildcard for a single digit.
/ALNUM to interpret the caret sign (^) as a wildcard for a single US-ASCII alphanumeric character. There are three syntaxes: ALNUMIC to ignore case, ALNUMLC to match only lower case, and ALNUMUC to match only upper case.

eg: email\u003d\"notification+mkuw__3d@facebookmail.com
In the above server response if you have to capture the highlighted part(in RED), the left and right boundaries have some alphanumeric chars and if they are dynamic then you have to follow the below approach.

web_reg_save_param("account_notify",
        "LB/DIG=email\u00#d\\"",
        "RB/DIG=__#d@facebookmail.com",
        "Ord=1",
        "Search=Body",
        LAST);

and for instance lets say that "u003d" in the left boundary itself is dynamic, then below will be our approach.

web_reg_save_param("account_notify",
        "LB/ALNUMIC=email\^^^^^\\"",
        "RB/DIG=__#d@facebookmail.com",
        "Ord=1",
        "Search=Body",
        LAST);

ALNUMIC means the boundary is an alphanumeric and IC asks it to ignore the case(upper or lower)