Web and Load Test FAQs
Q: Where can I find more information about Web and Load Testing?
A:Links to info on web and load tests
Q: How can I record special requests that use ActiveX controls or client-side JavaScript?
A: Under certain conditions, the Web test recorder cannot capture requests that use ActiveX controls/client-side JavaScript, or pop-up window requests. A proxy recorder like Fiddler (http://www.fiddler2.com/fiddler2/) can be used to capture and export the requests to a Web test. Another option is to enable logging on the Web servers and retrieve all the requests from the Web logs. For more information, seeAbout JavaScript and ActiveX Controls in Web Tests.
Q: Shall I block dependent requests?
A: By default, when a Web test is executed, HTML responses received from the Web server are parsed and any dependent requests, such as images or style sheets, are automatically submitted. This is usually desirable because it places the most realistic load on the server.
However, if the goal of your load test is to place the maximum load on your application server as opposed to your Web server, you might wish to disable this behavior by setting the ParseDependentRequests properties on all of your Web test requests to false. If you do this, only the top level request is fetched, but not the images, style sheets, and so forth.
You can also selectively block certain dependent requests.
1)Set Parse Dependent Requests to false.
2)Manually add the dependent links back onto the page by right-clicking on the request and selecting Add Dependent Request.
Q: How can I test Web services?
A: Web services can be tested in a Web test or a unit test. To test a Web service in a Web test, readCreate a Web Service Test. To test a Web service in a unit test, read
Load Testing Web Services with Unit Tests.
Q: How is data source binding handled in Web and load tests?
A:Visual Studio Team System loads data sources at the start of a test run. Therefore, any updates to the data records in the data sources during test execution will not be picked up by the test.
To add different data sources from the Web test editor, read the blog postAdding Different Types of Data Sources to a Web Test.
To create custom data binding, read the blog postCustom Data Binding in Web Tests.
For more information, seeAbout Data Binding in Web Tests.
Q: What can extraction and validation rules do?
A: Extraction rules are useful when you have a piece of data from one page that is needed on another page. For example, extraction rules can be used to extract information from responses, such as http header, attribute value, form field, hidden field, text, and regular expression. The extracted values can then be added to the Web test context for future usage. For general information on Extraction rules, seeAbout Extraction Rules. To try the feature, seeWalkthrough: Adding Validation and Extraction Rules to a Web Test. To write a custom extraction rule for your test, readHow to: Create a Custom Extraction Rule andCustom ExtractionRule to extract form fields by index.
Validation rules are useful when you want to verify that expected data, such as http tag, attribute, request time and text, appear in the responses or perform Web functional testing. For general information on Validation rules, seeAbout Validation Rules. To try the feature, seeWalkthrough: Adding Validation and Extraction Rules to a Web Test. To write a custom validation rule, readHow to: Create a Custom Validation Rule.A sample validation rule can be found atA custom ValidationRule to catch redirects to error pages.
Q: How are view state and hidden fields tracked in a Web test?
A: Refer toWeb Test Authoring and Debugging Techniques, section “Handling View State and Other Dynamic Parameters”.
Q: Why do I get the error “ Context parameter ‘HIDDEN1._EVENTTARGET’ not found in the test context”?
A: The error can happen under the following conditions.
1. Run a Web Test through the Web Test Editor.
When a hidden field is modified by JavaScript in an OnClick event handler, it is possible that automatic hidden field binding will be incorrectly applied. This is a known bug in the release version of Visual Studio 2005.
<input name=”btnNext” type=”button” value=”Next”
Onclick=”_doPostBack(‘btnNext’,’’);” />
With ASP.NET sites, this problem most commonly occurs when a Web control calls the __doPostBack() JavaScript method to set the __EVENTTARGET hidden field as shown above. Automatic hidden field binding results in the form parameter having a value such as {{$HIDDEN1.__EVENTTARGET}}, instead of the actual value —btnNext. To correct this problem, the parameter value must be set to the value being set in JavaScript (for example,btnNext).
See Josh's explanationhere. Search for "Web Server Responds Differently During Execution than Recording". The most common reason is that during a load test, the application server redirects to an error page rather than the expected page. Some extraction rules (including hidden field extraction) will not fail the page when this happens, so the error doesn't actually occur in the test until the subsequent page tries to use the extracted value.
Q: What can Web test/request plug-ins do?
A: Web test plug-ins provide a way for you to isolate some code outside the main declarative statements in your Web test. A customized Web test plug-in allows you to call some code as the Web test is run. The Web test plug-in is run one time for every test iteration. Code in the PreWebTest handler will be called before the Web test starts issuing requests. Code in the PostWebTest handler will be called after the Web test completes. For example, the cleanup code can be inserted here.
Web request plug-ins provide a way for you to isolate code outside the main individual requests in your Web test. A customizedrequest plug-in allows you to call code as each request is run in a Web test. Code in the PreRequestHandler function will be called before a request is issued. For example, you can skip a request if certain condition is met. Code in the PostRequestHandler function can be used to retrieve information from response pages.
Because you have access to the Web test object in the plug-ins, you can manipulate the Web test context, database cursor, or stop a Web test by adding code in different event handlers. For details about how to create a custom Web test plug-in, readHow to: Create a Web Test Plug-In. For details about how to create a custom Web request plug-in, readHow to: Create a Web Test Request Plug-In.
Q: Why is my Web test aborted after five minutes when it is run standalone?
A: It is possible that you are running into the default test timeout. To change the timeout setting, follow these steps:
1)Oh the Test menu, choose Edit Run Configurations and then select the run configuration file.
2)Click Test Timeouts
3)Reset the timeout limits.
However, when your run the web test in a load test, the test timeout setting for a web test is not enforced.
Q: How can I use test context and pass data between tests?
A: To access Web test context in a coded Web test, use this.Context in C#.
To access Web test context in a rule or Web test/request plug-in, use e.WebTest.Context in C#..
To access load test context in a load test plug-in, use loadTest.Context in Initialize() in C#..
To access load test context from a Web test or a unit test,add the following code (C# ) to the TestStarting handler in a load test plug-in.
privatevoid TestStarting(object source,TestStartingEventArgs testStartingEventArgs)
{
foreach (KeyValuePair<string,object> keyValuePairin m_loadTest.Context)
{
testStartingEventArgs.TestContextProperties.Add(keyValuePair.Key,
keyValuePair.Value);
}
}
To access Web/unit test context in a load test plug-in, use thetestStartingEventArgs.TestContextProperties (C#)in the TestStarting handler.
To override the value of a Web test context parameter in a load test, add the same context parameter to the load test and reset the value.
Q: What is the difference between Connection Pool and Connection Per User?
A: Refer toAdvanced Load Testing Features of Visual Studio Team System, section “Choose the Appropriate Connection Pool Model.” For more information, seeConsiderations for Large Load Tests.
Q: How many simultaneous connections can a load test make?
A: Factors that limit the number of simultaneous connections the load test agent can make are
1)The operation system that a Web server (IIS) is running on. With Windows XP, the maximum is 10.
2)The Web server. Some versions of Web servers (IIS) set limit on the maximum number of connections.
When the number of connection the load agent tries to make exceeds the limit, HttpError 403 will occur and be logged in the Error table in the Load Test Monitor/Analyzer.
Q: How can I choose the appropriate user load in a load test?
A: Refer toAdvanced Load Testing Features of Visual Studio Team System, section “Choose an appropriate load profile.” For more information, seeConsiderations for Large Load Tests.
Q: What are “Think Time”, “Think Time Between Test Iterations”, and “Think Profile”?
A: Think times are used to simulate human behavior that causes people to wait between interactions with a Web site. Think times occur between requests in a Web test and between test iterations in a load test scenario. Think times of requests are recorded when requests are recorded. Using think times in a load test can be useful in creating more accurate load simulations. You can change whether think times are used or ignored in load tests. TheThink Profile is a setting that applies to a scenario in a load test. The setting determines whether the think times saved in the individual Web tests are used during the load test. By default, think time is turned on in a load test. For more information, seeAbout Think Times andConsiderations for Large Load Tests.
Q: How can I collect counters on a remote server in a load test?
A: You can configure the load test to collect the performance counters on the remote computers by adding mappings in the load test.
1)Right-click Run Settings and select Manage Counter Sets.
2)Add the performance counter sets to the load test which you would like to collect data for.
3)Right-click the counter set to launch the “Pick Performance Counters” dialog, and select counters for the remote server.
If a test is run remotely, then the controller service account needs to have permission to collect performance counters. If the account is not a member of the Admin or Power User group on the remote computer, then you need to add that user to the "Performance Monitor Users" group.
For more information, seeHow to: Manage Counter Sets.
Q: What are the common issues associated with counter collection?
A:The three most common issues associated with counter collection are:
1)Gaps in the graph. This indicates the controller or the target machines are too busy to collect/provide performance counters.
2)LoadTestCounterCategoryNotFoundException. The possible reasons for this are the counter category is not installed on the computer, the user account does have permission to collect the counters, or a firewall issue.
3)Could not find dependent counter needed to apply threshold rule:\\server\LoadTest
age\Avg. Page Time(_Total). This does not indicate a problem with the load test. The exception comes from one of the default threshold rules.It happens because this rule depends on the Avg. Page Time(_Total) counter which was not collected for the sample period the error occurred.
For more information, seeTroubleshooting Load Tests.
Q: How is load test execution affected by database access methods?
A:When you add data source to a Web test, you can choose from three different access options.
1) Sequential - If you set a data source to sequential, it uses the records from the data source in the order it reads them in. When it reaches to end of the records, it will loop back to the beginning and start again. It will continue to do this for the duration of the load test.
2) Random - If you select random, then a random row will be used for each test iteration. This will continue to happen for the duration of the load test.
3) Unique - If you select unique, the load test will access the data sequentially, but it will only execute 1 test iteration for each row in the data source regardless of what the test duration is set to. So if you have 20 rows, then the load test will execute 20 tests and then stop.
For more information, seeAbout Data Binding in Web Tests.
Q: How can I control caching in a load test?
A: ThePercentage of New Users under Scenarioproperty affects the way in which the load test run time engine simulates the caching that would be performed by a Web browser. For more information,seeAdvanced Load Testing Features of Visual Studio Team System,section “Choose a value for the ‘Percentage of New Users’ property.”For more information, seeConsiderations for Large Load Tests.
Q: Why is a validation rule in a Web test not executed in a load test?
A: Execution of validation rules has impact on performance.By default the validation level of a request is set to high. When you create a load test, you can specify what level of validation rules you want to execute under the Run Settings node. By default, it is set to low, therefore only rules which have a level of low will be executed. If you set the validation level in the load test to High, then the load test will execute all rules. For more information, seeAbout Validation Rules.
Q: How can I limit the number of tests run in a load test?
A: Please read blog postUsing a load test plug-in to limit the number of tests run in a load test.
Q: How can I clean up or change the load test results repository?
A: By default, load test results are written into an instance of SQL Express. SQL Express is limited to using a maximum size of 4 GB of disk space.
If you want to delete old load test runs in the database, read the blog postDeleting Old Load Test Results. If you need a larger database, you should consider configuring the Load Test Results Store to use an instance of the full Microsoft SQL server product. Detailed information on this can be found inUsing Results Stores.
Q: How can I turn on logging in the test processes?
A: There are different processes involved depending on whether you are running your tests locally or against a rig. All have a similar pattern: you need to edit the process's config file to enable logging:
* Add the following system.diagnostics section to the appropriate config files (change the path to VSTestHost.log if you want):
<system.diagnostics>
<trace autoflush="true" indentsize="4">
<listeners>
<add name="myListener" type="System.Diagnostics.TextWriterTraceListener" initializeData="c:\VSTestHost.log" />
</listeners>
</trace>
<switches>
<!-- You must use integral values for "value". Use 0 for off, 1 for
error, 2 for warn, 3 for info, and 4 for verbose. -->
<add name="EqtTraceLevel" value="3" />
</switches>
</system.diagnostics>
change the value in this line from “no” to “yes”:
<add key="CreateTraceListener" value="yes"/>
When running locally, make these changes to
<Program Files>\Microsoft Visual Studio 8\Common7\IDE\VSTestHost.exe.config
If running on a rig, change
<Program Files>\Microsoft Visual Studio 2005 Team Test Load Agent\LoadTest\QTController.exe.config
<Program Files>\Microsoft Visual Studio 2005 Team Test Load Agent\LoadTest\QTAgentService.exe.config
<Program Files>\Microsoft Visual Studio 2005 Team Test Load Agent\LoadTest\QTAgent.exe.config
If more than one of these processes is running on one machine you'll need to give the log files unique names.
