How does code coverage works ?
Hello,
I have a web site that uses a DLL.
I have also written some unit tests. Part of them are calling directly the DLLs (and I have set up my run config so VS calculates the code coverage) and part of them are launching HTTP Web Request on the web site (and so of course using behind the same DLL).
With my second kind of test, even if the samee DLL is used, no code coverage is calculated.
So my questions:
- Is this normal ? The same code is executed so I was thinking code coverage would be calculated
- How can I set up my run config to ask VS to calculate also code coverage for that kind of tests ?
As I have read on some other posts, it may be because when I run my Web Request, my website si recompiled on the fly. But if my web pages are compiled on the fly, the DLL is not (afaik).
Thanks for your help !
Hello Jarod,
You have to do the following in order to be able to get code coverage for your second type of tests:
1. You must have the website in the same box where you have your solution
2. You have to add the website to your solution
3. You have to select your website in the artifacts to instrument from the Code Coverage tab in the testrunconfig file of your solution
After setting up these steps, you should be ready to go
Thanks,
David Gorena Elizondo
[MSFT] VSTS
Hello David
thanks for your answer ! However, I still have some problems. I will try to explain the situation here the best I can.
Here is how my solution looks like
- Solution
- DLL Project
- Test DLL
- Web Site (referencing my DLL project)
On each development machine, when we want to run our tests, we publish the website to IIS (http://localhost/web) and then run our HTTP Web Request against this address. (No atttribute specified on our methods except TestMethod). I don't care of Code Coverage at that point.
During our automated build process, I want to publish my website to IIS. So I create a new virtual directory in IIS (under folder c:\build\web) and then I copy all my sources to it (sources being the result of my build process from the folder ...\Binaries\Mixed Platforms\Release\_PublishedWebsites
I have modified the testrunconfig file to specify that I want to instrument my website (I had to manually update it). Here is the section related to my website :
<
aspNetProjectCoverageItems type="System.Collections.Generic.List`1[[Microsoft.VisualStudio.TestTools.Common.AspNetProjectItem, Microsoft.VisualStudio.QualityTools.Common, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a]]"><
_items type="Microsoft.VisualStudio.TestTools.Common.AspNetProjectItem[]"><
element type="Microsoft.VisualStudio.TestTools.Common.AspNetProjectItem"><
id type="System.Guid">0528bc1b-4775-4120-8fc4-1bc0fab13b3c</id><
name type="System.String">C:\...\web\</name><
applicationRoot type="System.String">/web</applicationRoot><
url type="System.String">C:\Builds\web\</url><
isIis type="System.Boolean">True</isIis></
element></
_items><
_size type="System.Int32">1</_size><
_version type="System.Int32">0</_version></
aspNetProjectCoverageItems>As you can see, I let the "name" tag to the original version (the name of my web site in my solution). I have modified the "application root" to give the name of the virtual directory in IIS and the "URL" to give the path of my website.
When I run my team build, I have the following error in the log file : "...doesn't map to a valid local website"
I tried also to modify "isExecutedRemotely" to "true" but I have the same error.
When I change the URL to something like http://localhost/web I have another error :
Cannot initialize post instrumentation for the ASP.NET project 'http://localhost/'.
Exception was thrown: Microsoft.VisualStudio.Enterprise.Common.AspNetHelperException: The web site could not be configured correctly; getting ASP.NET process information failed. Requesting 'http://localhost/VSEnterpriseHelper.axd' returned an error: The remote server returned an error: (404) Not Found. > System.Net.WebException: The remote server returned an error: (404) Not Found.
at System.Net.HttpWebRequest.GetResponse()
at Microsoft.VisualStudio.Enterprise.Common.BaseHelper.GetWorkerProcessInfo()
End of inner exception stack trace
at Microsoft.VisualStudio.Enterprise.Common.BaseHelper.GetWorkerProcessInfo()
at Microsoft.VisualStudio.Enterprise.Common.AspNetHelper.SetupListener()
at Microsoft.VisualStudio.Enterprise.Common.AspNetHelper.SetupInstrument(Type type)
at Microsoft.VisualStudio.TestTools.CodeCoverage.CoveragePlugIn.InitAspNetInstrumentation(TestRun testRun).
The code coverage data for this project may be incomplete.
Do you have any idea of what can be wrong ?
Thanks,
Hello Jarod,
"isExecutedRemotely" should be false, as the test execution is on the same machine.
Both <URL> and <Name> should contain the URL to your web site: http://localhost/web
If it still does not work, ensure that the web site is really configured before executing the test task as part of Team Build.
Regards,
Boris
Hello Boris !
Better like that !
an extra question now... as I instrument my website, all the DLLs included in the website are also instrumented (normal). But I use some third parties DLLs (like the Microsoft.Practices.EnterpriseLibrary.Data.dll) and these DLLs are also getting instrumented and so are "faking" my code" coverage results.
So is it possible to instrument my web site but "bypass" some specific DLLs ?
Thanks !
Hi Jarod,
I am glad that you make it work :).
There is no standard way to filter out those DLLs for websites. However, we instrument only DLLs for which we could find symbols (PDB files), assuming that those are DLLs developer/tester cares about. So if you simply remove the PDB files for the 3rd party DLLs before running the tests, we won't instrument them and they won't pollute your code coverage data.
Regards,
Boris
Hello Boris,
In fact there is no pdb files that I can remove.
To give more details, I have the Team Edition For Software Developpers so I run the tests in command line in my proj file.
So, here is what I do:
- I copy my files (coming from "Binaries\Mixed Platforms\Release\_PublishedWebsites") to a folder exposed in IIS - No pdb here
- Then I run my tests via mstest, asking to publish the results in the current build
The only place where I find some ".instr.pdb" files, it's in the folder "BuildType\TestResults\Build NUmber_Mixed Platforms_Release\Out" folder
Should I removed the files from that folder ?
Hi Jarod,
No you should not remove the .instr.pdb files from test results directory, as they are required for the code coverage analysis. However, those .instr.pdb files are generated from the original .pdb files. This means that our execution is able to find those symbol files. Sometimes compiler places the full path to the pdb file in the execution image. If this is the case, you may need to remove the PDB files from the build directory.
Also, check whether the _NT_SYMBOL_PATH or _NT_ALTERNATE_SYMBOL_PATH are set, as we may be actually downloading the symbols from internet for the system DLLs.
Regards,
Boris
two critical things I found when trying to solve this issue:
1. Web.config must not be read-only, which means that you have to check it out of source control. Launching from the performance explorer causes some modifications to your web.config file as the application is launched which registeres an http handler which handles the actual request for VSEnterpriseHelper.axd. The launcher will not error out if the web.config is read only, instead, it will just give you the 404 - Not found error.
2. Do not have a default handler registered in the http modules similar to:
<add verb="*" path="*" type="MyHandler,MyHandlerAssembly" />
Because the launcher adds the handler for VSEnterpriseHelper.axd at the end of the handler chain, the anonymous handler will intercept and return 404.
Hope this helps all you profilers out there... ;)
Thanks,
Nick.
nick(@t)chi-town.com
