Context

We've recently been on a big drive to improve code coverage on all of our code bases. Our tech stack include Angular 1.6, Angular 6, ASP.net 4 and asp.net Core. Test coverage in the JavaScript world is pretty amazing with tools like Istanbul, but we've battled to find the same for C# without paying a fortune. That is until we came across OpenCover.

Code coverage tools work by profiling the binaries while the tests are run and then recording various metrics such as lines/branches covered. See this wiki article for more details.

OpenCover is an open source .net code coverage tool built specifically for the traditional .net framework. This post is the result of my research into getting it working with .net Core as well as the resultant dotnet tool I built to assist us.

Running tests in .net Core

.net Core tests are executed using the dotnet command line tool:

dotnet test [project]

The dotnet tool ensures the relevant binaries are built and then tests are run using the vstest tool. The tool only allows one to run tests on a single project at a time, more on this later.

Running OpenCover

I'm not going to go into the details of running OpenCover (check their page, linked above, for their docs), but in general, one provides an executable to profile as well as the arguments for that executable.

So, for .net Core, we need to pass in the dotnet command line to be profiled and then build up the console arguments for our tests

OpenCover.Console.Exe -target:"c:\program files\dotnet\dotnet.exe" -targetargs:"test [Project File]"

The tests run successfully, but we get the following output indicating that no coverage was recorded:

This is because dotnet core binaries do not expose the required debug information by default. We'll need to add the following to each project file we want to profile:

<PropertyGroup>  
  <DebugType>Full</DebugType>
</PropertyGroup>  

Now when we run it, we get a different error:

It turns out that some of the APIs OpenCover uses to profile the app are missing in .net Core. Luckily, there is a work around, we can pass -oldstyle to OpenCover to use different APIs.

OpenCover.Console.Exe -target:"c:\program files\dotnet\dotnet.exe" -targetargs:"test [Project File]" -oldstyle

Success!

The results are written to a file called results.xml by default which is a bit tricky to analyze. There are some tools available on nuget that will convert the file to HTML or to a format called Cobertura which is useful for code coverage reporting in VSTS/TFS.:

• ReportGenerator
• OpenCoverToCoberturaConverter

Introducing dotnet-testx

The above commands can get out of hand quickly, especially when one starts passing in more arguments to dotnet test or to OpenCover directly. Also, when trying to improve coverage, one may want to keep generating the report after every test written.

The dotnet cli has an extension mechanism that allows one to write custom commands. In .net core 2.1, these can be installed globally (similar to npm install -g). Details here.

dotnet-testx is a dotnet tool that handles the following for you:

• Detect the project file in the current folder, or discover all project files in sub folders
• Resolve the location of OpenCover and the reporting tools in nuget's cache
• Execute OpenCover with some sensible defaults:
  • Merge results from multiple runs (1 per project) into 1
  • Output to folder named coverage so it doesn't pollute the source code
  • Pass in some filters for DLLs to be ignored in the coverage report
• Generate an HTML report using ReportGenerator
• Open the report in the browser
• Convert the report to Cobertura for your TFS/VSTS build
• Generate trx test results report for TFS/VSTS
• Used together with dotnet watch enables re-running when code changes

The tool can be installed with: dotnet tool install --global dotnet-testx

Usage:

dotnet testx

dotnet testx --html --browser

dotnet testx --cobertura

dotnet testx --discover-projects "*Tests.csproj"

dotnet watch --project [project-file] testx -- [dotnet testx arguments]  

Note: The tool is not bundled with OpenCover or any of the reporting tools. You will need to install them from nuget into your project.

The code lives here
The nuget package is here

I really hope someone else finds this useful. The tool currently works for my needs, but I'll be happy to expand it or accept PRs to extend the usage for other unique use cases.