Code Coverage for TypeScript and Other Transpiled Languages

By on September 29, 2015 11:01 am

Transpiling or compiling code has become a necessity today for JavaScript-based web development. Whether you are using TypeScript, Babel, Dart, Traceur, or CoffeeScript to provide additional language features, or trying to optimise your code with the likes of UglifyJS, r.js, or Closure Compiler, once you have modified your source code, you start to run into challenges.

The major browser vendors have adopted Source Maps over the past few years as the de facto standard for finding source code origin. While this has made in-environment debugging easier, it still left some challenges for fully testing your code. While Intern has been able to use source maps for a while to map back the stack traces of errors that get thrown, it has been more challenging when trying to understand how well your code is covered by your tests.

Intern as well as other JavaScript testing frameworks utilise Istanbul for their code coverage analysis. As we started to adopt more and more TypeScript for our own projects, we continued to struggle with getting a clear picture of our code coverage as all the reports only included the coverage of our emitted code. We had to try to use the compilers in our minds to try to figure out where we were missing test coverage. We also like to set metrics around our coverage to let us track if we are headed the right direction.

A couple of us started exploring how we might be able to accomplish mapping the coverage report back to its origins and after a bit of work, we created remap-istanbul, a package that allows Istanbul coverage information to be mapped back to its source when there are Source Maps available. While we have been focused on TypeScript, it can be used wherever the coverage is being produced on emitted code, including the tools mentioned above!

Using remap-istanbul

So how can you use remap-istanbul? Well let’s take an example, where we are using the Intern 3 as our testing framework and Grunt as our build tool on a library of TypeScript code. From the root of our project (where your package.json sits), lets install Intern and remap-istanbul as development dependencies:

$ npm install intern remap-istanbul --save-dev

There are many different ways to use both Intern and remap-istanbul in our development chain. With both, you can use Grunt, Gulp or other tools for your task orchestration. We are going to focus on using the command line for this blog post.

At a high level, our workflow is that we need to generate or transpile our code and write Intern tests cases to test our transpiled code. That workflow contains too many options to cover here, but you need to ensure that your transpiled files are emitted with source maps. Once we have done that, we will want to run Intern to run our tests and generate the coverage information for those tests, then we will use remap-istanbul to remap the coverage information and then output an HTML report which will show the coverage for the original source files.

We need to add the Intern reporter that will write the JSON coverage information remap-istanbul needs to map back to the original source. From the command line, running our tests remotely, from the root of our project, it would look something like this:

$ node_modules/.bin/intern-runner config=tests/intern reporters=Console reporters=node_modules/remap-istanbul/lib/intern-reporters/JsonCoverage

This will instruct Intern to use the configuration to run the tests and report on the tests via the console and output a JSON coverage file which defaults to ./coverage-final.json. Once our tests complete, we will then want to take that file and remap it against our source file using remap-istanbul:

$ node_modules/.bin/remap-istanbul -i coverage-final.json -t html -o html-report

This will read in the coverage-final.json, remap all the locations by inspecting our emitted files and their source maps, and then outputting an HTML report into the path ./html-report. You can then browse to the html-report directory and see how well your code is covered based on its original source files!

The Output

Once you have everything strung together, you can start reaping the benefits. remap-istanbul is now integrated into Dojo 2’s dojo/core. When you look at the output of coverage information, you get all your information mapped back to it’s original sources. For example:

Code Coverage output for dojo/core

And if you view the HTML report that is generated, you can see actual line and branch coverage mapped back to its original source as well:

Code Coverage Report for src/request.ts

Further Reading

Your workflow might not exactly fit the pattern we described above. There are many other ways to ways integrate and use remap-istanbul. So here are some locations for some further reading:

Note that while we think that Intern is the best option for automated JavaScript testing, you can use remap-istanbul with any tool that leverages Istanbul such as Karma.

Learning more

If you’re interested in learning more about automated testing, we offer a one-day online Intern workshop, and we also help organizations with their approach to testing under our expert JavaScript support plans. If you’re not sure where to start with automated JavaScript testing, or you need some help making your source code more testable, or want assistance in defining a testing strategy and workflow for your organization, SitePen can help! Contact us for a free 30-minute consultation.

Comments

  • Pingback: | Intern test suite and CI updates | Blog | SitePen()

  • Adil shaikh

    Hi,
    I am using remap-istanbul. Is there a way to add files that are not covered in unit testin the coverage report.

  • Kitson P. Kelly

    That is a function of Istanbul and not specifically `remap-istanbul`. When you use Istanbul to instrument files, their inclusion or exclusion is determined there. In the case you point out, the `xhr.js` file was instrumented, but not covered in the test run. When it was remapped, the locations in the `coverage.json` which had no coverage mapped back to original locations in `xhr.ts`.

  • Adil shaikh

    Thanks for clearing out my understanding. So basically I’m using karma to generate code coverage with includeAllSources tag. Which serves the purpose

  • i used nyc (which wraps istanbul) + typescript, and it required no changes out of the box (with ts sourceMaps enabled). neat!

  • fwiw, our work on remap-istanbul made its way into the new Istanbul.js. It’s not a final release 1.0 yet, but our plan is for Intern 4 to hopefully also switch to Istanbul.js which includes source maps out of the box.

  • I’d like to know how you did it, did you checked coverage on typescript files or transpiled js?

  • The functionality of remap-istanbul was ported into the new version of Istanbul which nyc uses, so the coverage works against source maps to provide coverage for the TS, though the instrumentation is done on the transpiled js. So you get coverage reports based on your source.

  • Thanks, the root cause for my problems was WebStorm not picking up the tsconfig.json, after fixing that the coverage worked perfectly and I’ve learned to not entirely trust webstorm