Code coverage with Siesta 2.0

Siesta 2.0 was released last week during SenchaCon 2013 in Florida. The main features of this release are the brand new UI (based on Ext JS 4.2) and the ability to generate code coverage reports. Code coverage is a big and exciting addition that was frequently requested in the past year, and we already use it ourselves internally. Since we think there might a few additional features left to be implemented to further improve the coverage support, we’ve marked the coverage feature as “experimental” for another couple of releases. We reserve the right to slightly change the API if required, strictly based on your feedback. Please give it a try and let us know what you think, and how we can improve it even further!

What is code coverage?

Code coverage is a metric, measuring what parts of your codebase are exercised when running your test suite. In other words it highlights code that has never been executed by the test suite and therefore should be treated as unsafe and a potential source of bugs.

At the same time, code coverage by itself does not guarantee that your code actually works. “Covered” code can still produce wrong results, such as logical or algorithmic errors. The only real assumption you can make about covered code is that it won’t throw unexpected exceptions. The main point of interest is actually the absense of coverage. That is a much more valuable piece of information, as it tells you where you should focus your testing efforts and write additional unit tests. If a core class in your model layer is not exercised by any test, it should be a clear call to action!

How does it work?

To be able to collect code coverage data, your codebase first has to be instrumented. This means it’s transformed, and additional code statements are injected that count which statements are actually executed. Siesta is using the awesome Istanbul library for this purpose.

For example, if your file contains only one simple assignment:

var a = 1;

After being instrumented, it will look like this:

if (typeof __coverage__ === 'undefined') { __coverage__ = {}; }
if (!__coverage__['filename.js']) {
__coverage__['filename.js'] = {"path":"filename.js","s":{"1":0},"b":{},"f":{},"fnMap":{},"statementMap":{"1":{"start":{"line":1,"column":0},"end":{"line":1,"column":10}}},"branchMap":{}};
var __cov_1 = __coverage__['filename.js'];

__cov_1.s['1']++;var a=1;

As you can see the amount of code increases and naturally instrumented code will take longer to execute. But don’t worry, the instrumentation is unobtrusive and does not change the semantic of your code.

Enabling code coverage in Siesta

To enable the code coverage module, simply set the enableCodeCoverage : true config option in your harness:

    enableCodeCoverage      : true

Additionally, you need to include the “siesta-coverage-all.js” file on your harness page after the regular Siesta JS file:

<script type="text/javascript" src="../siesta-coverage-all.js"></script>

You can find a sample setup in the “/examples/code_coverage.html” file inside the Siesta package.

Siesta will instrument only files marked explicitly in the `preload` config. To tell Siesta to instrument a JS file, provide a instrument : true config in the file’s preload descriptor:

        preload         : [
                type        : 'js',  // optional if file ends with ".js"
                url         : 'some_file.js',
                instrument  : true

Additionally when using the Ext JS testing layer, Siesta will install an on-the-fly instrumentation hook into the `Ext.Loader` code.

Coverage per class

Normally code coverage information is collected on a per-file basis. However, in web development it’s common to concatenate several JavaScript files (each containing one class) into one big bundle file. Getting file based coverage for such a “bundle” file will not be very useful, as it can be huge and if you want to check the coverage for one specific class in the bundle you will have to search and scroll a lot.

For the Ext JS testing layer, Siesta can collect coverage information on a per class basis. In this mode Siesta looks for Ext.define() calls in the source files, and instruments only the detected classes (which is normally what you want). The resulting report in this mode will contain information about Ext JS classes, not files. As you can see in the image below, when clicking a covered class you can immediately see any source code missing coverage (highlighted in red).

Reports and Continuous Integration

Code coverage metrics should naturally be part of your Continuous Integration process. You can easily generate code coverage reports using our Phantom JS or WebDriver launcher scripts. To generate a coverage report, simply add --coverage-report-format when launching your test suite. Recognized values are `html`, `lcov` or `raw`.

  • “html” generates coverage data in the form of a plain web page (as seen in the screen shots).
  • “lcov” is a format which can be consumed by the lcov tool
  • “raw” is a format that can be consumed by Siesta itself when several reports need to be combined.


Code coverage information is an important metric measuring one aspect of your test suite health. It indicates clearly where additional unit tests are required. The core classes of your application should definitely have a high level of code coverage.

We’re hoping Siesta will make it really easy for you to gather coverage information. Please give the Siesta 2.0 release a try and let us know how it works for you!

Leave a Reply

2 Comment threads
0 Thread replies
Most reacted comment
Hottest comment thread
2 Comment authors
casual plus size dressesair jordan shoes Recent comment authors
newest oldest most voted
Notify of
air jordan shoes

I like the valuable info you provide in your articles.

I’ll bookmark your blog and check again here frequently.
I’m quite sure I will learn a lot of new stuff right here!
Good luck for the next!

casual plus size dresses

Not all teenagers are that rich enough to spend money on apparels in order
to catch up with these trends. This year I had one of
the most glamorous experiences in my life – I attended Toronto
Fashion Week. Have you ever had to turn down an invitation because you literally had
“nothing to wear.

Stop by my weblog :: casual plus size dresses