CoverallsTruffle: Language Agnostic Coverage Tracking for Truffle

Truffle is an open-source framework for the implementation of high-performance language runtimes using Java and the Java Virtual Machine (JVM). Coveralls.io is a web service to help you track code coverage, for instance to ensure that new code is covered by tests.

CoverallsTruffle uses the Truffle Instrumentation framework to collect coverage information, i.e., executed and not executed lines, and send it to Coveralls.io to evaluate the coverage.

How To Use

To use Coveralls.io, you'll need an account and register the relevant project repositories. Coveralls.io does not store source code itself, but accesses for instance GitHub or BitBucket.

To report results, Coveralls.io assigns each project a repo_token that needs to be passed to CoverallsTruffle.

CoverallsTruffle can be used similar to other Truffle instruments by enabling it when configuring the PolyglotEngine. It needs to be enabled and configure with the custom repository token and optionally a service name. For details see the Coveralls documentation.

PolyglotEngine engine = PolyglotEngine.newBuilder().build();
Instrument covInst = engine.getInstruments().get(Coverage.ID);
covInst.setEnabled(true); // enable the instrument

// configure coverage instrument
Coverage coverage = engineInst.lookup(Coverage.class);
coverage.setRepoToken("YOUR_TOKEN");  // Needs to be updated
coverage.setServiceName("travis-ci"); // Set the service_name, see Coveralls documentation

// start execution as usual.

For further examples, see Tests, which uses the SimpleLanguage to check that the coverage is determined correctly, as well as the coverage support for SOMns introduced with commit dc381d. An example coverage report for the SOMns core library is https://coveralls.io/builds/6016050.

Implementation Overview

The main classes are:

Coverage:

• onCreate(.) registers instrumentation for StatementTag.
• onDispose(.) processes the data from the instrumentation to determine the executed lines of code from the AST and sends the results as JSON to Coveralls.io

CountingNode: instruments AST nodes with the StatementTag and increments a counter each time the AST node is executed.

Counter: encapsulates the value of the counter and the source section to which is related.