Skip to content

Commit

Permalink
readme
Browse files Browse the repository at this point in the history
  • Loading branch information
Yaron Naveh committed Mar 9, 2013
1 parent af659c1 commit 71a8705
Show file tree
Hide file tree
Showing 7 changed files with 460 additions and 8 deletions.
71 changes: 69 additions & 2 deletions README.md
@@ -1,2 +1,69 @@
test-select
===========
## test-creep
Node.js selective tests execution. *Seamlessly* integrates with [Mocha](http://visionmedia.github.com/mocha/) (more frameworks coming soon).


## What is selective test execution?

It means running just the relevant subset of your tests instead of all of them. For example, if you have 200 tests, and 10 of them are related to some feature, then if you make a change to this feature you should run just the 10 tests and not the whole 200. test-select automatically chooses the relevant tests based on [istanbul](https://github.com/gotwarlost/istanbul) code coverage reports. All this is done for you behind the scenes and you can work normally with just Mocha.

For more information visit [my blog](http://webservices20.blogspot.com/) or [my twitter](https://twitter.com/YaronNaveh).


## Usage

1. You should use [Mocha](http://visionmedia.github.com/mocha/) in your project to run tests. You should use git as source control.

2. You need to have Mocha installed locally and run it locally rather than globally:

$> npm install mocha

$> ./node_moduels/mocha/bin/mocha ./tests

3. You need to install test-creep:

$> npm install test-creep

4. When you run mocha specify to run a special test before all other tests

$> ./node_moduels/mocha/bin/mocha ./node_modules/test-creep/first.js ./tests

first.js is bundled with test-select and monkey patchs mocha with required instrumentation.

5. It is recommended to add ./.testdeps_.json to .gitignore (more on this file below
)

## How does this work?

The first time you execute the command all tests run. first.js monkey patches mocha with istanbul code coverage and tracks the coverage per test (rather than per the whole process). Then in your project root the test dependency file is created (./.testdeps_.json):


`````javascript
{
"should alert when dividing by zero": [
"tests/calc.js",
"lib/calc.js",
"lib/exceptions.js",

],

"should multiply with negative numbers": [
"tests/negative.js",
"lib/calc.js",
],
}

`````

Next time you run the test (assuming you add first.js to the command) test-creep runs 'git status' to see which files were added/deleted/modified since last commit. Then test-creep instructs mocha to only run tests that have dependency in changed files. In the example above, if you have uncommited changes only to lib/exceptions.js, then only the first test will be executed.

At any moment you can run mocha without first.js in which case all tests and not just relevant ones will run.


## limitations
* Dependency between test and code is captured at file and not function granularity. So sometimes test-select can run more tests than actually requiered (there is no harm in that).

* test-select cannot detect changes in global contexts. For example, if you have a one time global initialization of a dictionary, and some tests use this dictionary, then test-select will not mark these tests as dirty if there is a change in the initialization code.

* If you have a test suite that runs super fast (< 2 seconds) then test-select will probably add more overhead than help. test-select sweet spot is in long running test suites, though whenever tests run for more than a couple of seconds it can save you time.

For more information visit [my blog](http://webservices20.blogspot.com/) or [my twitter](https://twitter.com/YaronNaveh).
2 changes: 1 addition & 1 deletion first.js
@@ -1 +1 @@
require("./selective")
require("./lib/selective")
1 change: 1 addition & 0 deletions lib/consts.js
@@ -0,0 +1 @@
exports.depsFile = './.testdeps_.json'
179 changes: 179 additions & 0 deletions lib/coverage.js
@@ -0,0 +1,179 @@
var path = require('path'),
fs = require('fs'),
istanbul = require('istanbul'),
hook = istanbul.hook,
Instrumenter = istanbul.Instrumenter,
Collector = istanbul.Collector,
instrumenter = new Instrumenter(),
Report = istanbul.Report,
collector,
globalAdded = false,
fileMap = {};

/**
* Facade for all coverage operations support node as well as browser cases
*
* Usage:
* ```
* //Node unit tests
* var coverage = require('/path/to/this/file');
* coverage.hookRequire(); // hooks require for instrumentation
* coverage.addInstrumentCandidate(file); // adds a file that needs to be instrumented; should be called before file is `require`d
*
* //Browser tests
* var coverage = require('/path/to/this/file');
* var instrumentedCode = coverage.instrumentFile(file); //alternatively, use `instrumentCode` if you have already loaded the code
* //collect coverage from the browser
* // this coverage will be stored as `window.__coverage__`
* // and...
* coverage.addCoverage(coverageObject); // rinse and repeat
* ```
*
* //in all cases, add an exit handler to the process
* process.once('exit', function () { coverage.writeReports(outputDir); }); //write coverage reports
*/

/**
* adds a file as a candidate for instrumentation when require is hooked
* @method addInstrumentCandidate
* @param file the file to add as an instrumentation candidate
*/
function addInstrumentCandidate(file) {
file = path.resolve(file);
fileMap[file] = true;
}
/**
* hooks require to instrument all files that have been specified as instrumentation candidates
* @method hookRequire
* @param verbose true for debug messages
*/
function hookRequire(verbose) {

var matchFn = function (file) {
/*
var match = fileMap[file],
what = match ? 'Hooking' : 'NOT hooking';
if (verbose) { console.log(what + file); }
return match;
*/
var res = file.indexOf("node_modules")==-1
var what = res ? 'Hooking ' : 'NOT hooking '
if (verbose) console.log(what + file)
return res

}, transformFn = instrumenter.instrumentSync.bind(instrumenter);

hook.hookRequire(matchFn, transformFn);
}
/**
* unhooks require hooks that have been installed
* @method unhookRequire
*/
function unhookRequire() {
hook.unhookRequire();
}


function getCollector() {
return getCollectorInternal(false)
}


/**
* returns the coverage collector, creating one if necessary and automatically
* adding the contents of the global coverage object. You can use this method
* in an exit handler to get the accumulated coverage.
*/
function getCollectorInternal(createNew) {
if (!collector || createNew) {
collector = new Collector();
}

if (globalAdded && !createNew) { return collector; }

if (global['__coverage__']) {
collector.addInternal(global['__coverage__'], true);
globalAdded = true;
} else {
console.error('No global coverage found for the node process');
}
return collector;
}
/**
* adds coverage to the collector for browser test cases
* @param coverageObject the coverage object to add
*/
function addCoverage(coverageObject) {
if (!collector) { collector = new Collector(); }
collector.add(coverageObject);
}
/**
* returns the merged coverage for the collector
*/
function getFinalCoverage() {
return getCollector().getFinalCoverage();
}

/**
* writes reports for an array of JSON files representing partial coverage information
* @method writeReportsFor
* @param fileList array of file names containing partial coverage objects
* @param dir the output directory for reports
*/
function writeReportsFor(fileList, dir) {
var collector = new Collector();
fileList.forEach(function (file) {
var coverage = JSON.parse(fs.readFileSync(file, 'utf8'));
collector.addCoverage(coverage);
});
writeReportsInternal(dir, collector);
}

/**
* writes reports for everything accumulated by the collector
* @method writeReports
* @param dir the output directory for reports
*/
function writeReports(dir) {
writeReportsInternal(dir, getCollector());
}

function writeReportsInternal(dir, collector) {
dir = dir || process.cwd();
var reports = [
Report.create('lcov', { dir: dir }),
Report.create('text'),
Report.create('text-summary')
];
reports.forEach(function (report) { report.writeReport(collector, true); })
}
/**
* returns the instrumented version of the code specified
* @param {String} code the code to instrument
* @param {String} file the file from which the code was load
* @return {String} the instrumented version of the code in the file
*/
function instrumentCode(code, filename) {
filename = path.resolve(filename);
return instrumenter.instrumentSync(code, filename);
}
/**
* returns the instrumented version of the code present in the specified file
* @param file the file to load
* @return {String} the instrumented version of the code in the file
*/
function instrumentFile(file) {
filename = path.resolve(file);
return instrumentCode(fs.readFileSync(file, 'utf8'), file);
}

module.exports = {
addInstrumentCandidate: addInstrumentCandidate,
hookRequire: hookRequire,
unhookRequire: unhookRequire,
instrumentCode: instrumentCode,
instrumentFile: instrumentFile,
addCoverage: addCoverage,
writeReports: writeReports,
getCollectorInternal: getCollectorInternal
};

0 comments on commit 71a8705

Please sign in to comment.