Quick start generator doesn't seem to work for karma + mocha

[jeznag]

Maybe I'm doing something wrong but these instructions don't seem to work:

Output:

[2017-03-15 14:29:32.963] [WARN] TestFrameworkOrchestrator - Could not find test framework `mocha`. Did you forget to load a plugin? Known test frameworks: [].
[2017-03-15 14:29:32.964] [FATAL] Stryker - Configured coverage analysis "perTest" requires there to be a testFramework configured. Either configure a testFramework or set coverageAnalysis to "all" or "off".

It seems to work with mocha-runner installed. The quick start instructions should probably include stryker-mocha-runner as well as stryker-karma-runner.

I'd say this line is responsible: https://github.com/stryker-mutator/stryker-mutator.github.io/blob/master/javascripts/main.js#L50

[simondel]

Nice find! I'll fix it

[nicojs]

This won't work. The problem is that the TestFramework implementation for mocha will only work for the stryker-mocha-runner. This is because the mocha test framework at this moment does not have a way to filter out individual tests. I've hacked it in using an ugly hack which only works for the (nodejs) mocha test runner. Maybe we should even remove this feature, as this is clearly confusing others.

I've opened an issue there with a long history: mochajs/mocha#2605

Basically we're at a dead end. The way mocha is set up prevents it from easily filtering tests during runtime. If it is added, it is most likely added in the test runner part of mocha and not in the test framework part. This is important because we've designed our interfaces and apis around a fact that the TestFramework is responsible for the filtering of the tests.

If you have some time, please read through that issue, see if you can think of a way around this issue. Otherwise maybe its an idea to fork their repo and just add filtering of individual tests the way we want to.

[jeznag]

Karma runner + mocha framework does actually work for me (though presumably without filtering tests): http://github.com/COzero/nvd3/blob/2c62c1e28e7df00bcbaebfbb0d93ead8827f30a5/stryker.conf.js#L13

It did have this regrettable side effect but that was probably my fault for being too aggressive with concurrent runners:

[simondel]

This has been fixed and it works using the stryker cli :)