[WIP] Mutant integration

[mbj]

Opening early to allow discussions.

Changes

• Adds mutant rake task
• Sets up mutant in "run all tests per mutation" (no test selection, fast enough for this small project)
• Set expected coverage threshold in rake task. It'll fail above or below that threshold! (Needs manual adjustment if coverage increases!)

TODO:

• The above adjustment will be pushed down to a mutant config option before PR should be merged.
• Bring back simplecov integration in spec_helper.rb and guard it in a way it does not interfere with mutant.
• Fix coveralls integration interfering with mutant on travis.

Questions:

• Hook mutant rake task on CI and make it fail if coverage is not meet? I like it that way, but some people (greez @solnic) prefer not to fail on violated metric tools.

The current mutant report looks like this: https://gist.github.com/mbj/10641431

This PR is not intended to be merged right now. I just open it early to give interested participants a context for discussion.

My intention is to identify more TODOs, and fix them before asking for the real merge.

@avdi, @sferik If you think that WIP PR is too much noise, I'll reopen it against my fork. Feel free to close it in this case.

[coveralls]

Coverage decreased (-32.48%) when pulling 4ddfe0c on mbj:mutant into 286274a on avdi:master.

[coveralls]

Coverage decreased (-32.89%) when pulling 04144a8 on mbj:mutant into 286274a on avdi:master.

[solnic]

Yeah failing build because of metrics is just noise. Slows down the development process, discourages refactoring, significantly increases the time you need to spend just to introduce simple changes and so on and so forth.

ps. you summoned me, what did you expect? 😃

[mbj]

@solnic Expected exactly what you did. I wanted to have your opinion here.

[solnic]

Yeah I think using metrics in a way that actually helps you requires a ton of experience and knowledge. Otherwise it can hurt you. Same applies to mutation testing. Actually it especially applies to mutation testing. If a library is in a flux, if it's unstable and experimental using mutation testing will pretty much waste your time (with the exception that you will learn something and that's cool but the code you're working on will not evolve as quickly as it could've evolved if you didn't use mutation testing). Code must be more or less established before you dive into metric-driven refactoring mode.

[mdespuits]

1. These closing comments are flipped around
2. Are they really necessary for this small class?

[mbj]

@mattdbridges Flip: fixed. Needed: Just my preference, if its not the preference of this project I'll drop them.

[mbj]

I'm closing this pull request since I do not have any time to finish it.