This gem sounds great and I want to use it. I'm just not sure how.
The documentation seems to have most of the parts, but it lacks an overall explanation of work flow.
I've added the gem to my gemfile, made the tmp/coverband_baseline.json file, added the rake tasks and the middleware. Then I wanted to test it locally, but the README kind of stops there.
The coverband_baseline rake task seemed like a good place to start, so I ran it (with --trace). A bunch of json spat out and no errors, so hurray. I did notice, however, that the filenames listed in the json only included my initializers.
With the baseline taken, I assumed the only other thing to do was run the coverband rake task, so I did. When it was finished, my browser popped open to show me a lovely simplecov page. The only problem was that it only listed my initializer files.
Could someone explain to me 1) what the ideal coverband workflow is (add all the stuff from the readme, deploy it, run the baseline, then the coverband tasks?) and 2) any reason why the rest of my production code was missed out?
I'd be happy to submit a PR with more setup instructions in the README once I get my head wrapped around these concepts.
Good call, the directions were a bit incomplete. Basically after you configure things you need to start your development server and make sure cover band is loaded. I also recommend first testing in development with coverage set at 100%, with verbose setting true, and logging enabled.
I updated the usage section of the README, let me know if this helps: https://github.com/danmayer/coverband#usage
I'm just curious what is the purpose of the baseline? I checked the definition of parse_baseline and it defaults to an empty hash if it can't find the tmp/coverband_baseline.json file. How important is the baseline and what happens if the baseline does just default to an empty hash?
hmm Sorry @sdhull that the purpose isn't clear enough. I will try to work on improving the docs some more. Basically the way coverband works is to record lines of code executed with set_trace_func, which this works well it doesn't give the proper output for many lines of code, for example most end lines that close blocks aren't ever 'executed'. It also doesn't really cover method definition, require lines, and includes... All of these get the proper output using the standard libraries Coverage, which is what we are trying to emulate with set_trace_func. Since Coverage can't be used with sampling during production runtime, the best compromise is to record a baseline first with Coverage just recording how it sees the code executed while loading. Then merging the lines executed during runtime later.
So it breaks down to two pieces.
When we generate the code coverage report we merge the two together.
Awesome, that helps a lot! So it's not important to run generate the baseline prior to running in production, only prior to generating the coverage with the rake task. That's the "usage" bit I was unsure about.
As for coverage, I wonder what the performance impact would be for running at 100%. If my app had 10 appservers, I could deploy with coverage running on one of them for 10% sample rate 😉
yep you can run it any time to have the baseline to merge when making the report. Glad that helped.
@sdhull so running on a single server as a way of sample is a good idea. Coverage does have pretty good perf, but there are two other issues.
Anyways, could be fun to play around with and see if it works out.