ProgressReporter takes 50% longer than DefaultReporter #103

Closed
brandonweiss opened this Issue Apr 21, 2014 · 2 comments

2 participants

@brandonweiss

This is weird. Maybe I'm missing something obvious, but my tests normally take around 11 seconds to run. I decided I'd try out the ProgressReporter, and noticed that inexplicably my tests are now taking 17 or so seconds to run. I literally did nothing but change the reporter type. Here's the output.

➜  canary (master) rake

# Running tests:

................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................

Finished tests in 11.733265s, 57.2731 tests/s, 108.9211 assertions/s.

672 tests, 1278 assertions, 0 failures, 0 errors, 0 skips
➜  canary (master) rake
Started

672/672: [==============================================================================================================================================================================================================] 32.7/s 100% 00:00:17

Finished in 17.44068s
672 tests, 1278 assertions, 0 failures, 0 errors, 0 skips

Just to be sure it wasn't a reporting error I timed it with a clock, and those numbers are right. Is this a bug in the reporter?

@os97673
Collaborator

Well, it is expected for ProgressReporter to work slower since it uses more complicated output.
I'm not sure if there is a way to improve it performance, but you are welcome to investigate this ;)

@os97673 os97673 closed this Apr 21, 2014
@brandonweiss

Are you sure about that? I mean, it's not like the progress reporter is calculating primes or something in order to make the output. I can't think of any reason it shouldn't be just as fast as any of the reporters. I just did a quick experiment and swapped out PowerBar for ruby-progressbar, and the test time returned to normal. With ruby-progressbar the ProgressReporter is just as fast as the other reporters, so it looks like the problem is in PowerBar. I'll make a patch and send a pull request in a bit.

@brandonweiss brandonweiss added a commit that referenced this issue Apr 21, 2014
@brandonweiss brandonweiss kern/minitest-reporters#103: Switching from PowerBar to ruby-progressbar
PowerBar seems to be a bit defunct (last updated a year ago, years-old
issues sitting open). But, most importantly, the `ProgressReporter`
makes my tests run 50% slower than any other reporter. That doesn’t
seem right—whatever is being done to generate the progress bar couldn’t
possibly be complicated enough to increase the test run time by 50%.

Since PowerBar is defunct I figured the easiest way is to just switch
to something actively developed, like ruby-progressbar, and that did
the trick. Test run times with the `ProgressReporter` now average the
same as the other reporters.

The output will almost be completely identical. The one change I made
is that now only the progress bar will be colored. Previously the
entire line would be green, and then change to yellow or red if there
was a skip or failure. Now only the progress bar itself will change
color, which makes the output visually easier to read.
25ac5cd
@waratuman waratuman added a commit to waratuman/minitest-reporters that referenced this issue Jan 21, 2015
@brandonweiss brandonweiss kern/minitest-reporters#103: Switching from PowerBar to ruby-progressbar
PowerBar seems to be a bit defunct (last updated a year ago, years-old
issues sitting open). But, most importantly, the `ProgressReporter`
makes my tests run 50% slower than any other reporter. That doesn’t
seem right—whatever is being done to generate the progress bar couldn’t
possibly be complicated enough to increase the test run time by 50%.

Since PowerBar is defunct I figured the easiest way is to just switch
to something actively developed, like ruby-progressbar, and that did
the trick. Test run times with the `ProgressReporter` now average the
same as the other reporters.

The output will almost be completely identical. The one change I made
is that now only the progress bar will be colored. Previously the
entire line would be green, and then change to yellow or red if there
was a skip or failure. Now only the progress bar itself will change
color, which makes the output visually easier to read.
d08d59e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment