Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Specs are horribly slow (help wanted!) #547

Closed
justinfrench opened this Issue · 8 comments

4 participants

@justinfrench
Owner

I know that there's some really lazy "mock the whole word" stuff in here, and I know we're doing a lot of rendering, but if someone with some experience in profiling ruby can figure out where the biggest bottlenecks are and find some quick wins to speed things up, that'd be fantastic.

Maybe it's a performance issue in the code, but I suspect it's more likely an issue in the mocking and stubbing that takes place before each spec. They never used to be this slow, and I find it surprising given there's no database or filesystem calls.

@jwinter

I ran perftools.rb against rake spec and it looks like a lot of time is being spent in garbage collection. I switched from MRI 1.8.7-p249 to ree-1.8.7.2011.03 and that cut spec runs in half: https://gist.github.com/1001677

I'm still learning to read the output of perftools, but here's two graphs, one from MRI (http://dl.dropbox.com/u/217539/formtastic_mri_perf.pdf) and one from REE (http://dl.dropbox.com/u/217539/formtastic_ree_perf.pdf). Apart from garbage collection, it looks like a substantial amount of time is being spent in stubbing methods, plus some time in Object#localized_string.

@justinfrench
Owner

@jwinter this is awesome! So, is there a way to manage GC better ourselves in MRI187? Perhaps something like http://makandra.com/notes/950-speed-up-rspec-by-deferring-garbage-collection

Stubbing doesn't surprise me (every model for every scenario is stubbed), so I should have a look at how much stuff we stub on every scenario (then evaluate), and how much is just done at the start of the suite (and if it can be done better).

@justinfrench
Owner

Wow, just applied the DeferredGarbageCollector patch above:

# before
Finished in 178.55 seconds
1148 examples, 0 failures

#after
Finished in 66.23 seconds
1148 examples, 0 failures

That's a start!

@justinfrench justinfrench referenced this issue from a commit
@justinfrench Deferring garbage collection cuts the time to run the test suite down (
…#547).

Just over a third of the time it used to take.

    # before
    Finished in 178.55 seconds
    1148 examples, 0 failures

    #after
    Finished in 66.23 seconds
    1148 examples, 0 failures
2787d87
@justinfrench
Owner

Looking into stubs and mocks, there doesn't seem to be a "faster" way in rspec. There are ways to stub more than one method on an object at a time, but under the hood, those methods simply call the singular stub method, so I can't see how there's much if any to gain.

The win is probably much harder...

  1. more mock objects set up once properly for re-use in different scenarios without re-stubbing the same stuff over and over again
  2. moving some of the things we repeatedly stub up into the base objects (Post, Author) as real instance and class methods

Both doable, but I'm not sure if it's worth the pain.

@justinfrench
Owner

Object#localized_string has me baffled. We have a method name #localized_string (and I'd bet good money it's very slow), but I can't see where it would be included in Object. It's included in a few other classes, so perhaps these are aggregated together as one item in the PDF?

@sobrinho
Collaborator

I will take a look into spec tonight to find what is happenning.

Maybe creating models with database instead of mocking will be more faster.

@justinfrench
Owner

I find things are mostly fast enough for 2.0 good to keep bringing it down, but no urgency.

@rhulse

!!!

Finished in 870.17 seconds
1149 examples, 0 failures
Coverage: 95.77% (threshold: 95%)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.