Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Specs are horribly slow (help wanted!) #547

Closed
justinfrench opened this Issue May 5, 2011 · 8 comments

Comments

Projects
None yet
4 participants
Owner

justinfrench commented May 5, 2011

I know that there's some really lazy "mock the whole word" stuff in here, and I know we're doing a lot of rendering, but if someone with some experience in profiling ruby can figure out where the biggest bottlenecks are and find some quick wins to speed things up, that'd be fantastic.

Maybe it's a performance issue in the code, but I suspect it's more likely an issue in the mocking and stubbing that takes place before each spec. They never used to be this slow, and I find it surprising given there's no database or filesystem calls.

Contributor

jwinter commented Jun 1, 2011

I ran perftools.rb against rake spec and it looks like a lot of time is being spent in garbage collection. I switched from MRI 1.8.7-p249 to ree-1.8.7.2011.03 and that cut spec runs in half: https://gist.github.com/1001677

I'm still learning to read the output of perftools, but here's two graphs, one from MRI (http://dl.dropbox.com/u/217539/formtastic_mri_perf.pdf) and one from REE (http://dl.dropbox.com/u/217539/formtastic_ree_perf.pdf). Apart from garbage collection, it looks like a substantial amount of time is being spent in stubbing methods, plus some time in Object#localized_string.

Owner

justinfrench commented Jun 1, 2011

@jwinter this is awesome! So, is there a way to manage GC better ourselves in MRI187? Perhaps something like http://makandra.com/notes/950-speed-up-rspec-by-deferring-garbage-collection

Stubbing doesn't surprise me (every model for every scenario is stubbed), so I should have a look at how much stuff we stub on every scenario (then evaluate), and how much is just done at the start of the suite (and if it can be done better).

Owner

justinfrench commented Jun 1, 2011

Wow, just applied the DeferredGarbageCollector patch above:

# before
Finished in 178.55 seconds
1148 examples, 0 failures

#after
Finished in 66.23 seconds
1148 examples, 0 failures

That's a start!

justinfrench added a commit that referenced this issue Jun 1, 2011

Deferring garbage collection cuts the time to run the test suite down (
…#547).

Just over a third of the time it used to take.

    # before
    Finished in 178.55 seconds
    1148 examples, 0 failures

    #after
    Finished in 66.23 seconds
    1148 examples, 0 failures
Owner

justinfrench commented Jun 1, 2011

Looking into stubs and mocks, there doesn't seem to be a "faster" way in rspec. There are ways to stub more than one method on an object at a time, but under the hood, those methods simply call the singular stub method, so I can't see how there's much if any to gain.

The win is probably much harder...

  1. more mock objects set up once properly for re-use in different scenarios without re-stubbing the same stuff over and over again
  2. moving some of the things we repeatedly stub up into the base objects (Post, Author) as real instance and class methods

Both doable, but I'm not sure if it's worth the pain.

Owner

justinfrench commented Jun 1, 2011

Object#localized_string has me baffled. We have a method name #localized_string (and I'd bet good money it's very slow), but I can't see where it would be included in Object. It's included in a few other classes, so perhaps these are aggregated together as one item in the PDF?

Collaborator

sobrinho commented Jun 3, 2011

I will take a look into spec tonight to find what is happenning.

Maybe creating models with database instead of mocking will be more faster.

Owner

justinfrench commented Jun 6, 2011

I find things are mostly fast enough for 2.0 good to keep bringing it down, but no urgency.

rhulse commented Jun 17, 2011

!!!

Finished in 870.17 seconds
1149 examples, 0 failures
Coverage: 95.77% (threshold: 95%)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment