Profile object allocations with ruby 2.1's trace_object_allocations #1167

Closed
myronmarston opened this Issue Nov 9, 2013 · 9 comments

Comments

Projects
None yet
5 participants
Owner

myronmarston commented Nov 9, 2013

Just got done watching a talk on ruby 2.1 and it's new trace_object_allocations feature. The allocation_stats gem packages up some nice features to sort through the data, and I think we could find some low-hanging fruit to optimize and reduce the number of objects RSpec creates (and thus, reduce GC time). We should do this for core, mocks and expectations.

Owner

JonRowe commented Nov 10, 2013

👍 but we need to fix RSpec to run reliably on 2.1 first

Owner

myronmarston commented Nov 10, 2013

From what I heard, 2.1 is supposed to be fully compatible with 2.0 so I suspect that any issues are more likely to be ruby bugs in the prerelease than RSpec bugs.

Owner

JonRowe commented Nov 10, 2013

Either/or

Contributor

tundal45 commented Dec 17, 2013

I would like to work on this but I will need a little bit of guidance since I am new to the codebase as well as ruby 2.1's tracing capabilities.

Member

xaviershay commented Dec 18, 2013

@tundal45 try running https://github.com/srawlins/allocation_stats against your spec suite, or against rspec's test suite. See if anything looks interesting, or if any new questions are raised. Rinse and repeat :)

Owner

myronmarston commented Dec 18, 2013

You can also watch the talk I attended (which inspired me opening this ticket) at rubyconf:

http://confreaks.com/videos/2870-rubyconf2013-new-ruby-2-1-awesomeness-fine-grained-object-allocation-tracing

Note that this requires ruby 2.1 (as it uses new features provided by MRI), but any improvements we make based on what we learn will benefit all ruby versions.

FWIW, when I opened this, I planned to create a bunch of example files that will focus on individual aspects of RSpec and contain only RSpec constructs (so as to eliminate the noise of non-rspec code):

  • A file that defines 1000 example groups, each with 1 example. Analyize object allocation while loading (but not running) the specs.
  • A file that defines 1 example group with 1000 examples. Analyze object allocation while loading (but not running) the specs.
  • Use the same files as above, but analyze running all the examples (but have the analysis disabled while loading the specs).
  • Define a file that uses rspec-expectations on its own (w/o the structure provided by rspec-core -- simply require it and include RSpec::Matchers in main) and have it run a bunch of expectations.
  • Do a similar thing for rspec-mocks.

Based on the results we get here, we can hopefully find some low-hanging fruit to optimize, to make RSpec allocate fewer objects, which would reduce the amount of garbage and improve perf on all ruby implementations.

Another video that may give you ideas here is @xaviershay's talk at rubyconf:

http://confreaks.com/videos/2860-rubyconf2013-profiling-ruby-finding-10x-gains-in-rspec-and-cruby

He talked about looking at profiler output for rspec-mocks that showed a surprising number of calls to Kernel#caller, which in turn provided the impetus to make some significant perf improvements to test double creation in rspec-mocks. I imagine we could see some similar things here: maybe we'll see a surprising number of allocations of a particular kind of object that will lead us to move it out of a tight loop or something. Hard to say what the improvements we'll find will be but hopefully that gives you some ideas.

@myronmarston myronmarston modified the milestone: Post 3.0, 3.0 Mar 24, 2014

ashag commented Jun 9, 2014

Hi. I work with @myronmarston. Since the last comments on this issue are from last year, I was thinking I could work on this.

Owner

JonRowe commented Jun 9, 2014

I had this on my to do list but I've yet to get round to it so feel free to take a stab at it

Owner

myronmarston commented Oct 10, 2014

Closing as this has been addressed in #1738. There'll be more on going work for this but keeping this issue open is just noise, I think.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment