Heapy (Ruby Heap Dump Inspector)
A CLI for analyzing Ruby Heap dumps. Thanks to Sam Saffron for the idea and initial code.
Add this line to your application's Gemfile:
And then execute:
Or install it yourself as:
$ gem install heapy
Diff 2 heap dumps
Run with two inputs to output the values of today.dump that are not present in yesterday.dump
$ heapy diff tmp/yesterday.dump tmp/today_morning.dump Allocated STRING 9991 objects of size 399640/491264 (in bytes) at: scratch.rb:24
Run with three inputs to show the diff between the first two, but only if the objects are still retained in the third
$ heapy diff tmp/yesterday.dump tmp/today_morning.dump tmp/today_afternoon.dump Retained STRING 9991 objects of size 399640/491264 (in bytes) at: scratch.rb:24 # ...
Pass in the name of an output file and the objects present in today.dump that aren't in yesterday.dump will be written to that file
$ heapy diff tmp/yesterday.dump tmp/today.dump --output_diff=output.json Allocated STRING 9991 objects of size 399640/491264 (in bytes) at: scratch.rb:24 # ... Writing heap dump diff to output.json
Due to how the garbage collector in MRI manages objects on the Ruby heap,
heapy diff may produce incomplete
or incorrect diffs under the following circumstances:
- Heap compaction. When compacting the Ruby heap either manually via
GC.compactor with auto-compaction enabled, heapy cannot produce accurate diffs, because objects may move into different heap slots and will appear as newly allocated even if they weren't. In general, any mechanism that moves existing objects from one heap slot to another will invalidate diff reports. Always turn off compaction before taking the
- Temporary allocations. Diffs might omit objects from an
<after>dump if Ruby allocated them into heap slots that were previously occupied by different objects in the
<before>dump, and which were then deallocated between both dumps. To minimize the chance of this happening, triggering a major GC three or more times between heap dumps can help tenuring survivors and thus stabilizing the heap prior to taking a diff.
Read a Heap Dump
Step 2) Once you've got the heap dump, you can analyze it using this CLI:
$ heapy read tmp/2015-10-01T10:18:59-05:00-heap.dump Generation: nil object count: 209191 Generation: 14 object count: 407 Generation: 15 object count: 638 Generation: 16 object count: 748 Generation: 17 object count: 1023 Generation: 18 object count: 805 # ...
NOTE: The reason you may be getting a "nil" generation is these objects were loaded into memory before your code began tracking the allocations. To ensure all allocations are tracked you can execute your ruby script this trick. First create a file
trace.rb that only starts allocation tracing:
# trace.rb require 'objspace' ObjectSpace.trace_object_allocations_start
Now make sure this command is loaded before you run your script, you can use Ruby's
-I to specify a load path and
-r to specify a library to require, in this case our trace file
$ ruby -I ./ -r trace script_name.rb
If the last line of your file is invalid JSON, make sure that you are closing the file after writing the ruby heap dump to it.
Digging into a Generation
You can drill down into a specific generation. In the previous example, the 17'th generation looks strangely large, you can drill into it:
$ heapy read tmp/2015-10-01T10:18:59-05:00-heap.dump 17 Analyzing Heap (Generation: 17) ------------------------------- allocated by memory (44061517) (in bytes) ============================== 39908512 /app/vendor/ruby-2.2.3/lib/ruby/2.2.0/timeout.rb:79 1284993 /app/vendor/ruby-2.2.3/lib/ruby/2.2.0/openssl/buffering.rb:182 201068 /app/vendor/bundle/ruby/2.2.0/gems/json-1.8.3/lib/json/common.rb:223 189272 /app/vendor/bundle/ruby/2.2.0/gems/newrelic_rpm-126.96.36.1992/lib/new_relic/agent/stats_engine/stats_hash.rb:39 172531 /app/vendor/ruby-2.2.3/lib/ruby/2.2.0/net/http/header.rb:172 92200 /app/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.3/lib/active_support/core_ext/numeric/conversions.rb:131
You can limit the output by passing in a
$ heapy read tmp/2015-10-01T10:18:59-05:00-heap.dump 17 --lines=6
Note: Default lines value is 50
Reviewing all generations
If you want to read all generations you can use the "all" directive
$ heapy read tmp/2015-10-01T10:18:59-05:00-heap.dump all
After checking out the repo, run
$ bundle install to install dependencies. Then, run
rake spec to run the tests.
To install this gem onto your local machine, run
bundle exec rake install. To release a new version, update the version number in
version.rb, and then run
bundle exec rake release, which will create a git tag for the version, push git commits and tags, and push the
.gem file to rubygems.org.
Bug reports and pull requests are welcome on GitHub at https://github.com/schneems/heapy. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the Contributor Covenant code of conduct.
The gem is available as open source under the terms of the MIT License.