BleakHouse
A library for finding memory leaks. Patched for Ruby 1.8.7p72.
An experiment.
Run git clone and then rake gem;
gem install bleak_house
== License
Copyright 2007, 2008 Cloudburst, LLC. Licensed under the AFL 3. See the included LICENSE file. Portions copyright 2006 Eric Hodel and used with permission. See the included LICENSE_BSD file.
The public certificate for this gem is here[http://rubyforge.org/frs/download.php/25331/evan_weaver-original-public_cert.pem].
If you use this software, please {make a donation}[http://blog.evanweaver.com/donate/], or {recommend Evan}[http://www.workingwithrails.com/person/7739-evan-weaver] at Working with Rails.
== Features
* leak-proof C instrumentation
* minimal impact on runtime performance
* fast analysis step
* tracks all objects allocated on the heap, including internal types like <tt>T_NODE</tt>
* easy integration into any program, not just Rails
== Requirements
* A unix-like operating system
* Ruby 1.8.7
= Usage
== Installation
Install the gem:
sudo gem install bleak_house
The installation takes a long time because it compiles a patched Ruby 1.8.7 binary for you. It is installed as <tt>ruby-bleak-house</tt> alongside your regular <tt>ruby</tt> binary.
Please see the forum ( http://rubyforge.org/forum/forum.php?forum_id=13983 ) if you have installation problems.
== Usage
We will profile a Rails app as an example. Note that BleakHouse works equally well in any Ruby program.
First, to setup the app for profiling, add the following at the bottom of <tt>config/environment.rb</tt>:
require 'bleak_house' if ENV['BLEAK_HOUSE']
Then, to engage the logger (possibly in a live deployment situation), start a server instance as so:
RAILS_ENV=production BLEAK_HOUSE=1 ruby-bleak-house ./script/server
Look for the message:
** Bleakhouse: installed
Exercise your app. After a couple hundred requests, hit CTRL-C. The server will stop and BleakHouse will produce a dumpfile in <tt>/tmp</tt>:
** BleakHouse: working...
** BleakHouse: complete
** Bleakhouse: run 'bleak /tmp/bleak.5979.000.dump' to analyze.
To analyze it, just run the listed command. The top 20 leakiest lines will be listed:
191691 total objects
Final heap size 191691 filled, 220961 free
Displaying top 20 most common line/class pairs
89513 __null__:__null__:__node__
41438 __null__:__null__:String
2348 /opt/local//lib/ruby/site_ruby/1.8/rubygems/specification.rb:557:Array
1508 /opt/local//lib/ruby/gems/1.8/specifications/gettext-1.90.0.gemspec:14:String
1021 /opt/local//lib/ruby/gems/1.8/specifications/heel-0.2.0.gemspec:14:String
951 /opt/local//lib/ruby/site_ruby/1.8/rubygems/version.rb:111:String
935 /opt/local//lib/ruby/site_ruby/1.8/rubygems/specification.rb:557:String
834 /opt/local//lib/ruby/site_ruby/1.8/rubygems/version.rb:146:Array
...
You can pass an integer as the second parameter to <tt>bleak</tt> if you want to see more lines than the default.
The underscored types are special Ruby internal structs, but can be real leaks just as easily as fullblown classes.
= Extras
== Injecting a signal
You can send <tt>SIGUSR2</tt> to a BleakHouse-instrumented program to snag a dump at any time. Once the dump completes, the program will continue to run. Dumps are named based on the host process id, and sequential dumps are numbered in ascending order.
== Tips
Do not try to detect Rails leaks in <tt>development</tt> mode. Make a separate <tt>benchmark</tt> environment if you need to, and make sure all your production caching is turned on.
It is normal to see lots of <tt>null:null</tt> references, especially for nodes. Using <tt>eval()</tt> too much can be a cause of node leaks. You can sometimes track <tt>eval()</tt> by using sourceline macros in your code:
eval("CODE", nil, __FILE__, __LINE__)
You may get library require errors if you install <tt>ruby-bleak-house</tt> 1.8.7 alongside a different verson of Ruby. You could try to patch your local version of Ruby instead, or you can get <tt>ruby-bleak-house</tt> to lie about its version. Just make sure that the <tt>bleak-house</tt> library is the first thing required (even before Rubygems):
ruby-bleak-house -I `ruby -e 'puts \`gem which bleak_house\`.split("\n")[1][0..-16]'` -rbleak_house
It is not recommended that you use <tt>ruby-bleak-house</tt> as your production Ruby binary, since it will be slightly slower and use slightly more memory. It is unlikely, however, to affect stability.
If BleakHouse doesn't report any heap growth but you still have memory growth, you might have a broken C extension, or have encounted a {real leak in the interpreter}[http://groups.google.com/group/god-rb/browse_thread/thread/01cca2b7c4a581c2]. Try using Valgrind[http://blog.evanweaver.com/articles/2008/02/05/valgrind-and-ruby/] instead.
== Methods
The easiest way to fix a leak is to make it repeatable.
First, write a script that exercises your app in a deterministic way. Run it for a small number of loops; then run <tt>bleak</tt>. Then run it for a larger number of loops, and run <tt>bleak</tt> again. The lines that grow significantly between runs are your leaks for that codepath.
Now, look at those lines in the source and try to figure out what references them. Where do the return values go? Add some breakpoints or output backtraces to <tt>STDERR</tt> as you go. Eventually you should find a point where it is relatively clear that a reference is getting maintained.
Try to remove that reference, run your script again, and see if the object counts have dropped.
== Reporting problems
The support forum is here[http://rubyforge.org/forum/forum.php?forum_id=13983].
Patches and contributions are very welcome. Please note that contributors are required to assign copyright for their additions to Cloudburst, LLC.
== Further resources
* http://blog.evanweaver.com/articles/2008/04/06/bleakhouse-4/
* http://blog.evanweaver.com/articles/2008/02/05/valgrind-and-ruby/
* http://blog.evanweaver.com/articles/2007/05/12/let-me-hit-you-with-some-knowledge
* http://blog.evanweaver.com/articles/2007/05/06/leak-proof-direct-heap-instrumentation-for-bleak_house
* http://blog.evanweaver.com/articles/2007/04/28/bleak_house
) is private.
