fauna / bleak_house

BleakHouse

A library for finding memory leaks.

== License

Copyright 2007, 2008 Cloudburst, LLC. Licensed under the AFL 3. See the included LICENSE file. Portions copyright 2006 
Eric Hodel and used with permission. See the included LICENSE_BSD file.

The public certificate for this gem is 
here[http://rubyforge.org/frs/download.php/25331/evan_weaver-original-public_cert.pem].

If you use this software, please {make a donation}[http://blog.evanweaver.com/donate/], or {recommend 
Evan}[http://www.workingwithrails.com/person/7739-evan-weaver] at Working with Rails.

== Features

* leak-proof C instrumentation
* minimal impact on runtime performance
* fast analysis step
* tracks all objects allocated on the heap, including internal types like <tt>T_NODE</tt>
* easy integration into any program, not just Rails

== Requirements

* A unix-like operating system
* Ruby 1.8.7

= Usage

== Installation

Install the gem:
  sudo gem install bleak_house

The installation takes a long time because it compiles a patched Ruby 1.8.7 binary for you. It is installed as 
<tt>ruby-bleak-house</tt> alongside your regular <tt>ruby</tt> binary.

Please see the forum ( http://rubyforge.org/forum/forum.php?forum_id=13983 ) if you have installation problems.

== Usage

We will profile a Rails app as an example. Note that BleakHouse works equally well in any Ruby program.

First, to setup the app for profiling, add the following at the bottom of <tt>config/environment.rb</tt>:
  require 'bleak_house' if ENV['BLEAK_HOUSE']

Then, to engage the logger (possibly in a live deployment situation), start a server instance as so:
  RAILS_ENV=production BLEAK_HOUSE=1 ruby-bleak-house ./script/server

Look for the message:
  ** Bleakhouse: installed

Exercise your app. After a couple hundred requests, hit CTRL-C. The server will stop and BleakHouse will produce a 
dumpfile in <tt>/tmp</tt>:

  ** BleakHouse: working...
  ** BleakHouse: complete
  ** Bleakhouse: run 'bleak /tmp/bleak.5979.000.dump' to analyze.

To analyze it, just run the listed command. The top 20 leakiest lines will be listed:

  191691 total objects
  Final heap size 191691 filled, 220961 free
  Displaying top 20 most common line/class pairs
  89513 __null__:__null__:__node__
  41438 __null__:__null__:String
  2348 /opt/local//lib/ruby/site_ruby/1.8/rubygems/specification.rb:557:Array
  1508 /opt/local//lib/ruby/gems/1.8/specifications/gettext-1.90.0.gemspec:14:String
  1021 /opt/local//lib/ruby/gems/1.8/specifications/heel-0.2.0.gemspec:14:String
   951 /opt/local//lib/ruby/site_ruby/1.8/rubygems/version.rb:111:String
   935 /opt/local//lib/ruby/site_ruby/1.8/rubygems/specification.rb:557:String
   834 /opt/local//lib/ruby/site_ruby/1.8/rubygems/version.rb:146:Array
  ...

You can pass an integer as the second parameter to <tt>bleak</tt> if you want to see more lines than the default.

The underscored types are special Ruby internal structs, but can be real leaks just as easily as fullblown classes.

= Extras

== Injecting a signal

You can send <tt>SIGUSR2</tt> to a BleakHouse-instrumented program to snag a dump at any time. Once the dump completes, 
the program will continue to run. Dumps are named based on the host process id, and sequential dumps are numbered in 
ascending order.

== Tips

Do not try to detect Rails leaks in <tt>development</tt> mode. Make a separate <tt>benchmark</tt> environment if you 
need to, and make sure all your production caching is turned on.

It is normal to see lots of <tt>null:null</tt> references, especially for nodes. Using <tt>eval()</tt> too much can be a 
cause of node leaks. You can sometimes track <tt>eval()</tt> by using sourceline macros in your code:

  eval("CODE", nil, __FILE__, __LINE__)

You may get library require errors if you install <tt>ruby-bleak-house</tt> 1.8.7 alongside a different verson of Ruby. 
You could try to patch your local version of Ruby instead, or you can get <tt>ruby-bleak-house</tt> to lie about its 
version. Just make sure that the <tt>bleak-house</tt> library is the first thing required (even before Rubygems):

  ruby-bleak-house -I `ruby -e 'puts \`gem which bleak_house\`.split("\n")[1][0..-16]'` -rbleak_house

It is not recommended that you use <tt>ruby-bleak-house</tt> as your production Ruby binary, since it will be slightly 
slower and use slightly more memory. It is unlikely, however, to affect stability.

If BleakHouse doesn't report any heap growth but you still have memory growth, you might have a broken C extension, or 
have encounted a {real leak in the 
interpreter}[http://groups.google.com/group/god-rb/browse_thread/thread/01cca2b7c4a581c2]. Try using 
Valgrind[http://blog.evanweaver.com/articles/2008/02/05/valgrind-and-ruby/] instead.

== Methods

The easiest way to fix a leak is to make it repeatable.

First, write a script that exercises your app in a deterministic way. Run it for a small number of loops; then run 
<tt>bleak</tt>. Then run it for a larger number of loops, and run <tt>bleak</tt> again. The lines that grow 
significantly between runs are your leaks for that codepath.

Now, look at those lines in the source and try to figure out what references them. Where do the return values go? Add 
some breakpoints or output backtraces to <tt>STDERR</tt> as you go. Eventually you should find a point where it is 
relatively clear that a reference is getting maintained.

Try to remove that reference, run your script again, and see if the object counts have dropped.

== Reporting problems

The support forum is here[http://rubyforge.org/forum/forum.php?forum_id=13983].

Patches and contributions are very welcome. Please note that contributors are required to assign copyright for their 
additions to Cloudburst, LLC.

== Further resources

* http://blog.evanweaver.com/articles/2008/04/06/bleakhouse-4/
* http://blog.evanweaver.com/articles/2008/02/05/valgrind-and-ruby/
* http://blog.evanweaver.com/articles/2007/05/12/let-me-hit-you-with-some-knowledge
* http://blog.evanweaver.com/articles/2007/05/06/leak-proof-direct-heap-instrumentation-for-bleak_house
* http://blog.evanweaver.com/articles/2007/04/28/bleak_house