- Do you have (server) processes that sometimes crash for mysterious reasons?
- Can you not figure out why?
- Do they not print any error messages to their log files upon crashing?
- Are debuggers complicated, scary things that you'd rather avoid?
crash-watch to the rescue! This little program will monitor a specified process and wait until it crashes. It will then print useful information such as its exit status, what signal caused it to abort, and its backtrace.
gem install crash-watch
You must also have GDB installed. Mac OS X already has it by default. If you're on Linux, try one of these:
apt-get install gdb yum install gdb
You can verify the authenticity of the gem by following The Complete Guide to Verifying Gems with rubygems-openpgp.
Use our PPA:
sudo add-apt-repository ppa:phusion.nl/misc sudo apt-get update sudo apt-get install crash-watch
Our Ubuntu Lucid packages are compatible with Debian 6.
sudo sh -c 'echo deb http://ppa.launchpad.net/phusion.nl/misc/ubuntu lucid main > /etc/apt/sources.list.d/phusion-misc.list' sudo sh -c 'echo deb-src http://ppa.launchpad.net/phusion.nl/misc/ubuntu lucid main >> /etc/apt/sources.list.d/phusion-misc.list' sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 2AC745A50A212A8C sudo apt-get update sudo apt-get install crash-watch
- Enable EPEL: RHEL, CentOS, Amazon Linux.
Enable our YUM repository:
# RHEL 6, CentOS 6 curl -L https://oss-binaries.phusionpassenger.com/yumgems/phusion-misc/el.repo | \ sudo tee /etc/yum.repos.d/phusion-misc.repo # Amazon Linux curl -L https://oss-binaries.phusionpassenger.com/yumgems/phusion-misc/amazon.repo | \ sudo tee /etc/yum.repos.d/phusion-misc.repo
Install our package:
sudo yum install rubygem-crash-watch
$ crash-watch <PID> Monitoring PID <PID>... (...some time later, <PID> exits...) Process exited. Exit code: 0 Backtrace: Thread 1 (process 95205): #0 0x00007fff87ea1db0 in _exit () No symbol table info available. #1 0x000000010002a260 in ruby_stop () No symbol table info available. #2 0x0000000100031a54 in ruby_run () No symbol table info available. #3 0x00000001000009e4 in main () No symbol table info available.
While monitoring the process, you may interrupt
crash-watch by pressing Ctrl-C.
crash-watch will then detach from the process, which will then continue normally. You may re-attach
crash-watch --help for more usage options.
Instead of waiting until a process crashes, you can also dump a live backtrace of a process.
crash-watch will immediately exit after dumping the backtrace, letting the process continue as normally.
$ crash-watch --dump <PID> Current thread (1) backtrace: #0 0x00007fff81fd9464 in read () No symbol table info available. #1 0x0000000100060d3e in ?? () No symbol table info available.
I've written a small library for controlling gdb, which
crash-watch uses internally. With CrashWatch::GdbController you can send arbitrary commands to gdb and also get its response.
require 'crash_watch/gdb_controller' gdb = CrashWatch::GdbController.new
This will spawn a new GDB process. Use
#execute to execute arbitary GDB commands. Whatever the command prints to stdout and stderr will be available in the result string.
gdb.execute("bt") # => backtrace string gdb.execute("p 1 + 2") # => "$1 = 3\n"
#close when you no longer need it.