Delayed Job 3.0 Memory Leak

[jimishjoban]

I recently upgraded to use delayed_job 3.0 gem and I am facing this issue of memory leak. Even if I dont start delayed job, my rails app takes on entire memory of 2GB and ultimately crashes.

The moment I downgrade delayed_job gem to 2.1.4, the rails app behaves normally. Is this something related to delayed_job or could it also be my rails app? (remember, i dont even need to start delayed_job to eat up all the memory using the 3.0 gem)

My environment:

Rails 3.1
ruby 1.9.2p290 (2011-07-09 revision 32553) [i686-linux]
Delayed_job 3.0.0
Daemons 1.0.10
delayed_job_active_record (0.3.1)

How should I go debugging the problem? Any idea?

[bryckbost]

Check the jobs table and daemons log. Could be failed jobs that are causing memory issues.

[fcheung]

This probably isn't really delayed job's fault. Prior to 3.0 delayed job forced the use of the syck YAML engine. 3.0 doesn't any more so you get the default of psych instead (unless you are specifying this elsewhere). Unfortunately the version of psych in currently released versions of 1.9.2 has quite a big memory leak in YAML.load. One fix is to use the psych gem (which has a newer version of psych), you can also switch back to the old syck YAML engine and I believe ruby 1.9.3 also has a fixed version of psych

[jjb]

Perhaps DJ could require the psych gem if the version of rails detected has the memory leak?

[fcheung]

It's gets a bit fiddly - bundler loads psych during startup so by the time delayed job turns up it's too late. (for that reason it's also not enough to just add it to your Gemfile). Printing a warning with information on where to get more details would be easy though

[ismell]

I'm assuming this can be closed as of 1.9.3?