high memory usage on idle

[m4tm4t]

I'm running a simple app on a VPS with 256mb of RAM, ruby-1.9.3-p125 & rails 3.2.3 on debian squeeze
DJ take 30% (~77mb) by worker ! (like unicorn worker)

DJ load the entire app or i'm experiencing a memory leak ?

it cost very much for send some mail at registration in background !!

[danielmorrison]

DJ does indeed load the entire Rails environment. That's a feature! ;)

On Apr 10, 2012, at 9:50 PM, fooheyreply@reply.github.com wrote:

I'm running a simple app on a VPS with 256mb of RAM, ruby-1.9.3-p125 & rails 3.2.3 on debian squeeze
DJ take 30% (~77mb) by worker ! (like unicorn worker)

DJ load the entire app or i'm experiencing a memory leak ?

it cost very much for send some mail at registration in background !!

---

Reply to this email directly or view it on GitHub:
#381

[m4tm4t]

Okay so I will look a custom environment for DJ, I see somewhere it's possible to load only needed classes like http://www.morellon.com.br/2011/07/tuning-resque-workers-memory-usage.html for resque. it's possible to make something like that with DJ ?

[saurabhnanda]

Facing the same issue here. DJ is taking 99 MB of precious 256 MB RAM. I understand that DJ requires the Rails environment to work, but does it load everything, i.e. even controllers?

Is there any way in which it DJ can be configured to run from within the Passenger process? That way it doesn't need to load a separate copy of the Rails environment.

[albus522]

DJ loads the complete environment, and always will, and no there is no way to run DJ within another process.

[saurabhnanda]

@albus522 -- Got it. However, if one were to change this default behaviour (in a private fork), could you please give some pointers as to what parts of DJ one should be looking at changing?

[albus522]

Honestly, you would have better luck using a different backgrounding solution. Loading the whole app is pretty fundamental to how DJ works. Rails 4 is adding an app config option "eager_load" that might be of use, but I am not convinced that will help enough to be useful. Unless your app is enormous, not loading your controller code is not going to do much for your memory footprint.

As an example, of the 1574 files loaded ($") by a non-trivial app 32 (2%) are actually app files. Also when you consider file size it moves to 30k (less than 0.1%) out of 38,632k.

To do your own app analysis, this is how I came up with those numbers

# Total files loaded
$".size

# Files not in vendor/bundle or vendor/ruby
# which is where your gems and ruby reside on heroku
$".reject {|i| /vendor\/(bundle|ruby)/ =~ i }.size

# Total file size in kilobytes
$".sum {|i| File.size(i) rescue 0 } / 1024.0

# App file size in kilobytes
$".reject {|i| /vendor\/(bundle|ruby)/ =~ i }.sum {|i| File.size(i) rescue 0 } / 1024.0