Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

high memory usage on idle #381

Closed
m4tm4t opened this Issue Apr 11, 2012 · 6 comments

Comments

Projects
None yet
4 participants

m4tm4t commented Apr 11, 2012

I'm running a simple app on a VPS with 256mb of RAM, ruby-1.9.3-p125 & rails 3.2.3 on debian squeeze
DJ take 30% (~77mb) by worker ! (like unicorn worker)

DJ load the entire app or i'm experiencing a memory leak ?

it cost very much for send some mail at registration in background !!

Owner

danielmorrison commented Apr 11, 2012

DJ does indeed load the entire Rails environment. That's a feature! ;)

On Apr 10, 2012, at 9:50 PM, fooheyreply@reply.github.com wrote:

I'm running a simple app on a VPS with 256mb of RAM, ruby-1.9.3-p125 & rails 3.2.3 on debian squeeze
DJ take 30% (~77mb) by worker ! (like unicorn worker)

DJ load the entire app or i'm experiencing a memory leak ?

it cost very much for send some mail at registration in background !!


Reply to this email directly or view it on GitHub:
#381

m4tm4t commented Apr 11, 2012

Okay so I will look a custom environment for DJ, I see somewhere it's possible to load only needed classes like http://www.morellon.com.br/2011/07/tuning-resque-workers-memory-usage.html for resque. it's possible to make something like that with DJ ?

Facing the same issue here. DJ is taking 99 MB of precious 256 MB RAM. I understand that DJ requires the Rails environment to work, but does it load everything, i.e. even controllers?

Is there any way in which it DJ can be configured to run from within the Passenger process? That way it doesn't need to load a separate copy of the Rails environment.

Owner

albus522 commented Jun 12, 2013

DJ loads the complete environment, and always will, and no there is no way to run DJ within another process.

@albus522 albus522 closed this Jun 12, 2013

@albus522 -- Got it. However, if one were to change this default behaviour (in a private fork), could you please give some pointers as to what parts of DJ one should be looking at changing?

Owner

albus522 commented Jun 13, 2013

Honestly, you would have better luck using a different backgrounding solution. Loading the whole app is pretty fundamental to how DJ works. Rails 4 is adding an app config option "eager_load" that might be of use, but I am not convinced that will help enough to be useful. Unless your app is enormous, not loading your controller code is not going to do much for your memory footprint.

As an example, of the 1574 files loaded ($") by a non-trivial app 32 (2%) are actually app files. Also when you consider file size it moves to 30k (less than 0.1%) out of 38,632k.

To do your own app analysis, this is how I came up with those numbers

# Total files loaded
$".size

# Files not in vendor/bundle or vendor/ruby
# which is where your gems and ruby reside on heroku
$".reject {|i| /vendor\/(bundle|ruby)/ =~ i }.size

# Total file size in kilobytes
$".sum {|i| File.size(i) rescue 0 } / 1024.0

# App file size in kilobytes
$".reject {|i| /vendor\/(bundle|ruby)/ =~ i }.sum {|i| File.size(i) rescue 0 } / 1024.0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment