ActiveRecord objects are not cleared from memory when held in a hash #6929

Closed
vaharoni opened this Issue Jul 2, 2012 · 6 comments

5 participants

@vaharoni

It seems that in some cases ActiveRecord::Relation objects keep ActiveRecord objects from clearing in memory. This happens for Ruby >= 1.9.2. When tested in 1.8.7, the problem does not exist. Performing #reset on all ActiveRecord::Relation objects is a working workaround.

Here are reproducible steps from an irb session:

Good behavior:

ActiveRecord::Base.establish_connection(...)
class Student < ActiveRecord::Base; end

ObjectSpace.each_object(ActiveRecord::Base).count
# => 0

x = Student.limit(100)

ObjectSpace.each_object(ActiveRecord::Base).count
# => 100

x=nil
GC.start

ObjectSpace.each_object(ActiveRecord::Base).count
# => 0   Good!

Bad behavior:

ActiveRecord::Base.establish_connection(...)
class Student < ActiveRecord::Base; end

ObjectSpace.each_object(ActiveRecord::Base).count
# => 0

x = Student.limit(100).group_by(&:group_id)

ObjectSpace.each_object(ActiveRecord::Base).count
# => 100

x=nil
GC.start

ObjectSpace.each_object(ActiveRecord::Base).count
# => 100   Bad!

Workaround:

ActiveRecord::Base.establish_connection(...)
class Student < ActiveRecord::Base; end

ObjectSpace.each_object(ActiveRecord::Base).count
# => 0

x = Student.limit(100).group_by(&:group_id)

ObjectSpace.each_object(ActiveRecord::Base).count
# => 100

x=nil
ObjectSpace.each_object(ActiveRecord::Relation).each(&:reset)
GC.start

ObjectSpace.each_object(ActiveRecord::Base).count
# => 0 
@steveklabnik
Ruby on Rails member

This may be related to #2640.

@oscardelben

For reference, the objects are kept around by the @records instance variable on relation.

@thedarkone

I think you are just seeing some MRI's ObjectSpace.each_object and GC.start weirdness:

Running this:

./script/rails r 'Spree::Product.limit(1).inspect;GC.start;puts ObjectSpace.each_object(ActiveRecord::Base).count;GC.start;puts ObjectSpace.each_object(ActiveRecord::Base).count'

produces:

1
1

while adding some artificial GC pressure:

./script/rails r 'Spree::Product.limit(1).inspect;GC.start;puts ObjectSpace.each_object(ActiveRecord::Base).count;1_000_000.times {|i| ([] << i << i).join};GC.start;puts ObjectSpace.each_object(ActiveRecord::Base).count'

produces:

1
0
@vaharoni

@thedarkone I actually get 1 and 0 in your first example on Ruby 1.9.3-p0 Rails 3.1.0. You get the first "1" probably because the _ variable holds the result for the first GC.start.

Just wanted to clarify that this is not a theoretical issue. I reported the issue after debugging a problem in production in which OOM Killer killed nightly Resque processes due to lack of memory on my server. I specifically implemented a "process batch" method that caused all ActiveRecord objects to fall out of scope and called GC.start in between batches, but the memory still went up and up. I narrowed down the issue to the one I reported above. Hope this gives some more context.

@thedarkone

You get the first "1" probably because the _ variable holds the result for the first GC.start.

There is no _ variable as I wanted to avoid any potential IRB interference:

./script/rails r 'x=:a;puts _'
/Users/vit/.rvm/gems/ruby-1.9.3-p194/gems/railties-3.2.6/lib/rails/commands/runner.rb:53:in `eval': undefined local variable or method `_' for main:Object (NameError)
    from /Users/vit/.rvm/gems/ruby-1.9.3-p194/gems/railties-3.2.6/lib/rails/commands/runner.rb:53:in `eval'
    from /Users/vit/.rvm/gems/ruby-1.9.3-p194/gems/railties-3.2.6/lib/rails/commands/runner.rb:53:in `<top (required)>'
    from /Users/vit/.rvm/gems/ruby-1.9.3-p194/gems/railties-3.2.6/lib/rails/commands.rb:64:in `require'
    from /Users/vit/.rvm/gems/ruby-1.9.3-p194/gems/railties-3.2.6/lib/rails/commands.rb:64:in `<top (required)>'
    from ./script/rails:6:in `require'
    from ./script/rails:6:in `<main>'

Here's what I get with nils everywhere:

./script/rails r 'Spree::Product.limit(1).inspect;nil;GC.start;puts ObjectSpace.each_object(ActiveRecord::Base).count;nil;GC.start;nil;puts ObjectSpace.each_object(ActiveRecord::Base).count'
1
1

I actually get 1 and 0 in your first example on Ruby 1.9.3-p0 Rails 3.1.0.

I was testing with ruby 1.9.3p194 (2012-04-20 revision 35410) and Rails 3.2.6, so maybe that's why we're getting different results.

I specifically implemented a "process batch" method that caused all ActiveRecord objects to fall out of scope and called GC.start in between batches, but the memory still went up and up.

Can you gist your process batch script?

@rafaelfranca
Ruby on Rails member

Closing this for the same reason that #7112.

See #7112 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment