Callbacks on large number of Embedded Docs blowing stack

[jamieorc]

A monthly process that worked with no problem on REE 1.8.7, Rails 2.3.14, and MM 0.8.6 now blows the stack on Rails 3.1.4 and MM >= 0.9.0 (tested on 0.11.0, 0.11.1 and 0.9.x).

My main model has a few fields and then an embedded doc with few fields that also has an embedded doc:

Doc => many EDoc1 => many EDoc2

I've verified the same code still works with Rails 2.3/MM 0.8.6.

In Ruby 1.9.3/Rails 3.1.4 I'd get a SystemStackError that the stack trace was too deep. I switched to REE 1.8.7 and got actual output:

/Users/jamieorc/.rvm/gems/ree-1.8.7-2011.12@pm-rails-3.1/gems/activesupport-3.1.4/lib/active_support/callbacks.rb:81:in send' /Users/jamieorc/.rvm/gems/ree-1.8.7-2011.12@pm-rails-3.1/gems/activesupport-3.1.4/lib/active_support/callbacks.rb:81:inrun_callbacks'
/Users/jamieorc/.rvm/gems/ree-1.8.7-2011.12@pm-rails-3.1/gems/mongo_mapper-0.9.2/lib/mongo_mapper/plugins/embedded_callbacks.rb:28:in run_callbacks' /Users/jamieorc/.rvm/gems/ree-1.8.7-2011.12@pm-rails-3.1/gems/mongo_mapper-0.9.2/lib/mongo_mapper/plugins/embedded_callbacks.rb:23:inrun_callbacks'
/Users/jamieorc/.rvm/gems/ree-1.8.7-2011.12@pm-rails-3.1/gems/activesupport-3.1.4/lib/active_support/callbacks.rb:390:in _run_validation_callbacks' /Users/jamieorc/.rvm/gems/ree-1.8.7-2011.12@pm-rails-3.1/gems/activesupport-3.1.4/lib/active_support/callbacks.rb:81:insend'

repeated ad naseum.

Just to underscore the point, an object that was saved with Rails 2.3.14 can be loaded in a Rails 3.1.4 console, but "SystemStackError: stack level too deep" if I try to save the unchanged object (Ruby 1.9.3).

I checked the size of my BSON object. It's only about 200k, an order of magnitude smaller than what Mongodb can store.

It seems to be related to the number of EmbeddedDocuments.

First layer of EmbeddedDocs:
asu.collections.count => 80

Second layer total:
asu.collections.reduce(0){|num, c| num + c.monthly_views.count} => 1600

Although it's a lot of docs, it's still only a 200k BSON object and Rails 2.3.14/MM 0.8.6 handle this with no problem at al..

Looking forward to feedback.

[jnunemaker]

I think we should kill embedded call acid or at least allow turning it off.

On Apr 15, 2012, at 1:02 PM, Jamie Orchard-Haysreply@reply.github.com wrote:

A monthly process that worked with no problem on REE 1.8.7, Rails 2.3.14, and MM 0.8.6 now blows the stack on Rails 3.1.4 and MM >= 0.9.0 (tested on 0.11.0, 0.11.1 and 0.9.x).

My main model has a few fields and then an embedded doc with few fields that also has an embedded doc:

Doc => many EDoc1 => many EDoc2

I've verified the same code still works with Rails 2.3/MM 0.8.6.

In Ruby 1.9.3/Rails 3.1.4 I'd get a SystemStackError that the stack trace was too deep. I switched to REE 1.8.7 and got actual output:

/Users/jamieorc/.rvm/gems/ree-1.8.7-2011.12@pm-rails-3.1/gems/activesupport-3.1.4/lib/active_support/callbacks.rb:81:in send' /Users/jamieorc/.rvm/gems/ree-1.8.7-2011.12@pm-rails-3.1/gems/activesupport-3.1.4/lib/active_support/callbacks.rb:81:inrun_callbacks'
/Users/jamieorc/.rvm/gems/ree-1.8.7-2011.12@pm-rails-3.1/gems/mongo_mapper-0.9.2/lib/mongo_mapper/plugins/embedded_callbacks.rb:28:in run_callbacks' /Users/jamieorc/.rvm/gems/ree-1.8.7-2011.12@pm-rails-3.1/gems/mongo_mapper-0.9.2/lib/mongo_mapper/plugins/embedded_callbacks.rb:23:inrun_callbacks'
/Users/jamieorc/.rvm/gems/ree-1.8.7-2011.12@pm-rails-3.1/gems/activesupport-3.1.4/lib/active_support/callbacks.rb:390:in _run_validation_callbacks' /Users/jamieorc/.rvm/gems/ree-1.8.7-2011.12@pm-rails-3.1/gems/activesupport-3.1.4/lib/active_support/callbacks.rb:81:insend'

repeated ad naseum.

Just to underscore the point, an object that was saved with Rails 2.3.14 can be loaded in a Rails 3.1.4 console, but "SystemStackError: stack level too deep" if I try to save the unchanged object (Ruby 1.9.3).

I checked the size of my BSON object. It's only about 200k, an order of magnitude smaller than what Mongodb can store.

It seems to be related to the number of EmbeddedDocuments.

First layer of EmbeddedDocs:
asu.collections.count => 80

Second layer total:
asu.collections.reduce(0){|num, c| num + c.monthly_views.count} => 1600

Although it's a lot of docs, it's still only a 200k BSON object and Rails 2.3.14/MM 0.8.6 handle this with no problem at al..

Looking forward to feedback.

---

Reply to this email directly or view it on GitHub:
#405

[bkeepers]

Duplicate of #265