Cursor.observe() in Server.startup() emits "Error: Can't call yield in a noYieldsAllowed block" #507

Closed
paulreimer opened this Issue Nov 23, 2012 · 11 comments

Projects

None yet

3 participants

@paulreimer

Previously working code such as this (in server/main.js):

Meteor.startup(function() {
  CollectionX.find().observe({
    added: function (doc, idx) {
    }
});

Now produces an error and the following trace:

/PATH/TO/METEOR/PROJECT/.meteor/local/build/server/server.js:152
  }).run();
     ^
Error: Can't call yield in a noYieldsAllowed block!
    at Function.Meteor._noYieldsAllowed.Fiber.yield (app/packages/meteor/fiber_helpers.js:13:11)
    at Function.wait (/usr/lib/meteor/lib/node_modules/fibers/future.js:111:14)
    at Object.Future.wait (/usr/lib/meteor/lib/node_modules/fibers/future.js:321:10)
    at _Mongo._createSynchronousCursor (app/packages/mongo-livedata/mongo_driver.js:369:23)
    at _Mongo._observe (app/packages/mongo-livedata/mongo_driver.js:493:14)
    at Object.Meteor._noYieldsAllowed (app/packages/meteor/fiber_helpers.js:17:12)
    at _Mongo._observe (app/packages/mongo-livedata/mongo_driver.js:485:10)
    at Cursor.observe (app/packages/mongo-livedata/mongo_driver.js:338:22)
    at app/server/main.js:4:25
    at run (/PATH/TO/METEOR/PROJECT/.meteor/local/build/server/server.js:144:63)

This worked as expected (by me, at least) in 0.5.0 and produces the error above in http://install.meteor.com [0.5.1] as well at the latest git as of right now.

@paulreimer

In case I was using a (newly) deprecated pattern for something common, my intention with observe({added}):

Context: another (non-meteor) app is pushing into CollectionX, with a unique string key referring to an entity of CollectionY.

  1. (optional) If a CollectionY is not found with the matching string key (e.g. using findOne), insert a new one.
  2. Add a reference to the appropriate CollectionY _id, for every observed CollectionX (e.g. CollectionX.update({_id: doc._id}).
@glasser
Member
glasser commented Nov 23, 2012

This is surprising. Is it possible to make a full reproduction instance
that I can try out myself (ie, a series of steps starting with "git clone")?
On Nov 23, 2012 5:07 PM, "Paul Reimer" notifications@github.com wrote:

Previously working code such as this (in server/main.js):

Meteor.startup(function() {
CollectionX.find().observe({
added: function (doc, idx) {
}});

Now produces an error and the following trace:

/home/paulreimer/Development/wits/wits-webapp/.meteor/local/build/server/server.js:152
}).run();
^
Error: Can't call yield in a noYieldsAllowed block!
at Function.Meteor._noYieldsAllowed.Fiber.yield (app/packages/meteor/fiber_helpers.js:13:11)
at Function.wait (/usr/lib/meteor/lib/node_modules/fibers/future.js:111:14)
at Object.Future.wait (/usr/lib/meteor/lib/node_modules/fibers/future.js:321:10)
at _Mongo._createSynchronousCursor (app/packages/mongo-livedata/mongo_driver.js:369:23)
at _Mongo._observe (app/packages/mongo-livedata/mongo_driver.js:493:14)
at Object.Meteor._noYieldsAllowed (app/packages/meteor/fiber_helpers.js:17:12)
at _Mongo._observe (app/packages/mongo-livedata/mongo_driver.js:485:10)
at Cursor.observe (app/packages/mongo-livedata/mongo_driver.js:338:22)
at app/server/main.js:4:25
at run (/home/paulreimer/Development/wits/wits-webapp/.meteor/local/build/server/server.js:144:63)

This worked as expected (by me, at least) in 0.5.0 and produces the error
above in stable 0.5.1 as well at the latest git as of right now.


Reply to this email directly or view it on GitHubhttps://github.com/meteor/meteor/issues/507.

@paulreimer

Starting from an empty meteor project, this is a simple example that demonstrates the problem. Only one file is needed:

server/main.js

with contents as follows:

CollectionX = new Meteor.Collection("CollectionX");
CollectionY = new Meteor.Collection("CollectionY");

Meteor.startup(function() {
  // code to run on server at startup
  CollectionX.find().observe({
    added: function (doc, idx) {
      var ref = CollectionY.findOne({name: doc.y.name});
      if (typeof (ref) == 'undefined')
        CollectionY.insert({name: doc.y.name});

      CollectionX.update({_id: doc._id},
                         {$set: {"y.ref_id": ref._id}},
                         {multi: false});
    }
    // don't care about moved, removed or changed
  });
});

to produce a similar error/trace as my example above. In fact, the added callback in observe() can be a simple as

function (doc, idx) {}

but I have left a minimal example to show my intent.

@glasser
Member
glasser commented Nov 24, 2012

OK, I see the issue. Thanks!

The problem is that _Mongo::_createSynchronousCursor can yield (via _withCollection, and I'm calling it in a place where I've added assertions that it can't yield. But in practice, the db.collection call doesn't yield, and the only time this actually yields is if we're calling this function while still doing our initial connection to Mongo (and the collection_queue is involved). eg, during startup.

I should just make sure to not call _createSynchronousCursor during the "no yields" block.

@glasser glasser added a commit that closed this issue Nov 24, 2012
@glasser glasser Fix use of cursor.observe at server-side startup.
A block that wasn't allowed to yield called _Mongo::_createSynchronousCursor,
which certainly can yield (it waits on a Future) but doesn't actually end up
needing to yield unless it is called during the initial connection to
Mongo... eg, during startup.

Fixes #507 (reported by paulreimer).
d073e5d
@glasser glasser closed this in d073e5d Nov 24, 2012
@glasser
Member
glasser commented Nov 24, 2012

I fixed this on the devel branch.

I'm not sure how many users this will cause a problem for. If we get more reports of users running into this error (eg, comments on this bug) I'm happy to go through the process of releasing a 0.5.2 that just contains this fix.

@javajosh

Thanks for fixing - this bug bit me hard today (but on the bright side I got to roll my own context dependency - nice idea btw). But yes, I think that observing a collection on the server is a very important function, and you should release this fix asap.

@glasser
Member
glasser commented Nov 27, 2012

@javajosh can you confirm that my commit fixes the issue that you saw?

@paulreimer

I can confirm that cherry-picking this commit d073e5d into master solved my server-side observer pattern problem.
I'm still blocked from using mainline as I require pull request #279, but 0.5.2 with this fix might remove confusion for others.

@javajosh

Yeah, I went ahead and just patched it by hand, and it works great. Def would have saved me a few hours of coding if it had worked the first time, but I don't regret exploring the stack a bit more.

BTW what is a convenient way for me to measure how often an observed query is being executed? I am, of course, somewhat wary of the performance implications of cursor.observe.

@glasser glasser added a commit that referenced this issue Nov 27, 2012
@glasser glasser Fix use of cursor.observe at server-side startup.
A block that wasn't allowed to yield called _Mongo::_createSynchronousCursor,
which certainly can yield (it waits on a Future) but doesn't actually end up
needing to yield unless it is called during the initial connection to
Mongo... eg, during startup.

Fixes #507 (reported by paulreimer).
0757e96
@glasser
Member
glasser commented Nov 27, 2012

OK, I've just released Meteor 0.5.2, which consists of 0.5.1 plus this fix.

@javajosh There's no direct way to do so right now; I have some ideas in my head about adding a simple internal stats smart package to Meteor but haven't had time to play with it yet.

@javajosh

@glasser Thanks David. meteor update and works like a champ.

@haircuttedfreak haircuttedfreak pushed a commit to tastemade/meteor that referenced this issue Jan 13, 2013
@glasser glasser Fix use of cursor.observe at server-side startup.
A block that wasn't allowed to yield called _Mongo::_createSynchronousCursor,
which certainly can yield (it waits on a Future) but doesn't actually end up
needing to yield unless it is called during the initial connection to
Mongo... eg, during startup.

Fixes #507 (reported by paulreimer).
62a149c
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment