New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Recover or crash on initial mongo connection failure #3038

Closed
pward123 opened this Issue Nov 7, 2014 · 6 comments

Comments

Projects
None yet
3 participants
@pward123

pward123 commented Nov 7, 2014

I'm getting the error below when the mongodb is down when meteor first starts. Any chance we can get a fix like http://bites.goodeggs.com/post/35878004826/reconnecting-to-mongodb-when-mongoose-connect-fails-at/ ?

Exception in callback of async function: Error: failed to connect to [127.0.0.1:27017]
    at null.<anonymous> (/Users/pward/.meteor/packages/mongo/.1.0.8.1kr7rhx++os+web.browser+web.cordova/npm/node_modules/mongodb/lib/mongodb/connection/server.js:536:74)
    at emit (events.js:106:17)
    at null.<anonymous> (/Users/pward/.meteor/packages/mongo/.1.0.8.1kr7rhx++os+web.browser+web.cordova/npm/node_modules/mongodb/lib/mongodb/connection/connection_pool.js:150:15)
    at emit (events.js:98:17)
    at Socket.<anonymous> (/Users/pward/.meteor/packages/mongo/.1.0.8.1kr7rhx++os+web.browser+web.cordova/npm/node_modules/mongodb/lib/mongodb/connection/connection.js:516:10)
    at Socket.emit (events.js:95:17)
    at net.js:440:14
    at process._tickCallback (node.js:419:13)
@glasser

This comment has been minimized.

Member

glasser commented Dec 4, 2014

Can you provide the full context here? https://github.com/meteor/meteor/wiki/Contributing-to-Meteor#reporting-a-bug-in-meteor

Meteor apps that use the mongo package generally expect to be able to connect to Mongo at startup and crash the server otherwise (and should be run in a standard environment that knows how to deal with crashing processes). If your server is NOT crashing, that would be be concerning. Otherwise, it is working as intended.

@glasser glasser closed this Dec 4, 2014

@pward123

This comment has been minimized.

pward123 commented Dec 4, 2014

My app is currently running in a docker container and uses systemd to keep the container running. The server is NOT crashing. Here are the steps you can use to reproduce.

git clone https://github.com/meteor/simple-todos
cd simple-todos
meteor update
MONGO_URL=mongodb://bad_address:27017/todo meteor

The above will cause the error message in my original post to be displayed but the app remains running. It's just non-responsive. If you look at MUP, arunoda hacks around this problem by pinging the mongo instance until it's running and then starting the app.

This is actually indicative of a bigger problem (at least where my app is concerned). I can't find the module right now, but I traced it down to a common async callback handler that traps exceptions and just console logs the message then goes on about its business. I'd really like to see an environment switch that will force the app to crash on any unhanded exception inside a callback.

@glasser glasser reopened this Dec 5, 2014

@pward123

This comment has been minimized.

pward123 commented Dec 7, 2014

The mongo connect call is wrapped with bindEnvironment
https://github.com/meteor/meteor/blob/devel/packages/mongo/mongo_driver.js#L151

bindEnvironment ignores errors by calling Meteor._debug
https://github.com/meteor/meteor/blob/devel/packages/meteor/dynamics_nodejs.js#L92

@glasser

This comment has been minimized.

Member

glasser commented Dec 9, 2014

Hmm, looks like I broke this in 0.8.1 with 24a0006

@BlackYoup

This comment has been minimized.

BlackYoup commented Dec 24, 2014

Hi, I'm getting this error too using an external mongoDB database.
Any news on this ?

@glasser glasser changed the title from Recover from initial mongo connection failure to Recover or crash on initial mongo connection failure Jan 14, 2015

@glasser glasser added this to the Winter 2015 Mongo Maintenance milestone Jan 21, 2015

glasser added a commit that referenced this issue Jan 30, 2015

Make MongoConnection constructor always wait
Summary:
Previously, the MongoConnection constructor (which gets called
implicitly the first time you make a Mongo.Collection) would wait for a
successful connection before returning ... if an oplog URL is supplied.
If not in oplog mode, it would return before the connection is
successful, and the first subsequent calls that actually tried to do
something with the DB would block instead.

Having an inconsistent API that is sometimes sync and sometimes async is
not very clear.  Moreover, error handling from connect was strange.
Prior to 0.8.1 (24a0006), connect errors would be thrown
uncatchably (ie, from an unrelated context).  Starting with 0.8.1,
connect errors would effectively be ignored due to the new
bindEnvironment (they'd be logged, but all code would continue
normally).

It's pretty important that startup connection errors crash the server
process instead of letting it keep running unproductively.  And it would
also be nice if those errors could be caught (if you're making a
MongoConnection in your own code).  So this change ensures that
connection errors get thrown by the MongoConnection constructor.

Fixes #3038.

Test Plan: new unit test. manual test with MONGO_URL=xxx

Reviewers: ben

Subscribers: justinsb

Differential Revision: https://phabricator.meteor.io/D19

@glasser glasser closed this in dde3924 Jan 30, 2015

@glasser

This comment has been minimized.

Member

glasser commented Jan 30, 2015

This should be fixed on devel.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment