Index creation fails after creating the first background index #1365

Closed
hvrauhal opened this Issue Feb 28, 2013 · 11 comments

Projects

None yet

3 participants

@hvrauhal

Index creation fails with error MongoError: cannot add index with a background operation in progress for all but the first index if there are many indexes defined for a schema and the indexes take some time to create.

Run https://github.com/hvrauhal/mongoosebug to reproduce.

This is related to issue #1059

@aheckmann
Collaborator

"With mongo 2.2.3 and mongoose 3.5.4"

@aheckmann
Collaborator

One of the best bug reports ever! Thanks. Yet I still cannot reproduce. Any other tips for me?

@aheckmann
Collaborator

ah ok, reproduced it. thanks

@hvrauhal

Needed more kittens for the index creation to take a long enough time?

@aheckmann
Collaborator

Somewhat, I needed to add all of the docs before triggering index creation.

On Thu, Feb 28, 2013 at 10:43 AM, Heikki Rauhala
notifications@github.comwrote:

Needed more kittens for the index creation to take a long enough time?


Reply to this email directly or view it on GitHubhttps://github.com/LearnBoost/mongoose/issues/1365#issuecomment-14249647
.

Aaron
@aaronheckmann https://twitter.com/#!/aaronheckmann

@addisonj
addisonj commented Mar 2, 2013

Great bug report.

We have been bitten by this a few times but weren't sure where the fault lies. Disappearing indexes is not a very fun situation.

Any chance I could be of help or a possible ETA on this?

@aheckmann
Collaborator

@addisonj indexes never "disappeared". Listen to the index event of your model to be informed when the indexing attempts complete. The error, if any, is passed along.

@aheckmann
Collaborator

@addisonj if you'd like to submit the fix the function is in lib/model.js called ensureIndexes. Add a test that reproduces to test/model.indexing.test.js

If you don't get to it over the weekend I'll get it sometime next week.

@addisonj
addisonj commented Mar 2, 2013

@aheckmann apologies for not making that clear. We have some import jobs that drop a rebuild indexes and have seen this behavior. I am glad to have that mystery answered (we thought it was a mongodb issue).

Thanks for the hint on the event though. I can use that as a workaround for now.

Awesome project BTW and apologies again if that first message came off a bit harsh!

@aheckmann
Collaborator

@addisonj no worries, you didn't come off harsh. sorry if I gave that impression. Maybe I shouldn't write comments while walking in traffic :)

@aheckmann
Collaborator

Currently MongoDB does not permit multiple background indexes running. I've confirmed executing these indexes one-by-one fixes the problem. The commit will land soon.

If this is a production setup, its best to disable auto index creation in production due to the potential performance impact. You can grab the list of indexes from your Models schema like so: var indexes = YourModel.schema.indexes(), then follow the recommendations for creating them in production.

FYI: in my testing MongoDB 2.4 doesn't have this limitation.

Related MongoDB tickets:
https://jira.mongodb.org/browse/SERVER-2771
https://jira.mongodb.org/browse/SERVER-2374

@aheckmann aheckmann added a commit that referenced this issue Mar 2, 2013
@aheckmann aheckmann test; added for #1365 ffb1d2e
@aheckmann aheckmann added a commit that closed this issue Mar 2, 2013
@aheckmann aheckmann fixed; multiple bg index creation
In MongoDB < 2.4, multiple background indexes creation is not
well supported. We now create indexes one-by-one.

closes #1365
12574d9
@aheckmann aheckmann closed this in 12574d9 Mar 2, 2013
@aheckmann aheckmann added a commit that referenced this issue Mar 15, 2013
@aheckmann aheckmann fixed; multiple bg index creation
In MongoDB < 2.4, multiple background indexes creation is not
well supported. We now create indexes one-by-one.

closes #1365
98fe31c
@aheckmann aheckmann added a commit that referenced this issue Mar 15, 2013
@aheckmann aheckmann test; added for #1365 33ad51c
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment