Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Crippling CPU usage for upserts #504

Open
tsturzl opened this issue Apr 11, 2017 · 1 comment
Open

Crippling CPU usage for upserts #504

tsturzl opened this issue Apr 11, 2017 · 1 comment

Comments

@tsturzl
Copy link

tsturzl commented Apr 11, 2017

We use upserts to update data pulled from our API for offline storage in a electron client app. However, we are upserting a collection of thousands of relatively small documents and I'm seeing the CPU usage spike for a long duration while upserting this data into an nedb database. It becomes so intense on the CPU that the entire application stops responding as a result(sometimes crashing the chrome dev tools). It seems to only happen starting with this collection(its probably the largest) and then every collection after that seems to just slow to a crawl and consume 100% CPU for several minutes.

So additional info the upsert query is leveraging indices, because we are querying the document id for the upsert. We do a single upsert at a time for each document, because as far as I'm aware there is no means of bulk upserts. After running a CPU profile I've found that the most time is spent indexing into an AVLTree(BST), so I feel it may have something to do with indexing.

@tsturzl
Copy link
Author

tsturzl commented Apr 11, 2017

I can upload the CPU profile if they'd be helpful

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant