Skip to content
This repository has been archived by the owner on Aug 2, 2019. It is now read-only.

ElasticSearch bottleneck for batch record creation #264

Open
leplatrem opened this issue Nov 9, 2014 · 1 comment
Open

ElasticSearch bottleneck for batch record creation #264

leplatrem opened this issue Nov 9, 2014 · 1 comment
Assignees

Comments

@leplatrem
Copy link
Contributor

Importing a model with 3000 inline records is 20x slower with ES enabled :|

➜  time node import.js                     
node import.js  0,19s user 0,05s system 12% cpu 1,841 total
➜  import-geojson  time node import.js
node import.js  0,17s user 0,05s system 0% cpu 37,006 total

See spiral-project/daybed.js#28

See refresh param https://github.com/spiral-project/daybed/blob/1.0.1/daybed/indexer.py#L117

See #213

@leplatrem
Copy link
Contributor Author

Without index refresh, it's only 10x slower :) (13secs)

I guess some time should be invested in handling events asynchronously instead...

@leplatrem leplatrem self-assigned this Nov 10, 2014
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

1 participant