-
-
Notifications
You must be signed in to change notification settings - Fork 74
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
refresh_interval #27
Comments
So.. after playing with this for some time I can remember why it was disabled in the first place. In my local config ( "elasticsearch": {
"settings": {
"index": {
"refresh_interval": "-1"
}
}
} I personally prefer this setting because it speeds up imports (how much I'm not quite sure) BUT at the expense of obviously not refreshing the index, ever. What it basically means is that for newbies starting out with Pelias, they can run a long import and then query the index to find 0 results, which could be confusing and costly to debug. Plus I'm pretty sure the segments are not written to disk, meaning data loss on failure. For those reasons I'm going to leave the setting at the default (which is 1s) and allow advanced users to set their own I would recommend that you either leave this at the default setting or make liberal use of |
For more info on the ES internals, I would recommend this chapter: http://www.elasticsearch.org/guide/en/elasticsearch/guide/current/inside-a-shard.html |
We used to set a very high
refresh_interval
but somewhere along the way the setting seems to have been removed.I'm assuming we can get a performance improvement by reducing refresh to something like
1m
instead of1s
, which would mean newly indexed docs wouldn't appear in the search for at most1m
but would mean ES isn't creating new segments all the time.Additionally this may save RAM as we are building less inverted indexes and less FSTs, plus the
commit point
log would be 60x smaller, resulting in a faster startup times on larger indexes.The text was updated successfully, but these errors were encountered: