Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Indexing: If versionMap is too large we should trigger refresh #6378

Closed
mikemccand opened this issue Jun 2, 2014 · 1 comment
Closed

Indexing: If versionMap is too large we should trigger refresh #6378

mikemccand opened this issue Jun 2, 2014 · 1 comment
Labels
>enhancement v1.3.0 v2.0.0-beta1

Comments

@mikemccand
Copy link
Contributor

In #6363 we are fixing versionMap to clear on refresh not flush, but even once we fix that, if the user sets a big refresh (or turns it off), and big xlog flush triggers, then the versionMap can grow unbounded. I think we should trigger refresh once versionMap is too large?
mikemccand added a commit to mikemccand/elasticsearch that referenced this issue Jun 9, 2014
@mikemccand
Indexing: force refresh when versionMap is using too much RAM
0bdb433 
If the user sets a high refresh interval, the versionMap can use
unbounded RAM.  I fixed LiveVersionMap to track its RAM used, and
trigger refresh if it's > 25% of IW's RAM buffer.  (We could add
another setting for this but we have so many settings already?).

I also fixed deletes to prune every index.gc_deletes/4 msec, and I
only save a delete tombstone if index.gc_deletes > 0.

I think we could expose the RAM used by versionMap somewhere
(Marvel?  _cat?), but we can do that separately ... I put a TODO.

Closes elastic#6378
@mikemccand mikemccand mentioned this issue Jun 9, 2014
Closed
@mikemccand
Copy link
Contributor Author

Fixed with #6443

@clintongormley clintongormley changed the title If versionMap is too large we should trigger refresh Indexing: If versionMap is too large we should trigger refresh Jul 16, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
>enhancement v1.3.0 v2.0.0-beta1
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Force refresh when versionMap is using too much RAM mikemccand/elasticsearch
2 participants
@clintongormley @mikemccand