Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change default filter cache to 10% and circuit breaker to 60% #5990

Closed

Conversation

kimchy
Copy link
Member

@kimchy kimchy commented Apr 29, 2014

The defaults we have today in our data intensive memory structures don't properly add up to properly protected from potential OOM.
The circuit breaker, today at 80%, aims at protecting from extensive field data loading. The default threshold today is too permissive and can still cause OOMs.
The filter cache today is at 20%, and its too high when adding it to other limits we have, reduce it to 10%, which is still a big enough portion of the heap, yet provides improved safety measure.

The defaults we have today in our data intensive memory structures don't properly add up to properly protected from potential OOM.
The circuit breaker, today at 80%, aims at protecting from extensive field data loading. The default threshold today is too permissive and can still cause OOMs.
 The filter cache today is at 20%, and its too high when adding it to other limits we have, reduce it to 10%, which is still a big enough portion of the heap, yet provides improved safety measure.
@pickypg
Copy link
Member

pickypg commented Apr 29, 2014

LGTM. Are there perhaps any functional tests that exist to hammer these kind of settings?

@kimchy
Copy link
Member Author

kimchy commented Apr 29, 2014

there aren't, but we discussed potentially doing static check to verify that out of the current limits we have, the sum of the defaults of them does not exceed a certain threshold. So if we change one, we don't go overboard with it.

@jpountz
Copy link
Contributor

jpountz commented Apr 30, 2014

LGTM

@pickypg
Copy link
Member

pickypg commented Apr 30, 2014

@kimchy What kind of loads do we expect this behavior to occur at (amounts of data and number/size of concurrent queries)? I might have some time this weekend to try to make a CI-style test for this behavior moving forward that we can use to test memory and such.

@kimchy kimchy closed this in c9f1792 May 4, 2014
kimchy added a commit that referenced this pull request May 4, 2014
The defaults we have today in our data intensive memory structures don't properly add up to properly protected from potential OOM.
The circuit breaker, today at 80%, aims at protecting from extensive field data loading. The default threshold today is too permissive and can still cause OOMs.
 The filter cache today is at 20%, and its too high when adding it to other limits we have, reduce it to 10%, which is still a big enough portion of the heap, yet provides improved safety measure.
 closes #5990
@kimchy kimchy deleted the better_filter_circuitbreaker_defaults branch May 4, 2014 13:39
jpountz added a commit that referenced this pull request May 6, 2014
jpountz added a commit that referenced this pull request May 6, 2014
neilschelly added a commit to neilschelly/elasticsearch that referenced this pull request May 7, 2014
@clintongormley clintongormley added the :Core/Infra/Settings Settings infrastructure and APIs label Jun 6, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants