New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Change default filter cache to 10% and circuit breaker to 60% #5990
Change default filter cache to 10% and circuit breaker to 60% #5990
Conversation
The defaults we have today in our data intensive memory structures don't properly add up to properly protected from potential OOM. The circuit breaker, today at 80%, aims at protecting from extensive field data loading. The default threshold today is too permissive and can still cause OOMs. The filter cache today is at 20%, and its too high when adding it to other limits we have, reduce it to 10%, which is still a big enough portion of the heap, yet provides improved safety measure.
LGTM. Are there perhaps any functional tests that exist to hammer these kind of settings? |
there aren't, but we discussed potentially doing static check to verify that out of the current limits we have, the sum of the defaults of them does not exceed a certain threshold. So if we change one, we don't go overboard with it. |
LGTM |
@kimchy What kind of loads do we expect this behavior to occur at (amounts of data and number/size of concurrent queries)? I might have some time this weekend to try to make a CI-style test for this behavior moving forward that we can use to test memory and such. |
The defaults we have today in our data intensive memory structures don't properly add up to properly protected from potential OOM. The circuit breaker, today at 80%, aims at protecting from extensive field data loading. The default threshold today is too permissive and can still cause OOMs. The filter cache today is at 20%, and its too high when adding it to other limits we have, reduce it to 10%, which is still a big enough portion of the heap, yet provides improved safety measure. closes #5990
…ta circuit Relates to elastic#5990
The defaults we have today in our data intensive memory structures don't properly add up to properly protected from potential OOM.
The circuit breaker, today at 80%, aims at protecting from extensive field data loading. The default threshold today is too permissive and can still cause OOMs.
The filter cache today is at 20%, and its too high when adding it to other limits we have, reduce it to 10%, which is still a big enough portion of the heap, yet provides improved safety measure.