Switch memory breaking in process requests to use CircuitBreaker infrastructure #6129
Assignees
Comments
|
Closing in favour of #6739 |
dakrone
added a commit
to dakrone/elasticsearch
that referenced
this issue
Jul 28, 2014
Adds a breaker for request BigArrays, which are used for parent/child queries as well as some aggregations. Certain operations like Netty HTTP responses and transport responses increment the breaker, but will not trip. This also changes the output of the nodes' stats endpoint to show the parent breaker as well as the fielddata and request breakers. There are a number of new settings for breakers now: `indices.breaker.total.limit`: starting limit for all memory-use breaker, defaults to 70% `indices.breaker.fielddata.limit`: starting limit for fielddata breaker, defaults to 60% `indices.breaker.fielddata.overhead`: overhead for fielddata breaker estimations, defaults to 1.03 (the fielddata breaker settings also use the backwards-compatible setting `indices.fielddata.breaker.limit` and `indices.fielddata.breaker.overhead`) `indices.breaker.request.limit`: starting limit for request breaker, defaults to 40% `indices.breaker.request.overhead`: request breaker estimation overhead, defaults to 1.0 The breaker service infrastructure is now generic and opens the path to adding additional circuit breakers in the future. Fixes elastic#6129 Conflicts: src/main/java/org/elasticsearch/index/fielddata/IndexFieldData.java src/main/java/org/elasticsearch/index/fielddata/IndexFieldDataService.java src/main/java/org/elasticsearch/index/fielddata/RamAccountingTermsEnum.java src/main/java/org/elasticsearch/index/fielddata/ordinals/GlobalOrdinalsBuilder.java src/main/java/org/elasticsearch/index/fielddata/ordinals/InternalGlobalOrdinalsBuilder.java src/main/java/org/elasticsearch/index/fielddata/plain/AbstractIndexOrdinalsFieldData.java src/main/java/org/elasticsearch/index/fielddata/plain/DisabledIndexFieldData.java src/main/java/org/elasticsearch/index/fielddata/plain/IndexIndexFieldData.java src/main/java/org/elasticsearch/index/fielddata/plain/NonEstimatingEstimator.java src/main/java/org/elasticsearch/index/fielddata/plain/PackedArrayIndexFieldData.java src/main/java/org/elasticsearch/index/fielddata/plain/ParentChildIndexFieldData.java src/main/java/org/elasticsearch/index/fielddata/plain/SortedSetDVOrdinalsIndexFieldData.java src/main/java/org/elasticsearch/node/internal/InternalNode.java src/test/java/org/elasticsearch/index/aliases/IndexAliasesServiceTests.java src/test/java/org/elasticsearch/index/codec/CodecTests.java src/test/java/org/elasticsearch/index/fielddata/AbstractFieldDataTests.java src/test/java/org/elasticsearch/index/fielddata/IndexFieldDataServiceTests.java src/test/java/org/elasticsearch/index/mapper/MapperTestUtils.java src/test/java/org/elasticsearch/index/query/IndexQueryParserFilterCachingTests.java src/test/java/org/elasticsearch/index/query/SimpleIndexQueryParserTests.java src/test/java/org/elasticsearch/index/query/guice/IndexQueryParserModuleTests.java src/test/java/org/elasticsearch/index/search/FieldDataTermsFilterTests.java src/test/java/org/elasticsearch/index/search/child/ChildrenConstantScoreQueryTests.java src/test/java/org/elasticsearch/index/similarity/SimilarityTests.java
dakrone
added a commit
that referenced
this issue
Jul 28, 2014
Adds a breaker for request BigArrays, which are used for parent/child queries as well as some aggregations. Certain operations like Netty HTTP responses and transport responses increment the breaker, but will not trip. This also changes the output of the nodes' stats endpoint to show the parent breaker as well as the fielddata and request breakers. There are a number of new settings for breakers now: `indices.breaker.total.limit`: starting limit for all memory-use breaker, defaults to 70% `indices.breaker.fielddata.limit`: starting limit for fielddata breaker, defaults to 60% `indices.breaker.fielddata.overhead`: overhead for fielddata breaker estimations, defaults to 1.03 (the fielddata breaker settings also use the backwards-compatible setting `indices.fielddata.breaker.limit` and `indices.fielddata.breaker.overhead`) `indices.breaker.request.limit`: starting limit for request breaker, defaults to 40% `indices.breaker.request.overhead`: request breaker estimation overhead, defaults to 1.0 The breaker service infrastructure is now generic and opens the path to adding additional circuit breakers in the future. Fixes #6129 Conflicts: src/main/java/org/elasticsearch/index/fielddata/IndexFieldData.java src/main/java/org/elasticsearch/index/fielddata/IndexFieldDataService.java src/main/java/org/elasticsearch/index/fielddata/RamAccountingTermsEnum.java src/main/java/org/elasticsearch/index/fielddata/ordinals/GlobalOrdinalsBuilder.java src/main/java/org/elasticsearch/index/fielddata/ordinals/InternalGlobalOrdinalsBuilder.java src/main/java/org/elasticsearch/index/fielddata/plain/AbstractIndexOrdinalsFieldData.java src/main/java/org/elasticsearch/index/fielddata/plain/DisabledIndexFieldData.java src/main/java/org/elasticsearch/index/fielddata/plain/IndexIndexFieldData.java src/main/java/org/elasticsearch/index/fielddata/plain/NonEstimatingEstimator.java src/main/java/org/elasticsearch/index/fielddata/plain/PackedArrayIndexFieldData.java src/main/java/org/elasticsearch/index/fielddata/plain/ParentChildIndexFieldData.java src/main/java/org/elasticsearch/index/fielddata/plain/SortedSetDVOrdinalsIndexFieldData.java src/main/java/org/elasticsearch/node/internal/InternalNode.java src/test/java/org/elasticsearch/index/aliases/IndexAliasesServiceTests.java src/test/java/org/elasticsearch/index/codec/CodecTests.java src/test/java/org/elasticsearch/index/fielddata/AbstractFieldDataTests.java src/test/java/org/elasticsearch/index/fielddata/IndexFieldDataServiceTests.java src/test/java/org/elasticsearch/index/mapper/MapperTestUtils.java src/test/java/org/elasticsearch/index/query/IndexQueryParserFilterCachingTests.java src/test/java/org/elasticsearch/index/query/SimpleIndexQueryParserTests.java src/test/java/org/elasticsearch/index/query/guice/IndexQueryParserModuleTests.java src/test/java/org/elasticsearch/index/search/FieldDataTermsFilterTests.java src/test/java/org/elasticsearch/index/search/child/ChildrenConstantScoreQueryTests.java src/test/java/org/elasticsearch/index/similarity/SimilarityTests.java
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Followup for the work from #6050, we should use the circuit breaker infrastructure for the memory limiting.
The text was updated successfully, but these errors were encountered: