New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix memory leak when percolator uses bitset or field data cache #24115

Merged
merged 1 commit into from Apr 26, 2017

Conversation

Projects
None yet
3 participants
@martijnvg
Member

martijnvg commented Apr 14, 2017

The percolator doesn't close the IndexReader of the memory index any more.
Prior to 2.x the percolator had its own SearchContext (PercolatorContext) that did this,
but that was removed when the percolator was refactored as part of the 5.0 release.

I think an alternative way to fix this is to let percolator not use the bitset and fielddata caches,
that way we prevent the memory leak.

Adding a WIP label to this as I'm not happy with the current test. It is not easy to test that we don't use the bitset or fielddata cache for the percolator, because non percolator operations may use these caches, which is valid.

PR for #24108

@Override
@SuppressWarnings("unchecked")
public <IFD extends IndexFieldData<?>> IFD getForField(MappedFieldType fieldType) {
IndexFieldData.Builder builder = fieldType.fielddataBuilder();

This comment has been minimized.

@martijnvg

martijnvg Apr 14, 2017

Member

Still need to think how to add a test for this too...

@jpountz

This comment has been minimized.

Contributor

jpountz commented Apr 19, 2017

Ideally we would use the SearchContext.addReleasable API in order to tell ES to close the index reader once the request is processed. However it is not easy today to find the right place to get access to the search context...

@martijnvg

This comment has been minimized.

Member

martijnvg commented Apr 19, 2017

@jpountz What do you think about adding a QueryShardContext#addReleasable(...) method that delegates to SearchContext#addReleasable(...)? Then we can close the index readers the percolate query builder creates.

@jpountz

This comment has been minimized.

Contributor

jpountz commented Apr 19, 2017

If you can find a way to do that, that would be great!

@martijnvg

This comment has been minimized.

Member

martijnvg commented Apr 19, 2017

@jpountz what do you think of the current approach in this pr?

@clintongormley clintongormley removed the v5.3.1 label Apr 20, 2017

@jpountz

Thanks for looking into it, I think what you are suggesting would work, we just need to make the wiring a bit nicer?

core/src/main/java/org/elasticsearch/index/query/QueryShardContext.java Outdated
public void setDelegate(Consumer<Releasable> delegate) {
this.delegate = delegate;
}

This comment has been minimized.

@jpountz

jpountz Apr 24, 2017

Contributor

should it be a constructor arg?

This comment has been minimized.

@martijnvg

martijnvg Apr 24, 2017

Member

done - it is now a final field

core/src/main/java/org/elasticsearch/index/query/QueryShardContext.java Outdated
@@ -98,6 +100,10 @@ public void setTypes(String... types) {
private NestedScope nestedScope;
private boolean isFilter;
private Consumer<Releasable> delegate = (releasable) -> {

This comment has been minimized.

@jpountz

jpountz Apr 24, 2017

Contributor

let's try to make it final and give it a better name?

This comment has been minimized.

@jpountz

jpountz Apr 24, 2017

Contributor

I don't have suggestions though...

This comment has been minimized.

@martijnvg

martijnvg Apr 24, 2017

Member

Me neither... I've named it now releaseDelegator

@martijnvg martijnvg added review and removed WIP labels Apr 24, 2017

core/src/main/java/org/elasticsearch/index/IndexService.java Outdated
@@ -472,11 +473,19 @@ public IndexSettings getIndexSettings() {
* {@link IndexReader}-specific optimizations, such as rewriting containing range queries.
*/
public QueryShardContext newQueryShardContext(int shardId, IndexReader indexReader, LongSupplier nowInMillis) {

This comment has been minimized.

@jpountz

jpountz Apr 24, 2017

Contributor

should we just remove this one completely?

This comment has been minimized.

@martijnvg

martijnvg Apr 24, 2017

Member

That method is still being used in other places. For example in MetaDataCreateIndexService and MetaDataIndexAliasesService. In these places we don't have access to a search context and we test there if alias filters parse correctly, which means we will not be supporting percolate queries there, which I think is ok.

This comment has been minimized.

@martijnvg

martijnvg Apr 25, 2017

Member

I removed this method, wdyt?

@martijnvg

This comment has been minimized.

Member

martijnvg commented Apr 26, 2017

I've reverted this PR back to the initial approach (replacing field data and bitset cache in a custom QueryShardContext implementation) that was used to fix the problem. The change that added infrastructure for QueryShardContext#addReleasable(...) that allowed closing in-memory index is invasive and considering that this had to be backported it is better to use the initial fix as that change is mostly in one place. Closing the in-memory index is the right approach, but it should be explored in a followup change, in the meantime this change will fix the reported bug.

[percolator] Fix memory leak when percolator uses bitset or field dat…
…a cache.

The percolator doesn't close the IndexReader of the memory index any more.
Prior to 2.x the percolator had its own SearchContext (PercolatorContext) that did this,
but that was removed when the percolator was refactored as part of the 5.0 release.

I think an alternative way to fix this is to let percolator not use the bitset and fielddata caches,
that way we prevent the memory leak.

Closes #24108

@martijnvg martijnvg merged commit c17de49 into elastic:master Apr 26, 2017

1 of 2 checks passed

elasticsearch-ci Build started sha1 is merged.
Details
CLA Commit author is a member of Elasticsearch
Details

@martijnvg martijnvg added the v5.5.0 label Apr 26, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment