Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don't try to compute completion stats on a reader after we already closed it #18094

Merged
merged 1 commit into from May 2, 2016

Conversation

mikemccand
Copy link
Contributor

This can cause AlreadyClosedException and can also crash your JVM if mmapfs is in use and Lucene's best-effort check to catch this illegal usage fails, in 2.3.2.

I'll also fix this in 5.0.0, but there the bug is more harmless (computes stats twice, but without causing ACE/possible JVM crash).

@rmuir
Copy link
Contributor

rmuir commented May 2, 2016

+1

@jasontedor
Copy link
Member

LGTM.

@mikemccand
Copy link
Contributor Author

Looks like the bug first appeared in 2.3.0.

@mikemccand mikemccand merged commit 61cb1fc into elastic:2.x May 2, 2016
mikemccand pushed a commit that referenced this pull request May 2, 2016
Don't try to compute completion stats on a reader after we already closed it
mikemccand pushed a commit that referenced this pull request May 2, 2016
Don't try to compute completion stats on a reader after we already closed it

Conflicts:
	core/src/main/java/org/elasticsearch/index/shard/IndexShard.java
@pickypg pickypg removed the review label May 2, 2016
@debraj-manna
Copy link

@mikemccand @jasontedor - What is the impact of this fix if we are not using mmapfs. I am also seeing below exception and I am confused after seeing the below two discussion whether this can be ignored (even though this is showing up as DEBUG in logs )

The exception is like below

[2018-01-23 09:51:43,240][DEBUG][action.admin.cluster.node.stats] [Redneck] failed to execute on node [VBrl3cgpR2-ztgIpBLugTw]
RemoteTransportException[[Redneck][172.35.0.191:9300][cluster:monitor/nodes/stats[n]]]; nested: AlreadyClosedException[this IndexReader cannot be used anymore as one of its child readers was closed];
Caused by: org.apache.lucene.store.AlreadyClosedException: this IndexReader cannot be used anymore as one of its child readers was closed
        at org.apache.lucene.index.IndexReader.ensureOpen(IndexReader.java:279)
        at org.apache.lucene.index.FilterLeafReader.fields(FilterLeafReader.java:415)
        at org.elasticsearch.search.suggest.completion.Completion090PostingsFormat.completionStats(Completion090PostingsFormat.java:333)
        at org.elasticsearch.index.shard.IndexShard.completionStats(IndexShard.java:765)
        at org.elasticsearch.action.admin.indices.stats.CommonStats.<init>(CommonStats.java:164)
        at org.elasticsearch.indices.IndicesService.stats(IndicesService.java:253)
        at org.elasticsearch.node.service.NodeService.stats(NodeService.java:158)
        at org.elasticsearch.action.admin.cluster.node.stats.TransportNodesStatsAction.nodeOperation(TransportNodesStatsAction.java:82)
        at org.elasticsearch.action.admin.cluster.node.stats.TransportNodesStatsAction.nodeOperation(TransportNodesStatsAction.java:44)
        at org.elasticsearch.action.support.nodes.TransportNodesAction.nodeOperation(TransportNodesAction.java:92)
        at org.elasticsearch.action.support.nodes.TransportNodesAction$NodeTransportHandler.messageReceived(TransportNodesAction.java:230)
        at org.elasticsearch.action.support.nodes.TransportNodesAction$NodeTransportHandler.messageReceived(TransportNodesAction.java:226)
        at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:75)
        at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:376)
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

We are using 2.3.1 and in the process of upgrading to the latest 6. I know we are using a very old version of ES. This exceptions we started seeing in our production deployment. It will be helpful if you can provide some insight whether it needs serious attention.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants