... in TrackingConcurrentMergeScheduler.

These stack traces in the logs always precede extremely slow index times and the need to forcibly (with SIGKILL) restart Elasticsearch:

[19:43:57,851][WARN ][index.merge.scheduler ] [Droom, Doctor Anthony] [analytics][11] failed to merge
java.io.FileNotFoundException: _guwj_1.del
at org.elasticsearch.index.store.Store$StoreDirectory.fileLength(Store.java:448)
at org.apache.lucene.index.SegmentInfo.sizeInBytes(SegmentInfo.java:303)
at org.apache.lucene.index.MergePolicy$OneMerge.totalBytesSize(MergePolicy.java:174)
at org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:81)
at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:456)

Here are some who've been bitten by this bug:

https://groups.google.com/d/topic/elasticsearch/NCLWvGEz6dk/discussion
https://groups.google.com/d/topic/elasticsearch/7a0FKmqtbnM/discussion
http://elasticsearch-users.115913.n3.nabble.com/quot-Failed-to-merge-quot-java-io-FileNotFoundException-td3654491.html
http://elasticsearch-users.115913.n3.nabble.com/failed-to-mege-exception-td4021139.html

It looks like Lucene is being used inappropriately by Elasticsearch. See the response here:

https://issues.apache.org/jira/browse/LUCENE-3051?focusedCommentId=13576972&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13576972

We'll have to make this work with an older version of Elasticsearch, but I thought I'd raise a flag in case Lucene's suggested workaround (calling OneMerge.estimatedMergeBytes) is helpful.

Best,
Josh