NullPointerException during concurrent merges #3555
Assignees
Labels
Comments
|
The problem persists on master. |
ghost
assigned 
|
I think I know what the problem is.... your segment might have no docs in that field... I will try to write a test and come up with a patch |
s1monw
added a commit
that referenced
this issue
Aug 22, 2013
During segment merges FieldConsumers for all fields from the source segments are created. Yet, if all documents that have a value in a certain field are deleted / pruned during the merge the FieldConsumer will not be called at all causing the internally used FST Builder to return `null` from it's `build()` method. This means we can't store it and run potentially into a NPE. This commit adds handling for this corner case both during merge and during suggest phase since we also don't have a Lookup instance for this segments field. Closes #3555
|
@rdeaton can you try if that fixes your issue? Thanks |
|
Seems to be working. Thanks a bunch. |
|
thanks for verifying!!! |
mute
pushed a commit
to mute/elasticsearch
that referenced
this issue
Jul 29, 2015
During segment merges FieldConsumers for all fields from the source segments are created. Yet, if all documents that have a value in a certain field are deleted / pruned during the merge the FieldConsumer will not be called at all causing the internally used FST Builder to return `null` from it's `build()` method. This means we can't store it and run potentially into a NPE. This commit adds handling for this corner case both during merge and during suggest phase since we also don't have a Lookup instance for this segments field. Closes elastic#3555
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Looks like it may be related to the new suggestion stuff. I'm on 0.90.3, Java 7. Will be trying to reproduce on master shortly, but let me know if there's other stuff you need to debug.
[2013-08-21 22:02:05,166][WARN ][index.engine.robin ] [qanticharm] [quizlet][9] failed engine
org.apache.lucene.index.MergePolicy$MergeException: java.lang.NullPointerException
at org.elasticsearch.index.merge.scheduler.ConcurrentMergeSchedulerProvider$CustomConcurrentMergeScheduler.handleMergeException(ConcurrentMergeSchedulerProvider.java:99)
at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:518)
Caused by: java.lang.NullPointerException
at org.elasticsearch.search.suggest.completion.AnalyzingCompletionLookupProvider$1$1.finish(AnalyzingCompletionLookupProvider.java:134)
at org.elasticsearch.search.suggest.completion.Completion090PostingsFormat$SuggestFieldsConsumer$1.finish(Completion090PostingsFormat.java:152)
at org.apache.lucene.codecs.TermsConsumer.merge(TermsConsumer.java:204)
at org.apache.lucene.codecs.FieldsConsumer.merge(FieldsConsumer.java:72)
at org.apache.lucene.index.SegmentMerger.mergeTerms(SegmentMerger.java:365)
at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:98)
at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:3772)
at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3376)
at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:405)
at org.apache.lucene.index.TrackingConcurrentMergeScheduler.doMerge(TrackingConcurrentMergeScheduler.java:91)
at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:482)
[2013-08-21 22:02:05,180][WARN ][cluster.action.shard ] [qanticharm] sending failed shard for [quizlet][9], node[27LWb8rFQiafhZhgRH4vjg], [R], s[STARTED], reason [engine failure, message [MergeException[java.lang.NullPointerException]; nested: NullPointerException; ]]
The text was updated successfully, but these errors were encountered: