New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
NPE in BytesRefOrdValComparator #3470
Comments
Thanks for opening this! This is in-fact a bug that is triggered depending on how sparse your data in that field is. I will come up with a patch very soon and I try to increase the test coverage for this to make sure we are not running into this again. |
This test triggers a MultiDoc / MultiOrds in-memory representation even if the field is not multivalued Relates to elastic#3470
MultiOrdinals.MultiDocs returned 'null' ordinals which caused a NPE if the field was single valued and would allow a significantly smaller in memory representation than single packed int ordinals. Closes #3470
This test triggers a MultiDoc / MultiOrds in-memory representation even if the field is not multivalued Relates to #3470
thanks for reporting this so quickly! |
Hey, thank YOU for fixing it so quickly! ES is an amazing project, I'm just glad I can help. --- Original Message --- From: "Simon Willnauer" notifications@github.com thanks for reporting this so quickly! Reply to this email directly or view it on GitHub: |
Many thanks Simon! that error filled our log!! +1 |
MultiOrdinals.MultiDocs returned 'null' ordinals which caused a NPE if the field was single valued and would allow a significantly smaller in memory representation than single packed int ordinals. Closes elastic#3470
Great guys, when you would be able to release this bug. We are using this very heavily. +1 |
@tarunjangra it's coming very soon I hope we can move towards a release next week or so. |
@s1monw Yes, we'd love a fix for this bug. We're using it heavily at Yoursports.com http://techcrunch.com/2012/10/11/sports-section-2-0-yoursports-launches-its-ambitious-project-to-build-the-facebook-of-sports/ |
@s1monw i would like to share this behaviour. Query i am doing: And result i am getting: QueryPhaseExecutionException[[es_index][11]: query[ConstantScore(cache(org.elasticsearch.index.search.nested.NonNestedDocsFilter@953b6c6c))],from[0],size[10],sort[<custom:"title.untouched": org.elasticsearch.index.fielddata.fieldcomparator.BytesRefFieldComparatorSource@1ca8b405>!]: Query Failed [Failed to execute main query]]; nested: NullPointerException; Some thing interesting i just noticed. I am also getting some stats about Shards. total: 15 But yesterday i was getting total: 15 the question is, why these stats are changing? Is that some how usual ? Thank you. |
@tarunjangra this issue depends on a lot of things if your field data implementation uses the "non-broken" code paths due to segment merges you won't see the failure on all shards so this can change over time. |
Is there a known workaround in the meantime? I was also using not_analyzed for sorting, and that's breaking with an NPE in BytesRefFieldComparatorSource (I'm assuming it's the same issue). I tried using keyword analysis instead but got the same failures. |
@s1monw Yep: It make sense. We are waiting for next elasticsearch release. Would like to recognise all elasticsearch team for such a great community efforts. Thank you. |
Hi I am searching with following query: { After all I get into this exception: QueryPhaseExecutionException[[activity][0]: query[ConstantScore(+QueryWrapperFilter(status:published) +rate:[0 TO 5])],from[1],size[5],sort[<custom:"activity_id": org.elasticsearch.index.fielddata.fieldcomparator.LongValuesComparatorSource@bda96b>!]: Query Failed [Failed to execute main query]]; nested: ElasticSearchException[java.lang.NumberFormatException: Invalid shift value in prefixCoded bytes (is encoded value really an INT?)]; nested: UncheckedExecutionException[java.lang.NumberFormatException: Invalid shift value in prefixCoded bytes (is encoded value really an INT?)]; nested: NumberFormatException[Invalid shift value in prefixCoded bytes (is encoded value really an INT?)]; Actually this is weird, i was facing the problem at 0.90.5 .. then I upgraded my elastic search binary to 0.90.9 and I still face the same error. |
I think the sorting term is the cause of the problem! |
MultiOrdinals.MultiDocs returned 'null' ordinals which caused a NPE if the field was single valued and would allow a significantly smaller in memory representation than single packed int ordinals. Closes elastic#3470
This test triggers a MultiDoc / MultiOrds in-memory representation even if the field is not multivalued Relates to elastic#3470
This issue relates to issue #3189 and I'm experiencing it with
v1.0.0.Beta
, but I guess that applies to0.90
branch as well.While sorting on some trivial fields I get the following error message:
lastname
mapping is as follows:Fairly standard IMO.
Now the funny thing is that I have another
name
mapping which is exactly the same and everything works just fine. From a data point of view, I cannot see a difference between one field and the other (upper case strings).The NPE happens in
BytesRefOrdValComparator
at line 388 becauseMultiDocs.ordinals()
returnsnull
.Now, i don't know why
lastname
happens to useMultiOrdinals
(orMultiDocs
for that matter) to do sorting, but makingMultiDocs
return itsMultiOrdinals
instance seems to fix the issue, although I'm not sure whether the null pointer was intentional and the problem is somewhere else.Thoughts?
Thanks a lot in advance.
The text was updated successfully, but these errors were encountered: