Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Error on Script Based Sorting #2920

Closed
lmenezes opened this issue Apr 20, 2013 · 7 comments
Closed

Error on Script Based Sorting #2920

lmenezes opened this issue Apr 20, 2013 · 7 comments
Assignees
@s1monw
Labels
>bug v0.90.0

Comments

@lmenezes
Copy link
Contributor

When used script based sorting on an index with mora than 1 shard, I get:

{"error":"ReduceSearchPhaseException[Failed to execute phase [query], [reduce] ]; nested: ClassCastException[java.lang.String cannot be cast to org.apache.lucene.util.BytesRef]; ","status":500}

Eg:

curl -XPOST http://localhost:9200/script -d '{"settings": { "index.number_of_replicas": 0, "index.number_of_shards": 2}}'

curl -XPUT http://localhost:9200/script/test/_mapping -d '{"profile":{"dynamic":"strict","properties":{"id":{"type":"integer","index":"not_analyzed","store":"yes"},"groups_code":{"properties":{"id":{"type":"integer","index":"not_analyzed"},"date":{"type":"date","index":"not_analyzed","format":"date_time_no_millis"}}}}}}'

curl -XPUT http://localhost:9200/script/test/1 -d '{"groups_code":[{"id":47642,"date":"2010-08-12T07:54:55Z"}]}'

curl -XPUT http://localhost:9200/script/test/2 -d '{"groups_code":[{"id":47642,"date":"2010-05-04T12:10:54Z"}]}'


curl -XGET http://localhost:9200/script/test/_search -d '{"query":{"match_all":{}},"fields":"","sort":[{"_script":{"script":"if ( ! _source.groups_code.empty ) { result = ($.date in _source.groups_code if $.id == id); if ( ! result.empty ) { result[0] } else { '' } }","type":"string","reverse":true,"params":{"id":47642}}}]}'
@lmenezes
Copy link
Contributor Author

sorry, forgot the version. This happens on 0.90.RC2 and also on the last build I had for RC3. Didn't use to happen on 0.20 afaik, but didn't check it.

@spinscale
Copy link
Contributor

This is not a shard issue, it also happens with one shard - and it also fails with 0.20.

To me it looks like your MVEL script is not correct and cannot be parsed. Check the last part of your error message, not the first name of the exception you are seeing.

@lmenezes
Copy link
Contributor Author

I did check the message and the line where it happens in the code. The exception is thrown at
ShardFieldDocSortedHitQueue line 102(final BytesRef s1 = (BytesRef) docA.fields[i];).
Didn't look much more into it, but as far as I know the script is correct, it's been running in production with 0.20.4. and also, just grabbed a 0.20.4 release, tested and it works fine. Doesn't on 0.90.0 though. Unless I'm doing something very wrong and not realizing, it still seems like a bug to me.

@spinscale
Copy link
Contributor

you sample above is not working, thats why I was confused. You put single ticks inside of the curl query and confused the shell with that as you also used them for the whole json data.
Slightly changing and running your query now does not give me an exception either. Something still has to be different I guess. Any hints to get it reproduced for me?

curl -XGET http://localhost:9200/script/test/_search -d '{"query":{"match_all":{}},"fields":"","sort":[{"_script":{"script":"if ( ! _source.groups_code.empty ) { result = ($.date in _source.groups_code if $.id == id); if ( ! result.empty ) { result[0] } }","type":"string","reverse":true,"params":{"id":47642}}}]}'

{"took":269,"timed_out":false,"_shards":{"total":1,"successful":1,"failed":0},"hits":{"total":2,"max_score":null,"hits":[{"_index":"script","_type":"test","_id":"1","_score":null,"sort":["2010-08-12T07:54:55Z"]},{"_index":"script","_type":"test","_id":"2","_score":null,"sort":["2010-05-04T12:10:54Z"]}]}}

@lmenezes
Copy link
Contributor Author

From your response: {"took":269,"timed_out":false,"_shards":{"total":1,"successful":1,"failed":0},
could you try it with 2 shards? For 1 shard it works fine for me too

@lmenezes
Copy link
Contributor Author

I mean, more than 1 shard on 0.90. On 0.20 works fine

@spinscale
Copy link
Contributor

Ok, I can reproduce it now, just do this:

curl -X DELETE localhost:9200/script

curl -XPOST http://localhost:9200/script -d '{"settings": { "index.number_of_replicas": 0, "index.number_of_shards": 2}}'

curl -XPUT http://localhost:9200/script/test/_mapping -d '{"profile":{"dynamic":"strict","properties":{"id":{"type":"integer","index":"not_analyzed","store":"yes"},"groups_code":{"properties":{"id":{"type":"integer","index":"not_analyzed"},"date":{"type":"date","index":"not_analyzed","format":"date_time_no_millis"}}}}}}'

curl -XPUT http://localhost:9200/script/test/1 -d '{"groups_code":[{"id":47642,"date":"2010-08-12T07:54:55Z"}]}'
curl -XPUT http://localhost:9200/script/test/2 -d '{"groups_code":[{"id":47642,"date":"2010-05-04T12:10:54Z"}]}'

curl -XGET http://localhost:9200/script/test/_search -d '{"query":{"match_all":{}},"fields":"","sort":[{"_script":{"script":"if ( ! _source.groups_code.empty ) { result = ($.date in _source.groups_code if $.id == id); if ( ! result.empty ) { result[0] } else { \u0027\u0027 } }","type":"string","reverse":true,"params":{"id":47642}}}]}'

And to make it even simpler, this leads to the exception as well:

curl -XGET http://localhost:9200/script/test/_search -d '{"query":{"match_all":{}},"fields":"","sort":[{"_script":{"script":"\u0027\u0027","type":"string" }}]}'

Looks like we forgot to change a BytesRef to a string (might explain that it happens since we switched to lucene 4 with ES 0.90). So yeah, bug.

Many many thanks for your time reporting and explaining!

@lmenezes lmenezes mentioned this issue Apr 23, 2013
Closed
@s1monw s1monw closed this as completed in f372f7c Apr 23, 2013
@ghost ghost assigned s1monw Apr 23, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@s1monw s1monw

Labels
>bug v0.90.0
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@spinscale @lmenezes @s1monw