New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error on Script Based Sorting #2920

Closed
lmenezes opened this Issue Apr 20, 2013 · 7 comments

Comments

Projects
None yet
3 participants
@lmenezes
Contributor

lmenezes commented Apr 20, 2013

When used script based sorting on an index with mora than 1 shard, I get:

{"error":"ReduceSearchPhaseException[Failed to execute phase [query], [reduce] ]; nested: ClassCastException[java.lang.String cannot be cast to org.apache.lucene.util.BytesRef]; ","status":500}

Eg:

curl -XPOST http://localhost:9200/script -d '{"settings": { "index.number_of_replicas": 0, "index.number_of_shards": 2}}'

curl -XPUT http://localhost:9200/script/test/_mapping -d '{"profile":{"dynamic":"strict","properties":{"id":{"type":"integer","index":"not_analyzed","store":"yes"},"groups_code":{"properties":{"id":{"type":"integer","index":"not_analyzed"},"date":{"type":"date","index":"not_analyzed","format":"date_time_no_millis"}}}}}}'

curl -XPUT http://localhost:9200/script/test/1 -d '{"groups_code":[{"id":47642,"date":"2010-08-12T07:54:55Z"}]}'

curl -XPUT http://localhost:9200/script/test/2 -d '{"groups_code":[{"id":47642,"date":"2010-05-04T12:10:54Z"}]}'


curl -XGET http://localhost:9200/script/test/_search -d '{"query":{"match_all":{}},"fields":"","sort":[{"_script":{"script":"if ( ! _source.groups_code.empty ) { result = ($.date in _source.groups_code if $.id == id); if ( ! result.empty ) { result[0] } else { '' } }","type":"string","reverse":true,"params":{"id":47642}}}]}'
@lmenezes

This comment has been minimized.

Show comment
Hide comment
@lmenezes

lmenezes Apr 20, 2013

Contributor

sorry, forgot the version. This happens on 0.90.RC2 and also on the last build I had for RC3. Didn't use to happen on 0.20 afaik, but didn't check it.

Contributor

lmenezes commented Apr 20, 2013

sorry, forgot the version. This happens on 0.90.RC2 and also on the last build I had for RC3. Didn't use to happen on 0.20 afaik, but didn't check it.

@spinscale

This comment has been minimized.

Show comment
Hide comment
@spinscale

spinscale Apr 20, 2013

Member

This is not a shard issue, it also happens with one shard - and it also fails with 0.20.

To me it looks like your MVEL script is not correct and cannot be parsed. Check the last part of your error message, not the first name of the exception you are seeing.

Member

spinscale commented Apr 20, 2013

This is not a shard issue, it also happens with one shard - and it also fails with 0.20.

To me it looks like your MVEL script is not correct and cannot be parsed. Check the last part of your error message, not the first name of the exception you are seeing.

@lmenezes

This comment has been minimized.

Show comment
Hide comment
@lmenezes

lmenezes Apr 20, 2013

Contributor

I did check the message and the line where it happens in the code. The exception is thrown at
ShardFieldDocSortedHitQueue line 102(final BytesRef s1 = (BytesRef) docA.fields[i];).
Didn't look much more into it, but as far as I know the script is correct, it's been running in production with 0.20.4. and also, just grabbed a 0.20.4 release, tested and it works fine. Doesn't on 0.90.0 though. Unless I'm doing something very wrong and not realizing, it still seems like a bug to me.

Contributor

lmenezes commented Apr 20, 2013

I did check the message and the line where it happens in the code. The exception is thrown at
ShardFieldDocSortedHitQueue line 102(final BytesRef s1 = (BytesRef) docA.fields[i];).
Didn't look much more into it, but as far as I know the script is correct, it's been running in production with 0.20.4. and also, just grabbed a 0.20.4 release, tested and it works fine. Doesn't on 0.90.0 though. Unless I'm doing something very wrong and not realizing, it still seems like a bug to me.

@spinscale

This comment has been minimized.

Show comment
Hide comment
@spinscale

spinscale Apr 20, 2013

Member

you sample above is not working, thats why I was confused. You put single ticks inside of the curl query and confused the shell with that as you also used them for the whole json data.
Slightly changing and running your query now does not give me an exception either. Something still has to be different I guess. Any hints to get it reproduced for me?

curl -XGET http://localhost:9200/script/test/_search -d '{"query":{"match_all":{}},"fields":"","sort":[{"_script":{"script":"if ( ! _source.groups_code.empty ) { result = ($.date in _source.groups_code if $.id == id); if ( ! result.empty ) { result[0] } }","type":"string","reverse":true,"params":{"id":47642}}}]}'

{"took":269,"timed_out":false,"_shards":{"total":1,"successful":1,"failed":0},"hits":{"total":2,"max_score":null,"hits":[{"_index":"script","_type":"test","_id":"1","_score":null,"sort":["2010-08-12T07:54:55Z"]},{"_index":"script","_type":"test","_id":"2","_score":null,"sort":["2010-05-04T12:10:54Z"]}]}}
Member

spinscale commented Apr 20, 2013

you sample above is not working, thats why I was confused. You put single ticks inside of the curl query and confused the shell with that as you also used them for the whole json data.
Slightly changing and running your query now does not give me an exception either. Something still has to be different I guess. Any hints to get it reproduced for me?

curl -XGET http://localhost:9200/script/test/_search -d '{"query":{"match_all":{}},"fields":"","sort":[{"_script":{"script":"if ( ! _source.groups_code.empty ) { result = ($.date in _source.groups_code if $.id == id); if ( ! result.empty ) { result[0] } }","type":"string","reverse":true,"params":{"id":47642}}}]}'

{"took":269,"timed_out":false,"_shards":{"total":1,"successful":1,"failed":0},"hits":{"total":2,"max_score":null,"hits":[{"_index":"script","_type":"test","_id":"1","_score":null,"sort":["2010-08-12T07:54:55Z"]},{"_index":"script","_type":"test","_id":"2","_score":null,"sort":["2010-05-04T12:10:54Z"]}]}}
@lmenezes

This comment has been minimized.

Show comment
Hide comment
@lmenezes

lmenezes Apr 20, 2013

Contributor

From your response: {"took":269,"timed_out":false,"_shards":{"total":1,"successful":1,"failed":0},
could you try it with 2 shards? For 1 shard it works fine for me too

Contributor

lmenezes commented Apr 20, 2013

From your response: {"took":269,"timed_out":false,"_shards":{"total":1,"successful":1,"failed":0},
could you try it with 2 shards? For 1 shard it works fine for me too

@lmenezes

This comment has been minimized.

Show comment
Hide comment
@lmenezes

lmenezes Apr 20, 2013

Contributor

I mean, more than 1 shard on 0.90. On 0.20 works fine

Contributor

lmenezes commented Apr 20, 2013

I mean, more than 1 shard on 0.90. On 0.20 works fine

@spinscale

This comment has been minimized.

Show comment
Hide comment
@spinscale

spinscale Apr 20, 2013

Member

Ok, I can reproduce it now, just do this:

curl -X DELETE localhost:9200/script

curl -XPOST http://localhost:9200/script -d '{"settings": { "index.number_of_replicas": 0, "index.number_of_shards": 2}}'

curl -XPUT http://localhost:9200/script/test/_mapping -d '{"profile":{"dynamic":"strict","properties":{"id":{"type":"integer","index":"not_analyzed","store":"yes"},"groups_code":{"properties":{"id":{"type":"integer","index":"not_analyzed"},"date":{"type":"date","index":"not_analyzed","format":"date_time_no_millis"}}}}}}'

curl -XPUT http://localhost:9200/script/test/1 -d '{"groups_code":[{"id":47642,"date":"2010-08-12T07:54:55Z"}]}'
curl -XPUT http://localhost:9200/script/test/2 -d '{"groups_code":[{"id":47642,"date":"2010-05-04T12:10:54Z"}]}'

curl -XGET http://localhost:9200/script/test/_search -d '{"query":{"match_all":{}},"fields":"","sort":[{"_script":{"script":"if ( ! _source.groups_code.empty ) { result = ($.date in _source.groups_code if $.id == id); if ( ! result.empty ) { result[0] } else { \u0027\u0027 } }","type":"string","reverse":true,"params":{"id":47642}}}]}'

And to make it even simpler, this leads to the exception as well:

curl -XGET http://localhost:9200/script/test/_search -d '{"query":{"match_all":{}},"fields":"","sort":[{"_script":{"script":"\u0027\u0027","type":"string" }}]}'

Looks like we forgot to change a BytesRef to a string (might explain that it happens since we switched to lucene 4 with ES 0.90). So yeah, bug.

Many many thanks for your time reporting and explaining!

Member

spinscale commented Apr 20, 2013

Ok, I can reproduce it now, just do this:

curl -X DELETE localhost:9200/script

curl -XPOST http://localhost:9200/script -d '{"settings": { "index.number_of_replicas": 0, "index.number_of_shards": 2}}'

curl -XPUT http://localhost:9200/script/test/_mapping -d '{"profile":{"dynamic":"strict","properties":{"id":{"type":"integer","index":"not_analyzed","store":"yes"},"groups_code":{"properties":{"id":{"type":"integer","index":"not_analyzed"},"date":{"type":"date","index":"not_analyzed","format":"date_time_no_millis"}}}}}}'

curl -XPUT http://localhost:9200/script/test/1 -d '{"groups_code":[{"id":47642,"date":"2010-08-12T07:54:55Z"}]}'
curl -XPUT http://localhost:9200/script/test/2 -d '{"groups_code":[{"id":47642,"date":"2010-05-04T12:10:54Z"}]}'

curl -XGET http://localhost:9200/script/test/_search -d '{"query":{"match_all":{}},"fields":"","sort":[{"_script":{"script":"if ( ! _source.groups_code.empty ) { result = ($.date in _source.groups_code if $.id == id); if ( ! result.empty ) { result[0] } else { \u0027\u0027 } }","type":"string","reverse":true,"params":{"id":47642}}}]}'

And to make it even simpler, this leads to the exception as well:

curl -XGET http://localhost:9200/script/test/_search -d '{"query":{"match_all":{}},"fields":"","sort":[{"_script":{"script":"\u0027\u0027","type":"string" }}]}'

Looks like we forgot to change a BytesRef to a string (might explain that it happens since we switched to lucene 4 with ES 0.90). So yeah, bug.

Many many thanks for your time reporting and explaining!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment