Searches do not return expected number of hits when using rescoring

[jurriaan]

When trying to upgrade to elasticsearch 1.5.x we encountered this unexpected behaviour.

For example:

{
  "from": 3,
  "size": 4,
  "query": {
    "term" : { "text" : "hello" }
  },
  "rescore" : {
    "window_size" : 50,
    "query" : {
       "rescore_query" : {
          "match" : {
             "text" : {
                "query" : "world",
                "slop" : 2
             }
          }
       },
       "query_weight" : 0.7,
       "rescore_query_weight" : 1.2
    }
  }
}

Returns 4 hits on ES 1.4.5, but only one on 1.5.2

See https://gist.github.com/koenbollen/f7232e3b5e600c66b4b4 for a minimal example that returns unexpected results on elasticsearch >= 1.5.0

[clintongormley]

Hi @jurriaan

Yes, i can replicate this. Setting from to anything but zero returns the same list of documents, but skipping the first from documents. It looks to be related to #7707

@mikemccand could you have a look at this please?

A simpler recreation below:

DELETE test

PUT test 
{
  "settings": {
    "number_of_shards": 1,
    "number_of_replicas": 0
  }
}

POST test/item/_bulk
{"index": {"_id": 1}}
{ "text": "hello world"}
{"index": {"_id": 2}}
{ "text": "hello world"}
{"index": {"_id": 3}}
{ "text": "hello world"}
{"index": {"_id": 4}}
{ "text": "hello world"}
{"index": {"_id": 5}}
{ "text": "hello world"}

POST /test/item/_search
{
  "from": 1,
  "size": 4,
  "query": {
    "term": {
      "text": "hello"
    }
  },
  "rescore": {
    "window_size": 50,
    "query": {
      "rescore_query": {
        "match_all": {}
      }
    }
  }
}

[mikemccand]

Thanks @jurriaan this is indeed wrong ... I opened #11342 to fix this.

[jurriaan]

@mikemccand Thanks for the quick fix! Hopefully we can migrate to a newer version of elasticsearch soon :)