Match All Percolate Query Returns Unexpected Values

[marcoserrato]

Elasticsearch version (bin/elasticsearch --version): 6.4.2, 6.8.2, 7.6.0

Plugins installed: []

I use the elasticsearch docker images: docker.elastic.co/elasticsearch/elasticsearch-oss:x.x.x

Description of the problem including expected versus actual behavior:
First of, I'll mention that this bug is similar to #42361 which was supposedly fixed in 6.8.1 but I am able to reproduce similar behavior in higher versions.
When using nested docs and a percolate query with a match_all clause, I believe the percolate query is matching against the nested docs which is being manifested as negative indices in the _percolator_document_slot field.

Another note: I tested this on version 6.3.3 and am unable to reproduce it, so this must have been introduced in later versions.

Steps to reproduce:

Create a new index with a mapping for a nested doc containing two text fields and a query field for the percolator

Collapse/Expand

PUT test-index
{
  "mappings": {
    "properties": {
      "blob": {
        "type": "nested",
        "properties": {
          "inner_one": {
            "type": "text"
          },
          "inner_two": {
            "type": "text"
          }
        }
      },
      "query": {
        "type": "percolator"
      }
    }
  }
}

---

Create a percolate match_all percolate query

Collapse/Expand

PUT /test-index/_doc/2?refresh
{
  "query": {
    "bool": {
      "must": [
        {
          "match_all": {}
        }
      ]
    }
  }

---

Percolate some documents

Collapse/Expand

GET /test-index/_search
{
  "query": {
    "percolate": {
      "field": "query",
      "documents": [
        {
          "random": "hello"
        },
        {
          "random": "hello2",
          "blob": [
            {
              "inner_one": "none",
              "inner_two": "ntwo"
            }
          ]
        }
      ]
    }
  }
}

And the response:

{
  ...
  "hits" : {
    "total" : {
      "value" : 1,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "test-index",
        "_type" : "_doc",
        "_id" : "2",
        "_score" : 1.0,
        "_source" : {
          "query" : {
            "bool" : {
              "must" : [
                {
                  "match_all" : { }
                }
              ]
            }
          }
        },
        "fields" : {
          "_percolator_document_slot" : [
            0,
            -2,
            1
          ]
        }
      }
    ]
  }
}

---

I believe this is exactly the behavior described in the previously mentioned bug since increasing the number of nested docs in the percolated document increases the occurrences of negative indices.

For instance another percolate query

Collapse/Expand

GET /test-index/_search
{
  "query": {
    "percolate": {
      "field": "query",
      "documents": [
        {
          "random": "hello"
        },
        {
          "random": "hello2",
          "blob": [
            {
              "inner_one": "none",
              "inner_two": "ntwo"
            },
             {
              "inner_one": "none",
              "inner_two": "ntwo"
            },
             {
              "inner_one": "none",
              "inner_two": "ntwo"
            },
             {
              "inner_one": "none",
              "inner_two": "ntwo"
            },
             {
              "inner_one": "none",
              "inner_two": "ntwo"
            }
          ]
        }
      ]
    }
  }
}

returns the following

{
  ...
  },
  "hits" : {
    "total" : {
      "value" : 1,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "test-index",
        "_type" : "_doc",
        "_id" : "2",
        "_score" : 1.0,
        "_source" : {
          "query" : {
            "bool" : {
              "must" : [
                {
                  "match_all" : { }
                }
              ]
            }
          }
        },
        "fields" : {
          "_percolator_document_slot" : [
            0,
            -2,
            -2,
            -2,
            -2,
            -2,
            1
          ]
        }
      }
    ]
  }
}

[elasticmachine]

Pinging @elastic/es-search (:Search/Percolator)

[voxon2]

Is this a feature or a bug? I'm wondering cause i'm trying to determine if these is negatively impacting the performance of my percolator queries.

[martijnvg]

This is a bug. If percolator queries where to be highlighted then that would cause error if the returned slots are negative.