Wrong document score of nested knn query

[AlexanderOtt85]

Elasticsearch Version

9.2.1

Installed Plugins

No response

Java Version

bundled

OS Version

docker-image = docker.elastic.co/elasticsearch/elasticsearch:9.2.1

Problem Description

For a nested knn query with score_mode = max the document score does not match the expected score. I would expect the document score = 0.5852207 which is the maximum score of the inner hit (see 'nested knn query').

The score is not the expected one when "index_options": {"type":"bbq_disk"} is used. However, when "index_options": {"type":"hnsw"} is used, the score is the expected maximum from the inner_hits.

bbq_disk

"hits": { "total": { "value": 1, "relation": "eq" }, **"max_score": 0.57664675,** "hits": [ { ... **"_score": 0.57664675,** ..., "_explanation": { "value": 0.57664675, "description": "Score based on 1 child docs in range from 0 to 377, using score mode Max", "details": [ { "value": 0.57664675, "description": "sum of:", "details": [ { "value": 0.57664675, "description": "within top k documents", "details": [] }, { "value": 0.0, "description": "match on required clause, product of:", "details": [ { "value": 0.0, "description": "# clause", "details": [] }, { "value": 1.0, "description": "_nested_path:du.medium.keyframe_tc_hit", "details": [] } ] } ] } ] }, "inner_hits": { "du.medium.keyframe_tc_hit": { "hits": { "total": { "value": 352, "relation": "eq" }, **"max_score": 0.5852207,** ...

hnsw

"hits": { "total": { "value": 1, "relation": "eq" }, **"max_score": 0.5852207,** "hits": [ { ... **"_score": 0.5852207,** ... "_explanation": { "value": 0.5852207, "description": "Score based on 1 child docs in range from 0 to 377, using score mode Max", "details": [ { "value": 0.5852207, "description": "sum of:", "details": [ { "value": 0.5852207, "description": "within top 1 docs", "details": [] }, { "value": 0.0, "description": "match on required clause, product of:", "details": [ { "value": 0.0, "description": "# clause", "details": [] }, { "value": 1.0, "description": "_nested_path:du.medium.keyframe_tc_hit", "details": [] } ] } ] } ] }, "inner_hits": { "du.medium.keyframe_tc_hit": { "hits": { "total": { "value": 352, "relation": "eq" }, **"max_score": 0.5852207**, ...

Steps to Reproduce

1. Create index
2. Index document
3. knnQuery where the score is not OK. expected score = 0.5852207 current score = 0.5711336

create_index.txt
index_document.txt
nested_query.txt

Logs (if relevant)

No response

[elasticsearchmachine]

Pinging @elastic/es-search-relevance (Team:Search Relevance)

[tteofili]

I managed to reproduce this, basically there are parent docs in the result set with a score larger than the reported max_score for bbq_disk.

[tteofili]

after a lot of debugging, I figured the problem might be that the first parent block join query performs an approximate KNN query whereas the inner hits query is actually an exact KNN query, this leads to differing results. I'll keep digging into this.
update: this was originally implemented as such in #113949 therefore I would qualify this for now as an expected behavior (exact vs approximate KNN results are likely always slightly different).

[mayya-sharipova]

I confirm the behaviour.

I think there are two reasons for discrepancies of scores between top hits and inner hits:

• As @tteofili said: top hits are collected based on approximate search, while for inner hits we collect and score all inner hits of a parent doc. Because approximate search doesn't check every document, it is not guaranteed to find the absolute best-scoring documents. That's what I think is happening here for bbq_disk.
• Another factor is different method of scoring for top_hits and inner hits. For top_hits on heavily quantized data we automatically apply rescoring based on full precision vectors, that may not be applied on inner_hits

---

For example in case of bbq_flat, to remedy discrepancy between inner hits' scores and top hits scores, we can set rescore_vector.overample: 0.

GET my_index2/_search
{
  "size": 5,
  "_source": false,
  "query": {
    "nested": {
      "path": "du.medium.keyframe_tc_hit",
      "query": {
        "knn": {
          "query_vector": [...],
          "field": "du.medium.keyframe_tc_hit.vector.vector",
          "k": 10,
          "rescore_vector" : {
            "oversample": 0
          }
        }
      },
      "inner_hits": {
        "size": 5,
        "_source": false
      }
    }
  }
}

---

We can discuss within the team what we can do about it.

[tteofili]

As tteofili said: top hits are collected based on approximate search, while for inner hits we collect and score all inner hits of a parent doc.
Because approximate search doesn't check every document, it is not guaranteed to find the absolute best-scoring documents. That's what I think is happening here for bbq_disk.
Another factor is different method of scoring for top_hits and inner hits. For top_hits on heavily quantized data we automatically apply rescoring based on full precision vectors, that may not be applied on inner_hits

for DiskBBQ, we might introduce a way to do exact knn that leverages (more bits) quantized vectors, this would have the side effect of also fixing this issue /cc @benwtrent

[mayya-sharipova]

Adding more notes outside of DiskBBQ:

• if rescoring is applied, use rescoring also for inner_hits.