nested KNN query returns no results when using nested query on top level field as filter

[dimitris-athanasiou]

Elasticsearch Version

9.2.0

Installed Plugins

No response

Java Version

bundled

OS Version

Darwin Kernel Version 24.6.0

Problem Description

Calling _search with a nested knn query that is using as a filter another nested query on a top-level field produces 0 hits even though it should have matches.

Steps to Reproduce

PUT example
{
    "settings": {
        "number_of_shards": 1,
        "number_of_replicas": 0
    },
    "mappings": {
        "properties": {
            "nested_1": {
                "type": "nested",
                "properties": {
                    "vector": {
                        "type": "dense_vector",
                        "dims": 3
                    }
                }
            },
            "nested_2": {
                "type": "nested",
                "properties": {
                    "category": {
                        "type": "keyword"
                    }    
                }
                
            }
        }
    }
}

PUT example/_doc/1
{
    "nested_1" : {
        "vector": [1, 2, 3]
    },
    "nested_2": {
        "category": "label_1"
    }  
}

POST example/_search
{
    "query": {
        "nested": {
            "path": "nested_1",
            "query": {
                "knn": {
                    "field": "nested_1.vector",
                    "query_vector": [1, 2, 3],
                    "k": 10,
                    "num_candidates": 10,
                    "filter": {
                        "nested": {
                            "path": "nested_2",
                            "query": {
                                "term": {
                                    "nested_2.category": {
                                        "value": "label_1"
                                    }
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}

Logs (if relevant)

No response

[elasticsearchmachine]

Pinging @elastic/es-search-relevance (Team:Search Relevance)

[mayya-sharipova]

@dimitris-athanasiou

I confirm that I can execute your query through knn search (not knn query):

POST example/_search
{
    "knn" : {
        "field": "nested_1.vector",
        "query_vector": [1, 2, 3],
        "filter": {
            "nested": {
                "path": "nested_2",
                "query": {
                    "term": {
                        "nested_2.category": {
                            "value": "label_1"
                        }
                    }
                }
            }
        }
    }
}

Can you try using it instead of knn query. Nested knn query when executing sets the context of nested_1 that's why your filter will not work as it will search in nested_1.nested_2. knn search doesn't do that.

---

 PUT example
  {
      "mappings": {
          "properties": {
              "nested_items": {
                  "type": "nested",
                  "properties": {
                      "vector": {
                          "type": "dense_vector",
                          "dims": 3
                      },
                      "category": {
                          "type": "keyword"
                      }
                  }
              }
          }
      }
  }

  Then query:

  POST example/_search
  {
      "query": {
          "nested": {
              "path": "nested_items",
              "query": {
                  "knn": {
                      "nested_items.vector": {
                          "query_vector": [1, 2, 3],
                          "k": 10,
                          "num_candidates": 10,
                          "filter": {
                              "term": {
                                  "nested_items.category": "label_1"
                              }
                          }
                      }
                  }
              }
          }
      }
  }