Function score query and simple nearest_neighbors query return different values.

[ArtemKar123]

Hi! I have faced a new issue. I have 2 different queries:
1)

{
    "size": 5,
    "stored_fields": [ "title_name" ],
    "query":{
      "elastiknn_nearest_neighbors": {        
          "field": "title_vector",                  
          "vec": {                            
              "values": list(target_embedding),               
          },
          "candidates": 100, 
          "model": "lsh", 
          "probes": 3, 
          "similarity": "l2"     
      }
    }
}

{
    "size": 5,
    "stored_fields": [ "title_name" ],
    "query":
    {
        "function_score": {
          "query": {
            "terms": { "color_id": colors }           # colors = [1, 2, 3...]       
          },
          "functions": [                           
            {
              "elastiknn_nearest_neighbors": {        
                    "field": "title_vector",                  
                    "vec": {                            
                        "values": list(target_embedding),               
                    },
                    "candidates": 100, 
                    "model": "lsh", 
                    "probes": 3, 
                    "similarity": "l2"     
                }
            }]          
        }
      }
}

And they return different values even when the colors array in second query is array of every possible color in index.
Is it possible to make function score query return the same result as the first query?

[ArtemKar123]

Tried to do the same but with angular similarity and it works just fine!

[alexklibisz]

Hi Artem,
Can you look at the scores from both queries and make sure that they are virtually equivalent? Specifically, just look at the score of the last document in the results. Both query should have roughly the same score for this document. They might differ some minor decimal precision. If they are not equivalent, then we have a more interesting question or problem.

[alexklibisz]

On a related note, Elasticsearch runs the "query" section of the function score query as a pre-filtering step that can affect the order in which Elastiknn accesses the candidate documents. With such a large index (~100M vectors and ~50 shards, according to previous question), the order that candidates are visited can affect the overall outcome.