[CI] VectorSimilarityFunctionsIT testTopNSimilarityBetweenConstantVectorAndField {functionName=v_l2_norm similarityFunction=org.elasticsearch.xpack.esql.expression.function.vector.L2Norm$1@5b068087 elementType=byte} failing

[elasticsearchmachine]

Build Scans:

• elasticsearch-intake #30867 / part3
• elasticsearch-pull-request #102231 / part-3

Reproduction Line:

./gradlew ":x-pack:plugin:esql:internalClusterTest" --tests "org.elasticsearch.xpack.esql.vector.VectorSimilarityFunctionsIT" -Dtests.method="testTopNSimilarityBetweenConstantVectorAndField {functionName=v_l2_norm similarityFunction=org.elasticsearch.xpack.esql.expression.function.vector.L2Norm$1@5b068087 elementType=byte}" -Dtests.seed=EA04A247EAA69601 -Dtests.locale=zh-MO -Dtests.timezone=Europe/Isle_of_Man -Druntime.java=25

Applicable branches:
main

Reproduces locally?:
N/A

Failure History:
See dashboard

Failure Message:

java.lang.AssertionError: Expected [711.2003784179688, 695.7657470703125, 693.9632568359375, 687.3369140625, 683.2400512695312, 677.8502807617188, 670.509521484375, 665.8460693359375, 662.4877319335938, 659.2283325195312] but got [[711.2003784179688], [695.7657470703125], [687.3369140625], [683.2400512695312], [677.8502807617188], [670.509521484375], [665.8460693359375], [659.2283325195312], [651.5496826171875], [644.9922485351562]] expected:<693.9632568359375> but was:<687.3369140625>

Issue Reasons:

• [main] 2 failures in test testTopNSimilarityBetweenConstantVectorAndField {functionName=v_l2_norm similarityFunction=org.elasticsearch.xpack.esql.expression.function.vector.L2Norm$1@5b068087 elementType=byte} (12.5% fail rate in 16 executions)

Note:
This issue was created using new test triage automation. Please report issues or feedback to es-delivery.

[elasticsearchmachine]

This has been muted on branch main

Mute Reasons:

• [main] 2 failures in test testTopNSimilarityBetweenConstantVectorAndField {functionName=v_l2_norm similarityFunction=org.elasticsearch.xpack.esql.expression.function.vector.L2Norm$1@5b068087 elementType=byte} (12.5% fail rate in 16 executions)

Build Scans:

• elasticsearch-intake #30867 / part3
• elasticsearch-pull-request #102231 / part-3

[elasticsearchmachine]

Pinging @elastic/es-search-relevance (Team:Search Relevance)

[pmpailis]

Looking at the results and running locally (which fails <= 1 out of 100 runs) it seems through the similarities as if some docs are missing from the result set. E.g above:

Expected:

[711.2003784179688, 695.7657470703125, 693.9632568359375, 687.3369140625, 683.2400512695312, 
677.8502807617188, 670.509521484375, 665.8460693359375, 662.4877319335938, 659.2283325195312] but got 

Actual:

[[711.2003784179688], [695.7657470703125], <MISSING>, [687.3369140625], [683.2400512695312], 
[677.8502807617188], [670.509521484375], [665.8460693359375], <MISSING>,  [659.2283325195312], 
[651.5496826171875], [644.9922485351562]]   <-- these are extra results

[pmpailis]

Also, something interesting to note is that this seems to fail only for ElementType.BYTE

[pmpailis]

Logging the vectors along with the reported similarities for a specific failure it seems that we have:

indexed_vector: [-124, 18, 54, 33, 17, -62, 55, 38, 73, -65, 57, 106, 10, -76, 8, 100, 81, -33, 67, -19, 113, -110, 15, -43, -75, -16, 38, -116, 111, 38, -2, 76, -63, -123, -123, -27, 41, 114]

random_vector: [115, 108, -61, 49, -90, 122, 24, -15, 58, -20, 14, -43, -112, -7, 118, 43, -18, -49, 12, 7, -120, 33, 92, -114, -80, -43, 20, -71, -126, 103, -8, 61, -113, -116, 72, 15, 106, -99] 

index-based hamming distance: 174

expected hamming distance: 171

[pmpailis]

So, the response from index seems to be a bit off. Looking a bit more to track whether this is something a test only thing, or if there is an actual underlying bug.