Skip to content

Conversation

@ldematte
Copy link
Contributor

While running gpu:yamlRestTest tests, I realized that under some circumstances the scorer is not Lucene99MemorySegmentFloatVectorScorerSupplier, but DefaultFlatVectorScorer$FloatScoringSupplier. This PR extend reflection to support the "fast path" to access vector values to this class too.

@ldematte ldematte requested a review from a team as a code owner November 24, 2025 13:59
@ldematte ldematte added the test-gpu Run tests using a GPU label Nov 24, 2025
@elasticsearchmachine elasticsearchmachine added Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch v9.3.0 labels Nov 24, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-search-relevance (Team:Search Relevance)

@mayya-sharipova
Copy link
Contributor

mayya-sharipova commented Nov 24, 2025

@ldematte How do encounter DefaultFlatVectorScorer$FloatScoringSupplier?

I checked Lucene code, and DefaultFlatVectorScorer$FloatScoringSupplier does NOT implement HasIndexSlice, so we will always return null and fall back to generating merged float values directly:

FloatVectorValues floatVectorValues = MergedVectorValues.mergeFloatVectorValues(fieldInfo, mergeState);

So I think the changes in this PR are NOT necessary.


What do you think?

@ldematte
Copy link
Contributor Author

I checked Lucene code, and DefaultFlatVectorScorer$FloatScoringSupplier does NOT implement HasIndexSlice, so we will always return null

image

It's not the supplier that needs to implement HasIndexSlice, but the vectorValues it returns. As you can see in the picture, DefaultFlatVectorScorer$FloatScoringSupplier is returning a OffHeapFloatVectorValues, which implements slices/memorysegments.

You do see DefaultFlatVectorScorer$FloatScoringSupplier in YAML rest test; while rest tests do use a "real" cluster, there are differences, so I'm not sure how/if we can see the same in a full distribution. Still, it may happen, so I think it's worth adding.

@ldematte ldematte removed the test-gpu Run tests using a GPU label Nov 25, 2025
@mayya-sharipova
Copy link
Contributor

@ldematte Thanks for further clarifications.

Sorry I was checking how Lucene99MemorySegmentFlatVectorsScorer falls back to DefaultFlatVectorScorer::getRandomVectorScorer when vectorValues is not isntance of HasIndexSlice.

It makes sense to me to add it even just for tests.

@ldematte ldematte added auto-backport Automatically create backport pull requests when merged branch:9.2 labels Nov 28, 2025
@ldematte ldematte merged commit 7bb054f into elastic:main Nov 28, 2025
33 of 34 checks passed
@elasticsearchmachine
Copy link
Collaborator

💚 Backport successful

Status Branch Result
9.2

ldematte added a commit to ldematte/elasticsearch that referenced this pull request Nov 28, 2025
…rer$FloatScoringSupplier too (elastic#138503)

While running gpu:yamlRestTest tests, under some circumstances the scorer is not Lucene99MemorySegmentFloatVectorScorerSupplier, but DefaultFlatVectorScorer$FloatScoringSupplier. This PR extend reflection to support the "fast path" to access vector values to this class too.
elasticsearchmachine pushed a commit that referenced this pull request Nov 28, 2025
…rer$FloatScoringSupplier too (#138503) (#138743)

While running gpu:yamlRestTest tests, under some circumstances the scorer is not Lucene99MemorySegmentFloatVectorScorerSupplier, but DefaultFlatVectorScorer$FloatScoringSupplier. This PR extend reflection to support the "fast path" to access vector values to this class too.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto-backport Automatically create backport pull requests when merged >non-issue :Search Relevance/Vectors Vector search Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch v9.2.3 v9.3.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants