Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Test failure in TestKnnGraph.testMultiThreadedSearch #12940

Closed
vsop-479 opened this issue Dec 14, 2023 · 6 comments · Fixed by #12943
Closed

Test failure in TestKnnGraph.testMultiThreadedSearch #12940

vsop-479 opened this issue Dec 14, 2023 · 6 comments · Fixed by #12943
Assignees
@benwtrent
Labels
type:test

Comments

@vsop-479
Copy link
Contributor

Description

@msokolov Please take a look when you get a chance!

Gradle command to reproduce

./gradlew test --tests TestKnnGraph.testMultiThreadedSearch -Dtests.seed=15C41D6B3378DF2B -Dtests.locale=qu-Latn-PE -Dtests.timezone=CST6CDT -Dtests.asserts=true -Dtests.file.encoding=UTF-8
@msokolov
Copy link
Contributor

Thanks @vsop-479, it reproduces for me as well, both on main and 9x branches. The same test passes on 9.8.0 release. I'll try git bisect ... and it blames this commit:

[a47ba33] Refactoring HNSW to use a new internal FlatVectorFormat (#12729)

@benwtrent benwtrent mentioned this issue Dec 14, 2023
Merged
@benwtrent
Copy link
Member

#12943

@msokolov LOL

@benwtrent
Copy link
Member

We both figured it out at the same time.

@benwtrent benwtrent self-assigned this Dec 14, 2023
@msokolov
Copy link
Contributor

I can see that in this test run we are using a quantizing scorer, but I don't think the test case explicitly calls for that. I wonder if we beefed up the test framework to randomly use quantization and it is introducing "rounding" errors or something here?

@msokolov
Copy link
Contributor

ah, thanks @benwtrent I'll check your fix then

@benwtrent
Copy link
Member

I can see that in this test run we are using a quantizing scorer, but I don't think the test case explicitly calls for that. I wonder if we beefed up the test framework to randomly use quantization and it is introducing "rounding" errors or something here?

Your instincts are correct :)

benwtrent added a commit that referenced this issue Dec 14, 2023
@benwtrent
Fix flaky tests that are caused by small float vectors (#12943)
423f827 
While quantization generally works well, when the number of dimensions is tiny (just two like in our tests), and we are indexing a circle, and we have random merge policies, we can end up getting unexpected ordering on the resulting vectors.

closes: #12940
benwtrent added a commit that referenced this issue Dec 14, 2023
@benwtrent
Fix flaky tests that are caused by small float vectors (#12943)
26ba2ff 
While quantization generally works well, when the number of dimensions is tiny (just two like in our tests), and we are indexing a circle, and we have random merge policies, we can end up getting unexpected ordering on the resulting vectors.

closes: #12940
slow-J pushed a commit to slow-J/lucene that referenced this issue Dec 17, 2023
@benwtrent @slow-J
Fix flaky tests that are caused by small float vectors (apache#12943)
afe779e 
While quantization generally works well, when the number of dimensions is tiny (just two like in our tests), and we are indexing a circle, and we have random merge policies, we can end up getting unexpected ordering on the resulting vectors.

closes: apache#12940
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@benwtrent benwtrent

Labels
type:test
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix flaky tests that are caused by small float vectors benwtrent/lucene
3 participants
@msokolov @benwtrent @vsop-479