Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix flaky tests that are caused by small float vectors #12943

Merged
merged 1 commit into from
Dec 14, 2023
Merged

Fix flaky tests that are caused by small float vectors #12943

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
28 changes: 12 additions & 16 deletions lucene/core/src/test/org/apache/lucene/index/TestKnnGraph.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -100,22 +100,18 @@ public KnnVectorsFormat getKnnVectorsFormatForField(String field) {
}
};

if (vectorEncoding == VectorEncoding.FLOAT32) {
float32Codec = codec;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so this was relying on the default codec being a FLOAT32 codec? TBH I haven't kept up with how we now select whether or not to use quantization. Is it the default? Do we need to override the codec to select it?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@msokolov yeah, two tests are really relying on perfect scores & things being float. I mistakenly turned on quantization which adds some error bands to the scores (obviously, because its lossy) and thus its flaky.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there was just another failure on the dev list

FAILED: org.apache.lucene.util.hnsw.TestHnswFloatVectorGraph.testSortedAndUnsortedIndicesReturnSameResults

Error Message:
java.lang.AssertionError: expected:<[43, 199, 163, 3, 180]> but was:<[270, 43, 199, 163, 269]>

I wonder if it could be a similar root cause?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It could be, I was going to look into that one next.

Copy link
Contributor

@msokolov msokolov Dec 14, 2023 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK that does repro for me, but also on branch_9_8, so I think it must be a different issue - I'll open a new one to track. Umm correction it does not repro on branch_9_8, but it is in any case a different issue: #12945

} else {
float32Codec =
new FilterCodec(TestUtil.getDefaultCodec().getName(), TestUtil.getDefaultCodec()) {
@Override
public KnnVectorsFormat knnVectorsFormat() {
return new PerFieldKnnVectorsFormat() {
@Override
public KnnVectorsFormat getKnnVectorsFormatForField(String field) {
return new Lucene99HnswVectorsFormat(M, HnswGraphBuilder.DEFAULT_BEAM_WIDTH);
}
};
}
};
}
float32Codec =
new FilterCodec(TestUtil.getDefaultCodec().getName(), TestUtil.getDefaultCodec()) {
@Override
public KnnVectorsFormat knnVectorsFormat() {
return new PerFieldKnnVectorsFormat() {
@Override
public KnnVectorsFormat getKnnVectorsFormatForField(String field) {
return new Lucene99HnswVectorsFormat(M, HnswGraphBuilder.DEFAULT_BEAM_WIDTH);
}
};
}
};
}

private VectorEncoding randomVectorEncoding() {
Expand Down