New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
LUCENE-9356: Add a test that verifies that Lucene catches bit flips. #1569
Conversation
b0cf009
to
e231f66
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! So we now check checksums on every file when opening. It's good that we have a test to test the verification!
try (IndexInput in = dirCopy.openInput(name, IOContext.DEFAULT)) { | ||
try { | ||
CodecUtil.checksumEntireFile(in); | ||
System.out.println("TEST: changing a byte in " + victim + " did not update the checksum)"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
curious if you saw this much?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I haven't seen a single occurrence of it (fortunately! :) )
We only verify checksums of meta file when opening (those that we read entirely anyway). Checksums only get verified on other files when |
} | ||
|
||
RandomIndexWriter riw = new RandomIndexWriter(random(), dir, conf); | ||
// Use LineFileDocs so we (hopefully) get most Lucene features |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Woohoo!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is actually copy-pasted from TestAllFilesDetectTruncation
:)
private void checkBitFlips(Directory dir) throws IOException { | ||
for(String name : dir.listAll()) { | ||
if (name.equals(IndexWriter.WRITE_LOCK_NAME) == false) { | ||
corruptFile(dir, name); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you maybe need to exclude extraN
files for when ExtraFS
randomly strikes?
Edit: oh, nevermind, I see we suppress ExtraFS
above :)
Opening a reader and then calling checkIntegrity must throw a
CorruptIndexException
or anIndexFormatToo(Old|New)Exception
.