-
Notifications
You must be signed in to change notification settings - Fork 24.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Core: Don't verify adler32 for lucene 3.x terms dict/terms index. #9142
Conversation
These files are not write-once, but the 0.20.x checksum code didn't handle this properly, so we can't use the adler32.
LGTM. |
I added a commit enabling testing of 0.20.6 index. It fails with the checksum error without this fix. |
Still looks good! |
I thought about this .CFS case all day. For now, I would like the additional added logic (see latest commit) to avoid false checksum failures. If we want to throw an error on old versions i would like to separate that from doing the right thing here. Also note again we don't need this cruft on master, just 1.x |
Obviously this one is not really easy to test, our backwards tests config currently does not work with such ancient versions. But I know these old segments are still floating around, because I see people still hitting https://issues.apache.org/jira/browse/LUCENE-5975 on 1.3.x versions and so on. Enjoy reviewing :) |
looks good though. can we maybe have a simple unittest in StoreTests that checks if your conditionals are working ie with some bogus StoreFileMetadata? |
Good idea. We might as well put that fake segmentinfowriter to use. |
I added tests for these cases with fake metadata. |
very cool thanks rob! LGTM |
Don't verify adler32 for lucene 3.x terms dictionary (.tis), terms index (.tii), 3.0-3.3 compound file (.cfs), 3.x segments_N, or 3.x segments.gen. These files are not write-once, but the 0.20.x checksum code didn't handle .tis/.tii properly, so we can't use the adler32. Even older versions will have the same problems with other cases. Closes #9142
Don't verify adler32 for lucene 3.x terms dictionary (.tis), terms index (.tii), 3.0-3.3 compound file (.cfs), 3.x segments_N, or 3.x segments.gen. These files are not write-once, but the 0.20.x checksum code didn't handle .tis/.tii properly, so we can't use the adler32. Even older versions will have the same problems with other cases. Closes #9142 Conflicts: src/main/java/org/elasticsearch/index/store/Store.java src/test/java/org/elasticsearch/bwcompat/OldIndexBackwardsCompatibilityTests.java
Don't verify adler32 for lucene 3.x terms dictionary (.tis), terms index (.tii), 3.0-3.3 compound file (.cfs), 3.x segments_N, or 3.x segments.gen. These files are not write-once, but the 0.20.x checksum code didn't handle .tis/.tii properly, so we can't use the adler32. Even older versions will have the same problems with other cases. Closes #9142 Conflicts: src/main/java/org/elasticsearch/index/store/Store.java src/test/java/org/elasticsearch/bwcompat/OldIndexBackwardsCompatibilityTests.java
Don't verify adler32 for lucene 3.x terms dictionary (.tis), terms index (.tii), 3.0-3.3 compound file (.cfs), 3.x segments_N, or 3.x segments.gen. These files are not write-once, but the 0.20.x checksum code didn't handle .tis/.tii properly, so we can't use the adler32. Even older versions will have the same problems with other cases. Closes elastic#9142 Conflicts: src/main/java/org/elasticsearch/index/store/Store.java src/test/java/org/elasticsearch/bwcompat/OldIndexBackwardsCompatibilityTests.java
Don't verify adler32 for lucene 3.x terms dictionary (.tis), terms index (.tii), 3.0-3.3 compound file (.cfs), 3.x segments_N, or 3.x segments.gen. These files are not write-once, but the 0.20.x checksum code didn't handle .tis/.tii properly, so we can't use the adler32. Even older versions will have the same problems with other cases. Closes elastic#9142 Conflicts: src/main/java/org/elasticsearch/index/store/Store.java src/test/java/org/elasticsearch/bwcompat/OldIndexBackwardsCompatibilityTests.java
These files are not write-once, but the 0.20.x checksum code
didn't handle this properly, so we can't use the adler32.