Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mapping: Remove pulsing/bloom_pulsing postings format #7566

Closed
mikemccand opened this issue Sep 3, 2014 · 3 comments
Closed

Mapping: Remove pulsing/bloom_pulsing postings format #7566

mikemccand opened this issue Sep 3, 2014 · 3 comments

Comments

@mikemccand
Copy link
Contributor

In #7238 it looked like index corruption (checksum errors) but in fact it was simply that the user selected bloom_pulsing postings format, which we don't support yet still allow.

We recently removed documentation showing these postings format as a choice, but it's still really dangerous we allow this option at all since it creates unusable indices in ES when we migrate shards and try to check integrity. Before 1.3, ES didn't check Lucene checksums, so these postings formats worked fine, but with 1.3 any index using pulsing will fail.

The pulsing optimization has already been folded into the default postings format for quite a while now.

I think we should remove them; we are already removing pulsing from Lucene (https://issues.apache.org/jira/browse/LUCENE-5915)

@rmuir
Copy link
Contributor

rmuir commented Sep 3, 2014

Should we go further and disable (for now) any custom formats that don't have backwards compatibility support from lucene? These can change across releases in such a way that looks like corruption.

We are currently trying to figure out a way in Lucene to safely provide options to the user AND backwards compatibility, but this is not going to happen overnight.

@mikemccand
Copy link
Contributor Author

Should we go further and disable (for now) any custom formats that don't have backwards compatibility support from lucene? These can change across releases in such a way that looks like corruption.

+1, I'll do that.

@s1monw
Copy link
Contributor

s1monw commented Sep 3, 2014

++ to removing all the non-checksumming formats!

mikemccand added a commit that referenced this issue Sep 8, 2014
Lucene's experimental codecs (from the codecs module) do not provide
backwards compatibility and are free to change from release to
release.  When they do change, they typically cannot in general read
older indices and the resulting exceptions look like index corruption.
So, we are removing built-in support for them to prevent applications
from choosing one and then seeing strange exceptions on upgrade.

Closes #7566

Closes #7604
@clintongormley clintongormley changed the title Remove pulsing/bloom_pulsing postings format Mapping: Remove pulsing/bloom_pulsing postings format Sep 8, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants