Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.Sign up
Mapping: Remove pulsing/bloom_pulsing postings format #7566
In #7238 it looked like index corruption (checksum errors) but in fact it was simply that the user selected bloom_pulsing postings format, which we don't support yet still allow.
We recently removed documentation showing these postings format as a choice, but it's still really dangerous we allow this option at all since it creates unusable indices in ES when we migrate shards and try to check integrity. Before 1.3, ES didn't check Lucene checksums, so these postings formats worked fine, but with 1.3 any index using pulsing will fail.
The pulsing optimization has already been folded into the default postings format for quite a while now.
I think we should remove them; we are already removing pulsing from Lucene (https://issues.apache.org/jira/browse/LUCENE-5915)
Should we go further and disable (for now) any custom formats that don't have backwards compatibility support from lucene? These can change across releases in such a way that looks like corruption.
We are currently trying to figure out a way in Lucene to safely provide options to the user AND backwards compatibility, but this is not going to happen overnight.