Remove XPostingsHighlighter

[jpountz]

This class is a fork of Lucene's PostingsHighlighter. Quoting its javadocs:

FORKED from Lucene 4.5 to be able to:
1) support discrete highlighting for multiple values, so that we can return a different snippet per value when highlighting the whole text
2) call the highlightField method directly from subclasses and provide the terms by ourselves
3) Applied LUCENE-4906 to allow PassageFormatter to return arbitrary objects (LUCENE 4.6)

All our changes start with //BEGIN EDIT

I don't think we can maintain such forks in the long term as reconciliating changes that come from Lucene upgrades with changes that we did ourselves might even be impossible at some point. So I think it's important that everything that we need gets merged back to Lucene's PostingsHighlighter and that we remove XPostingsHighlighter from Elasticsearch.

[javanna]

I agree this is a problem, it is impossible to maintain.

To give you some more background here, the idea behind this experiment was to support the postings highlighter in elasticsearch, but trying to make it work as similar as possible to the other highlighters in terms of supported options and the result they provide.

One problem was supporting fragment_size: 0 which means highlighting the whole content of a field, against multiple values. Other highlighters return an array, one fragment per value, while the postings highlighter from lucene used to return all values in a single string. That's why we added "discrete per value highlighting" as a short-term solution.

One other problem was the require_field_match option that wasn't and probably will never be supported in lucene's postings highlighters, but other highlighters do support it. That's the ability to ignore which fields were queried, and just look for the terms in any field that needs to be highlighted.

In order to get rid of this custom class, either we

1. accept these differences between the standard postings highlighter from lucene and the other highlighters. We document them and we break bw comp as for the above two behaviours

2. find a way to contribute these two changes back to lucene

To be honest, I don't see 2) happen, especially for require_field_match so I'd vote for 1). At the end of the day, it is a different highlighter and we already tell people to try out the different highlighters without expecting exactly the same results.