Shingle filters that produce shingles of different size can create gigantic queries

[jimczi]

Shingle filters creates a graph token stream that the query parser is now able to consume.
Though when shingles of different size are produced the number of paths in the graph can explode.
This is also the case when output_unigram is set to true.
In 5.3 all paths are generated before building the query so a node can OOM easily on a single big input query. In 5.4 and beyond we detect the explosion earlier but we fail the entire request.
Instead we should be able to detect the problematic token filters and disable the graph analysis for these fields.

[clintongormley]

we should also update the docs to explain better config

i'd consider removing the min/max shingles settings in favour of a single size, and removing output_unigram too?

[jeantil]

@clintongormley 👍 for a doc on what a better config would be (if possible without altering the outcome of a request)