Properly propagate the search request context & headers to sub-requests

[uboness]

Each transport request holds context and headers that represent the general context of the executed actions. There are actions that spawn other requests and we make sure that the context/headers of the original request is passed along to the sub-requests.

In the case of a search request, we have some cases where this context/headers passing doesn't happen and we need to fix that (we need to make sure the context & headers are not lost at any point in the execution). The reason for this today is that some spawned request simply don't have access to the original request. For example, TermsLookup used by the external terms filter executes a search during the parsing of the query. It has no access to the original search request that was executed and therefore cannot pass along the context & headers.

To solve that we can use the SearchContext and let it hold the search request context & headers (maybe we can have the SearchContext extends ContextHolder). During the execution of a search, any phase that is part of the execution has access to the search context using the thread local based SearchContext.current(). We need to make sure, to set the request context & headers on the SearchContext when it is created. From there on, we can extract these request context & headers everywhere we need to and pass it to the sub-request (a la TermsLookup).

Other sub request we need to look at:

• MoreLikeThisQuery
• GeoShapeFilter and GeoShapeQuery with a indexed shape
• Indexed Scripts
• Phrase Suggester with a collate query or filter
• Search Templates with indexed templates

[clintongormley]

@spinscale please could you take a look when you have a moment

[spinscale]

checked the source, my current idea of implementing it is

• Make SearchContext implementing an interface that adheres to ContextHolder, propagate to ShardSearchRequest, maybe just changing DefaultSearchContext is enough, needs to be checked
• ShardSearchRequest also needs to implement that new interface, one of the two subclasses, ShardSearchTransportRequest already does, one needs to be extended ShardSearchLocalRequest
• Both classes have access to the original SearchRequest and thus can copy the context
• Make sure all of the above requests can make use of this infra
• Search for possible more requests that need this
• Copy headers need to be copied as well

Tricky: Testing this

[uboness]

Later: Do headers need to be copied as well

while we're at it, I'd just copy the headers as well.. why not

[spinscale]

Implementation note: Percolate-by-id must be supported as well