Set new providers before building FetchSubPhaseProcessors #97460
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
romseygeek
added
>bug
:Analytics/Aggregations
elasticsearchmachine
added
Team:Search
|
Pinging @elastic/es-analytics-geo (Team:Analytics) |
|
Hi @romseygeek, I've created a changelog YAML for you. |
|
Pinging @elastic/es-search (Team:Search) |
Bad bot
javanna
approved these changes
Jul 7, 2023
| @@ -100,6 +100,10 @@ private SearchHits buildSearchHits(SearchContext context, Profiler profiler) { | |||
| FetchContext fetchContext = new FetchContext(context); | |||
| SourceLoader sourceLoader = context.newSourceLoader(); | |||
|
|
|||
| PreloadedSourceProvider sourceProvider = new PreloadedSourceProvider(); | |||
| PreloadedFieldLookupProvider fieldLookupProvider = new PreloadedFieldLookupProvider(); | |||
| context.getSearchExecutionContext().setLookupProviders(sourceProvider, ctx -> fieldLookupProvider); | |||
There was a problem hiding this comment.
Would it be possible to add a unit test as well? I guess it really has to do with how top hits interacts with the fetch phase?
There was a problem hiding this comment.
Yes, unit tests for this will be tricky. We don't really have unit tests for any of the FetchPhase at the moment as the context structures that it uses are so heavy.
romseygeek
added
the
auto-backport-and-merge
💚 Backport successful
|
romseygeek
added a commit
to romseygeek/elasticsearch
that referenced
this pull request
Jul 10, 2023
) The FetchPhase sets source and field providers on its SearchLookup instance to ensure that we only have to load stored fields once per document. However, these were being set after subprocessors were constructed, and the FetchFieldsPhase subprocessor pulls a reference to the SearchLookup at construction time. This meant that the FieldsFetcher was not using these providers, instead using the standard source and fields lookup, in effect meaning that fields access would load source and fields separately. This would mostly just be an issue with performance; however, the top hits aggregation calls the fetch phase multiple times, with the result that the second and subsequent buckets in a top hits aggregation would see the source lookup from the previous bucket when building their FetchFieldsPhase subprocessor. This commit moves the call to SearchExecutionContext#setLookupProviders() in FetchPhase#buildSearchHits() before the call to build fetch phase subprocessors, ensuring that the fetch fields phase sees the correct SearchLookup. Fixes elastic#96284
elasticsearchmachine
pushed a commit
that referenced
this pull request
Jul 10, 2023
…97500) The FetchPhase sets source and field providers on its SearchLookup instance to ensure that we only have to load stored fields once per document. However, these were being set after subprocessors were constructed, and the FetchFieldsPhase subprocessor pulls a reference to the SearchLookup at construction time. This meant that the FieldsFetcher was not using these providers, instead using the standard source and fields lookup, in effect meaning that fields access would load source and fields separately. This would mostly just be an issue with performance; however, the top hits aggregation calls the fetch phase multiple times, with the result that the second and subsequent buckets in a top hits aggregation would see the source lookup from the previous bucket when building their FetchFieldsPhase subprocessor. This commit moves the call to SearchExecutionContext#setLookupProviders() in FetchPhase#buildSearchHits() before the call to build fetch phase subprocessors, ensuring that the fetch fields phase sees the correct SearchLookup. Fixes #96284
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The FetchPhase sets source and field providers on its SearchLookup instance
to ensure that we only have to load stored fields once per document. However,
these were being set after subprocessors were constructed, and the
FetchFieldsPhase subprocessor pulls a reference to the SearchLookup at
construction time. This meant that the FieldsFetcher was not using these
providers, instead using the standard source and fields lookup, in effect meaning
that
fieldsaccess would load source and fields separately.This would mostly just be an issue with performance; however, the top hits
aggregation calls the fetch phase multiple times, with the result that the second
and subsequent buckets in a top hits aggregation would see the source lookup
from the previous bucket when building their FetchFieldsPhase subprocessor.
This commit moves the call to
SearchExecutionContext#setLookupProviders()in
FetchPhase#buildSearchHits()before the call to build fetch phase subprocessors,ensuring that the fetch fields phase sees the correct SearchLookup.
Fixes #96284