Fix duplicate fetches in query plans #579
Closed
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #399 and #294.
The issue fixed by this PR requires a bit of unpacking. The visible result is that in some cases, the query plan will contain duplicate fetches to the same underlying service. For this query, for example:
We end up fetching the names of authors as many times as there are different types of products. (It isn’t just that we fetch the authors of the reviews of each product type separately, but we literally execute the same query with the same representations, so all authors for all products, multiple times.)
In the case above, this is triggered as a consequence of type explosion, but it is a separate issue and also occurs in other cases where we manually include duplicate nested fields under multiple type conditions.
This query, for example, will also contain as many fetches for authors as there are type conditions (so two fetches here):
The underlying cause is that we don't merge dependent fetch groups from subfields. In this case, we’ll create separate fetch groups for
{ name }
nested under the different type conditions, and even though these operate on the same path and fetch data from the same service, they are treated as separate fetches (that all have the same path however, which is why they take the same representations as input).This PR attempts to solve this by manually merging nested fetch groups whenever they are compatible. That fixes the issue in most cases, but there are more complicated situations it doesn't address and that will require more substantial changes to the query planner and execution.
In particular, when possible types are spread over multiple services, it will not be possible to merge nested fetch groups from subfields because these should actually result in separate fetches. What you don't want however, is for those fetches to operate on the same objects just fetched by the other, but since representations taken as input are collected by path, there is no way to differentiate between these.
I believe solving this latter problem requires additional capabilities in the query plan (which is something we’ve talked about before). The most straightforward way to fix this seems to be a
TypeCondition
node in the query plan, which filters input representations based on their type.