Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Query: Re-evaluate concept of pending selector in nav expansion #20291

Open
Tracked by #18923 ...
smitpatel opened this issue Mar 15, 2020 · 1 comment
Open
Tracked by #18923 ...

Query: Re-evaluate concept of pending selector in nav expansion #20291

smitpatel opened this issue Mar 15, 2020 · 1 comment
Labels
area-query punted-for-5.0 punted-for-7.0 Originally planned for the EF Core 7.0 (EF7) release, but moved out due to resource constraints. type-enhancement
Milestone

Comments

@smitpatel
Copy link
Member

smitpatel commented Mar 15, 2020

When we rewrote query in 3.0, we added a concept called pending selector in nav expansion. Since nav expansion is expanding navigations, we defer applying the projection and keep intermediate shape to reuse the navigations already expanded. This is useful in many scenarios but deferring projection has side effects. See #7776 for one example. That is just one scenario. Any time FirstOrDefault is used in any way (selectmany or next select), by deferring that projection we duplicate the subquery for any property access on that. Not only it causes ugly query but potentially bad perf.

Instead of deferring the projection if we applied projection in above case then, during translation it would cause a subquery and any further property access would be columns coming out subquery. Hence it can be translated without causing client eval error at the same time avoid repeating subquery. This approach would do exactly what @maumar requested in the #7776 (comment)

Making this work will require

  • Update navigation expansion to take a hybrid approach to apply projection at the same time remember the navigations already expanded to reuse them.
    • We only need to remember navigations which can be accessible by projection being applied.
  • Update translation pipeline to handle translating query operators after client eval phase if it can be server evaluated.

There are few more associated issues this would fix. #18775

@smitpatel smitpatel self-assigned this Mar 15, 2020
@ajcvickers ajcvickers added this to the 5.0.0 milestone Mar 16, 2020
@ajcvickers ajcvickers modified the milestones: 5.0.0, Backlog Jun 9, 2020
smitpatel added a commit that referenced this issue Jun 10, 2020
smitpatel added a commit that referenced this issue Jun 10, 2020
smitpatel added a commit that referenced this issue Jun 10, 2020
smitpatel added a commit that referenced this issue Jun 10, 2020
@smitpatel
Copy link
Member Author

#21201 added code path to process joins when outer is client eval. Currently in the absence of this issue the code path is never hit. When fixing this, consider the code path and change or remove it is not needed.

smitpatel added a commit that referenced this issue Jun 11, 2020
smitpatel added a commit that referenced this issue Apr 16, 2021
- ClientProjections is alternate when ProjectionMember binding is not possible using indexes
- ClientProjections is separate from SelectExpression.Projection where former holds complex client side mapped projections and later holds only scalars
- What it enables
  - Single result subquery is not joined right away
  - pendingCollections are removed
  - Ability to bind with above subquery translations after projection has converted to index based binding (enabling pending selector future change)
  - Ability to bind client projections smoothly like ProjectionMember binding so suppose translations post-client eval where it is supported
  - Grouping.FirstOrDefault can add subquery projection which applies separately so we can combine aggregate/non-aggregate projection on grouping
- Introduce CollectionResultExpression which holds info on how to create collection for a subquery
- ApplyProjection converts projectionMember/Index based bindings to actual scalar projection and updates shaper in same pass
- Unique collectionId are assigned by shaper
- Change in flow to projection to let sqlTranslator determines translatibility before processing for client side
- Regression in collection over single result subquery
  - Earlier we applied single result right away copying over pending collection to outer which allowed single result subquery to convert apply to join
  - Now all client projections apply at same time so pending collection (equivalent subquery) is applied inside single result subquery so we fail to convert apply to join

Ground work for #20291, #12088, #13805
Resolves #23571
smitpatel added a commit that referenced this issue Apr 16, 2021
- ClientProjections is alternate when ProjectionMember binding is not possible using indexes
- ClientProjections is separate from SelectExpression.Projection where former holds complex client side mapped projections and later holds only scalars
- What it enables
  - Single result subquery is not joined right away
  - pendingCollections are removed
  - Ability to bind with above subquery translations after projection has converted to index based binding (enabling pending selector future change)
  - Ability to bind client projections smoothly like ProjectionMember binding so suppose translations post-client eval where it is supported
  - Grouping.FirstOrDefault can add subquery projection which applies separately so we can combine aggregate/non-aggregate projection on grouping
- Introduce CollectionResultExpression which holds info on how to create collection for a subquery
- ApplyProjection converts projectionMember/Index based bindings to actual scalar projection and updates shaper in same pass
- Unique collectionId are assigned by shaper
- Change in flow to projection to let sqlTranslator determines translatibility before processing for client side
- Regression in collection over single result subquery
  - Earlier we applied single result right away copying over pending collection to outer which allowed single result subquery to convert apply to join
  - Now all client projections apply at same time so pending collection (equivalent subquery) is applied inside single result subquery so we fail to convert apply to join

Ground work for #20291, #12088, #13805
Resolves #23571
smitpatel added a commit that referenced this issue Apr 20, 2021
- ClientProjections is alternate when ProjectionMember binding is not possible using indexes
- ClientProjections is separate from SelectExpression.Projection where former holds complex client side mapped projections and later holds only scalars
- What it enables
  - Single result subquery is not joined right away
  - pendingCollections are removed
  - Ability to bind with above subquery translations after projection has converted to index based binding (enabling pending selector future change)
  - Ability to bind client projections smoothly like ProjectionMember binding so suppose translations post-client eval where it is supported
  - Grouping.FirstOrDefault can add subquery projection which applies separately so we can combine aggregate/non-aggregate projection on grouping
- Introduce CollectionResultExpression which holds info on how to create collection for a subquery
- ApplyProjection converts projectionMember/Index based bindings to actual scalar projection and updates shaper in same pass
- Unique collectionId are assigned by shaper
- Change in flow to projection to let sqlTranslator determines translatibility before processing for client side
- Regression in collection over single result subquery
  - Earlier we applied single result right away copying over pending collection to outer which allowed single result subquery to convert apply to join
  - Now all client projections apply at same time so pending collection (equivalent subquery) is applied inside single result subquery so we fail to convert apply to join

Ground work for #20291, #12088, #13805
Resolves #23571
ghost pushed a commit that referenced this issue Apr 20, 2021
- ClientProjections is alternate when ProjectionMember binding is not possible using indexes
- ClientProjections is separate from SelectExpression.Projection where former holds complex client side mapped projections and later holds only scalars
- What it enables
  - Single result subquery is not joined right away
  - pendingCollections are removed
  - Ability to bind with above subquery translations after projection has converted to index based binding (enabling pending selector future change)
  - Ability to bind client projections smoothly like ProjectionMember binding so suppose translations post-client eval where it is supported
  - Grouping.FirstOrDefault can add subquery projection which applies separately so we can combine aggregate/non-aggregate projection on grouping
- Introduce CollectionResultExpression which holds info on how to create collection for a subquery
- ApplyProjection converts projectionMember/Index based bindings to actual scalar projection and updates shaper in same pass
- Unique collectionId are assigned by shaper
- Change in flow to projection to let sqlTranslator determines translatibility before processing for client side
- Regression in collection over single result subquery
  - Earlier we applied single result right away copying over pending collection to outer which allowed single result subquery to convert apply to join
  - Now all client projections apply at same time so pending collection (equivalent subquery) is applied inside single result subquery so we fail to convert apply to join

Ground work for #20291, #12088, #13805
Resolves #23571
@smitpatel smitpatel modified the milestones: Backlog, 7.0.0 Nov 19, 2021
@smitpatel smitpatel added the punted-for-7.0 Originally planned for the EF Core 7.0 (EF7) release, but moved out due to resource constraints. label Aug 13, 2022
@smitpatel smitpatel modified the milestones: 7.0.0, Backlog Aug 13, 2022
@smitpatel smitpatel removed their assignment Sep 14, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-query punted-for-5.0 punted-for-7.0 Originally planned for the EF Core 7.0 (EF7) release, but moved out due to resource constraints. type-enhancement
Projects
None yet
Development

No branches or pull requests

2 participants