Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Query: Make pending collections non-pending #23571

Closed
smitpatel opened this issue Dec 3, 2020 · 2 comments · Fixed by #24675
Closed

Query: Make pending collections non-pending #23571

smitpatel opened this issue Dec 3, 2020 · 2 comments · Fixed by #24675
Assignees
@smitpatel
Labels
area-query closed-fixed The issue has been fixed and is/will be included in the release indicated by the issue milestone. customer-reported type-enhancement
Milestone
6.0.0

Comments

@smitpatel
Copy link
Member

Rather than storing them internally + identifiers + childidentifiers

We should come up with some other indicators/structures so that they are added to select expression like other components but when we need to cause pushdown in main query we can do it with preserving the collection join as collection on outer level still.
@smitpatel smitpatel self-assigned this Dec 3, 2020
@smitpatel
Copy link
Member Author

Possible only after #17337

@ajcvickers ajcvickers added this to the Backlog milestone Dec 4, 2020
@smitpatel smitpatel removed the blocked label Mar 23, 2021
@smitpatel
Copy link
Member Author

Few ideas around this
Split Projection separate from ClientProjections. If we have a Expression kind ClientProjection then we can put a collection (or even single result) projection there directly and use the index in it during binding. Post-translation SelectExpression can expand out client projection to actual scalar column projection.
This would allow us to bind better after client projection if it appears in the middle of query as now we have a subquery on which we are composing further (remember to clone the query expression in ShapedQueryExpression though).
Also it works well with #20291 since both of above combined removes our need to identifier what can appear in the middle of query in projection or not.

smitpatel added a commit that referenced this issue Apr 16, 2021
@smitpatel
Query: Introduce client projections in SelectExpression
749d904 
- ClientProjections is alternate when ProjectionMember binding is not possible using indexes
- ClientProjections is separate from SelectExpression.Projection where former holds complex client side mapped projections and later holds only scalars
- What it enables
  - Single result subquery is not joined right away
  - pendingCollections are removed
  - Ability to bind with above subquery translations after projection has converted to index based binding (enabling pending selector future change)
  - Ability to bind client projections smoothly like ProjectionMember binding so suppose translations post-client eval where it is supported
  - Grouping.FirstOrDefault can add subquery projection which applies separately so we can combine aggregate/non-aggregate projection on grouping
- Introduce CollectionResultExpression which holds info on how to create collection for a subquery
- ApplyProjection converts projectionMember/Index based bindings to actual scalar projection and updates shaper in same pass
- Unique collectionId are assigned by shaper
- Change in flow to projection to let sqlTranslator determines translatibility before processing for client side
- Regression in collection over single result subquery
  - Earlier we applied single result right away copying over pending collection to outer which allowed single result subquery to convert apply to join
  - Now all client projections apply at same time so pending collection (equivalent subquery) is applied inside single result subquery so we fail to convert apply to join

Ground work for #20291, #12088, #13805
Resolves #23571
@smitpatel smitpatel mentioned this issue Apr 16, 2021
Merged
smitpatel added a commit that referenced this issue Apr 16, 2021
@smitpatel
Query: Introduce client projections in SelectExpression
b606ab8 
- ClientProjections is alternate when ProjectionMember binding is not possible using indexes
- ClientProjections is separate from SelectExpression.Projection where former holds complex client side mapped projections and later holds only scalars
- What it enables
  - Single result subquery is not joined right away
  - pendingCollections are removed
  - Ability to bind with above subquery translations after projection has converted to index based binding (enabling pending selector future change)
  - Ability to bind client projections smoothly like ProjectionMember binding so suppose translations post-client eval where it is supported
  - Grouping.FirstOrDefault can add subquery projection which applies separately so we can combine aggregate/non-aggregate projection on grouping
- Introduce CollectionResultExpression which holds info on how to create collection for a subquery
- ApplyProjection converts projectionMember/Index based bindings to actual scalar projection and updates shaper in same pass
- Unique collectionId are assigned by shaper
- Change in flow to projection to let sqlTranslator determines translatibility before processing for client side
- Regression in collection over single result subquery
  - Earlier we applied single result right away copying over pending collection to outer which allowed single result subquery to convert apply to join
  - Now all client projections apply at same time so pending collection (equivalent subquery) is applied inside single result subquery so we fail to convert apply to join

Ground work for #20291, #12088, #13805
Resolves #23571
smitpatel added a commit that referenced this issue Apr 20, 2021
@smitpatel
Query: Introduce client projections in SelectExpression
3e07fd2 
- ClientProjections is alternate when ProjectionMember binding is not possible using indexes
- ClientProjections is separate from SelectExpression.Projection where former holds complex client side mapped projections and later holds only scalars
- What it enables
  - Single result subquery is not joined right away
  - pendingCollections are removed
  - Ability to bind with above subquery translations after projection has converted to index based binding (enabling pending selector future change)
  - Ability to bind client projections smoothly like ProjectionMember binding so suppose translations post-client eval where it is supported
  - Grouping.FirstOrDefault can add subquery projection which applies separately so we can combine aggregate/non-aggregate projection on grouping
- Introduce CollectionResultExpression which holds info on how to create collection for a subquery
- ApplyProjection converts projectionMember/Index based bindings to actual scalar projection and updates shaper in same pass
- Unique collectionId are assigned by shaper
- Change in flow to projection to let sqlTranslator determines translatibility before processing for client side
- Regression in collection over single result subquery
  - Earlier we applied single result right away copying over pending collection to outer which allowed single result subquery to convert apply to join
  - Now all client projections apply at same time so pending collection (equivalent subquery) is applied inside single result subquery so we fail to convert apply to join

Ground work for #20291, #12088, #13805
Resolves #23571
@ghost ghost closed this as completed in #24675 Apr 20, 2021
ghost pushed a commit that referenced this issue Apr 20, 2021
@smitpatel
Query: Introduce client projections in SelectExpression (#24675)
7e2c5ce 
- ClientProjections is alternate when ProjectionMember binding is not possible using indexes
- ClientProjections is separate from SelectExpression.Projection where former holds complex client side mapped projections and later holds only scalars
- What it enables
  - Single result subquery is not joined right away
  - pendingCollections are removed
  - Ability to bind with above subquery translations after projection has converted to index based binding (enabling pending selector future change)
  - Ability to bind client projections smoothly like ProjectionMember binding so suppose translations post-client eval where it is supported
  - Grouping.FirstOrDefault can add subquery projection which applies separately so we can combine aggregate/non-aggregate projection on grouping
- Introduce CollectionResultExpression which holds info on how to create collection for a subquery
- ApplyProjection converts projectionMember/Index based bindings to actual scalar projection and updates shaper in same pass
- Unique collectionId are assigned by shaper
- Change in flow to projection to let sqlTranslator determines translatibility before processing for client side
- Regression in collection over single result subquery
  - Earlier we applied single result right away copying over pending collection to outer which allowed single result subquery to convert apply to join
  - Now all client projections apply at same time so pending collection (equivalent subquery) is applied inside single result subquery so we fail to convert apply to join

Ground work for #20291, #12088, #13805
Resolves #23571
@ajcvickers ajcvickers modified the milestones: Backlog, 6.0.0-preview5 Apr 28, 2021
@ajcvickers ajcvickers added the closed-fixed The issue has been fixed and is/will be included in the release indicated by the issue milestone. label Apr 28, 2021
@ajcvickers ajcvickers modified the milestones: 6.0.0-preview5, 6.0.0 Nov 8, 2021
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@smitpatel smitpatel

Labels
area-query closed-fixed The issue has been fixed and is/will be included in the release indicated by the issue milestone. customer-reported type-enhancement
Projects
None yet
Milestone
6.0.0
Development

Successfully merging a pull request may close this issue.

Query: Introduce client projections in SelectExpression dotnet/efcore
2 participants
@ajcvickers @smitpatel