Skip to content

Query: support composing over GroupBy + First per group (navigations and joins)#38687

Open
ducmerida wants to merge 4 commits into
dotnet:mainfrom
ducmerida:fix/groupby-first-navigation-expansion
Open

Query: support composing over GroupBy + First per group (navigations and joins)#38687
ducmerida wants to merge 4 commits into
dotnet:mainfrom
ducmerida:fix/groupby-first-navigation-expansion

Conversation

@ducmerida

@ducmerida ducmerida commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Fixes #38686
Fixes #28125

Summary

Composing over the entity produced by a top-1-per-group query failed in two independent layers:

ctx.TimeSheets
    .GroupBy(t => t.OrderId)
    .Select(g => g.OrderBy(x => x.Id).First())
    .OrderBy(t => t.Order.Number)   // ← "Translation of member 'Order' ... failed" (misleading: it IS mapped)
  1. Navigation expansion built the result tree from Expression.Default, so the group element entity lost its entity reference and no navigation on it could be expanded (Query: Re-evaluate concept of pending selector in nav expansion #20291 territory).
  2. Even with the navigation expanded (or with a manual join, no navigations involved — The LINQ expression 'ProjectionBindingExpression: 0/1/2' could not be translated. #28125), the relational projection binder had no handling for a client projection slot holding a single-result subquery; its lowering only ran at the end of translation in ApplyProjection — "we don't allow this terms to compose (yet!)".

Implementation

  • NavigationExpandingExpressionVisitor.ProcessSelect(GroupBy…) now snapshots the selector structure before reducing (same mechanism ProcessDistinct uses for its pending selector), so a cardinality-reduced group element keeps its entity reference and later navigation accesses expand into regular joins over the grouped query.
  • The single-result to-one lowering is extracted from ApplyProjection into a shared core (LowerSingleResultSubqueryCore, parameterized on the projection sink — behavior-identical for the existing deferred path) and the projection binder now invokes it on demand when a composed query binds a single-result slot: pushdown if needed (remapping the binder's already-collected projections with the pushdown's remapping visitor), prunable to-one OUTER APPLY, shaper remap. Since the lowering mutates the select expression, the binder memoizes the lowered shaper per projection slot (same idiom as its existing _projectionBindingCache), so a projection referencing the slot more than once reuses the single join.

Scalar member access after the per-group First() already worked (#26748 fixed the second-projection case; OrderBy by scalar translates as a correlated scalar subquery). This PR adds the missing entity-level composition. Known follow-up: the join key over the group element still binds as a correlated scalar subquery in the ON clause instead of reusing the lowered join's column — correct but suboptimal (the duplication described in #20291).

All currently-failing shapes throw today, so no existing SQL baselines change.

Aware of #37859/#32957 (moving navigation expansion into translation, 12.0): the navigation-expansion piece is a targeted fix for the 11 pipeline; the on-demand lowering lives on the translation side, aligned with that direction, and the new tests pin the behavior for the rewrite.

Testing

…vigation expansion

ProcessSelect over a grouping built the resulting navigation tree from
Expression.Default, so the entity produced by a cardinality-reduced group
element (GroupBy(k).Select(g => g.OrderBy(...).First())) lost its entity
reference and any navigation accessed on it afterwards failed with the
misleading "Translation of member ... failed. This commonly occurs when the
specified member is unmapped." Snapshot the selector structure before
reducing, the same way ProcessDistinct preserves its pending selector, so
navigations on the result can still be expanded.
… composed over

Composing a join or an entity re-projection over
GroupBy(k).Select(g => g.First()) failed with "ProjectionBindingExpression
could not be translated": the projection binder had no handling for a client
projection slot holding a single-result subquery, and its lowering only
happened at the end of translation in ApplyProjection.

Extract the single-result to-one lowering from ApplyProjection into a shared
core (parameterized on the projection sink) and invoke it on demand from the
projection binder, pushing the select down first and remapping the binder's
already-collected projections with the pushdown's remapping visitor.

Fixes dotnet#28125
Navigation OrderBy/Where/projection on the group element entity and a manual
join over it, following the GroupBy_Select_Entire_Entity family; skipped on
InMemory like the rest of the family (dotnet#31209).

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes translation failures when composing further query operators over the entity produced by a “top-1-per-group” pattern (GroupBy(...).Select(g => g.OrderBy(...).First()/FirstOrDefault())), especially when later traversing navigations or performing joins. It does so by preserving entity shape through navigation expansion and by enabling on-demand lowering of single-result subqueries during relational projection binding.

Changes:

  • Navigation expansion now snapshots selector structure before reducing cardinality, preserving entity references for later navigation expansion.
  • Relational SelectExpression extracts/reuses the single-result subquery lowering logic and exposes an internal on-demand lowering hook used by the projection binder when a single-result subquery is composed over.
  • Adds new spec + SQL Server functional tests for navigation OrderBy/Where/projection and a manual Join over the per-group First() element; skips these in InMemory consistently with the existing family.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
test/EFCore.SqlServer.FunctionalTests/Query/NorthwindGroupByQuerySqlServerTest.cs Adds SQL assertions for the new GroupBy+First-per-group composition shapes on SQL Server.
test/EFCore.Specification.Tests/Query/NorthwindGroupByQueryTestBase.cs Adds new cross-provider specification tests covering navigation and join composition over the grouped “first” entity.
test/EFCore.InMemory.FunctionalTests/Query/NorthwindGroupByQueryInMemoryTest.cs Skips the new tests for InMemory in line with existing GroupBy/entire-entity limitations.
src/EFCore/Query/Internal/NavigationExpandingExpressionVisitor.cs Preserves selector structure across cardinality reduction to keep entity references for later navigation expansion.
src/EFCore.Relational/Query/SqlExpressions/SelectExpression.cs Refactors single-result subquery lowering into a shared core and adds an internal on-demand lowering entry point for projection binding.
src/EFCore.Relational/Query/Internal/RelationalProjectionBindingExpressionVisitor.cs Invokes on-demand lowering when binding a single-result subquery projection so composed queries can continue translating.

Comment thread src/EFCore.Relational/Query/SqlExpressions/SelectExpression.cs
The lowering mutates the select expression (pushdown + to-one join), so a
projection referencing the same slot twice would have added a duplicate join
of the same inner select. Cache the lowered shaper per slot in the projection
binder and add a test pinning the single-join shape.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

2 participants