Query: Compiled Queries: Support "non-rooted" query expressions

[anpete]

User compiled queries allow for more expressiveness in creating query LINQ expressions. E.g.

var query = EF.CompileQuery(
                (NorthwindContext context)
                    => context.Customers.OrderBy(c => c.CustomerID).Select(c => c.CustomerID).FirstOrDefault() 
                        + context.Orders.OrderBy(o => o.CustomerID).Select(o => o.CustomerID).FirstOrDefault());

This particular query currently fails, I think because of a re-linq issue. In general, we should decide whether we want to spend time on making this work and testing it.

[Dunge]

I have a situation that looks like it fit in this issue.
Having multiple compiled queries I want them all to have the same list of .Include() elements, so I prepared an extension method to re-use the code:

public static IQueryable<ContainerBase> AddIncludes(this IQueryable<ContainerBase> baseExpression)
{
    return baseExpression.Include("Module").Include("DeviceNode");
}

Which I would call in multiple compiled queries looking like:

EF.CompileQuery<MyDataContext, int, ContainerBase>((ctx, contId) =>
    ctx.Container.AddIncludes().FirstOrDefault(d => d.Id == contId));

I get this exception:

System.NotSupportedException : 'Could not parse expression 'value(Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryable`1[Model.Containers.ContainerBase]).AddIncludes()': This overload of the method 'RepositoriesEF.QueryExtensions.AddIncludes' is currently not supported.'

But doing it directly in the compiled query works:

EF.CompileQuery<MyDataContext, int, ContainerBase>((ctx, contId) =>
    ctx.Container.Include("Module").Include("DeviceNode").FirstOrDefault(d => d.Id == contId));

[smitpatel]

@Dunge - Your issue is not same as reported issue here.
Compiled query uses lambda expression so whatever call you put inside the lambda will be converted to expression tree. Hence your function AddIncludes does not actually add include but put that method call in the tree. We don't have any knowledge of such user functions and it will never work. Using functions to create IQueryable only works for queries where you are not using the function inside a lambda expression.

[Dunge]

@smitpatel - Thank you. I'm sorry if the context doesn't match the original issue, I tried to search for a similar one not to create pollution, but I guess I just made things worse instead.

So there's no way to create a part of reusable expression tree inside lambda queries?

[smitpatel]

So there's no way to create a part of reusable expression tree inside lambda queries?

Yes, that is how compiler works and generates expression tree. Nothing EF Core can do about it.

[jnm2]

It may be helpful to show how moving IQueryable operations behind a method like that in an expression could be a dead end based on how C# and EF work.

Methods mentioned in expressions are not executed prior to translation:

(ctx, contId) => ctx.Container.AddIncludes().FirstOrDefault(d => d.Id == contId)

If Entity Framework had to execute AddIncludes before translating to a SQL query, it would also have to execute the whole thing.

[jnm2]

This:

EF.CompileQuery<MyDataContext, int, ContainerBase>((ctx, contId) =>
    ctx.Container.AddIncludes().FirstOrDefault(d => d.Id == contId));

Is sugar for:

(oops, I did (queryable, contId) => queryable.AddIncludes().FirstOrDefault(d => d.Id == contId) instead, but you get the point)

var queryableParameter = Expression.Parameter(typeof(IQueryable<ContainerBase>), "queryable");
var idParameter = Expression.Parameter(typeof(int), "id");
var dParameter = Expression.Parameter(typeof(ContainerBase), "d");

var expression = Expression.Lambda<Func<IQueryable<ContainerBase>, int, ContainerBase>>(
    Expression.Call(
        typeof(Queryable),
        nameof(Queryable.FirstOrDefault),
        typeArguments: new[] { typeof(ContainerBase) },
        new Expression[]
        {
            Expression.Call(typeof(YourClass).GetMethod("AddIncludes"), new[] { queryableParameter }),
            Expression.Quote(Expression.Lambda<Func<ContainerBase, bool>>(
                Expression.Equal(Expression.Property(dParameter, "Id"), idParameter),
                new[] { dParameter }))
        }),
    new[] { queryableParameter, idParameter });

EF.CompileQuery(expression);

Note how AddIncludes is referenced above. In order for the calls inside your AddIncludes method to become available, Entity Framework would have to somehow decompile your AddIncludes method and construct an expression which represents what it does.

[smitpatel]

This could be working in 3.1 release already.

[maumar]

Exception:

The 'InitializeStateManager' method has been called multiple times on the current query context. This method is intended to be called only once before query enumeration starts.
  Stack Trace: 
    QueryContext.InitializeStateManager(Boolean standAlone) line 137
    Enumerator.InitializeReader(DbContext _, Boolean result) line 217
    ExecutionStrategy.ExecuteImplementation[TState,TResult](Func`3 operation, Func`3 verifySucceeded, TState state) line 173
    ExecutionStrategy.Execute[TState,TResult](TState state, Func`3 operation, Func`3 verifySucceeded) line 160
    Enumerator.MoveNext()

[smitpatel]

Note to implementer: Remove the exception and make it no-op.