Somehow a mismatch in the ShapedCommandContext entries resulting in KeyNotFoundException

[NinoFloris]

Error Message:
 System.Collections.Generic.KeyNotFoundException : The given key was not present in the dictionary.
Stack Trace:
   at System.ThrowHelper.ThrowKeyNotFoundException()
   at System.Collections.Generic.Dictionary`2.get_Item(TKey key)
   at Microsoft.EntityFrameworkCore.Query.Internal.ShaperCommandContext.CommandCacheKey.Equals(Object obj)
   at System.Collections.Generic.ObjectEqualityComparer`1.Equals(T x, T y)
   at System.Collections.Concurrent.ConcurrentDictionary`2.TryAddInternal(TKey key, Int32 hashcode, TValue value, Boolean updateIfExists, Boolean acquireLock, TValue& resultingValue)
   at System.Collections.Concurrent.ConcurrentDictionary`2.TryAdd(TKey key, TValue value)
   at Microsoft.EntityFrameworkCore.Query.Internal.ShaperCommandContext.GetRelationalCommand(IReadOnlyDictionary`2 parameters)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryingEnumerable.Enumerator.BufferlessMoveNext(Boolean buffer)
   at Microsoft.EntityFrameworkCore.Query.QueryMethodProvider.GetResult[TResult](IEnumerable`1 valueBuffers)
   at lambda_method(Closure , QueryContext )
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.<>c__DisplayClass20_0`1.<CompileQueryCore>b__0(QueryContext qc)
   at System.Linq.Queryable.Count[TSource](IQueryable`1 source)
   at Roadrunner.Authorization.Authorizers.Authorizer`1.IsAuthorized(Boolean doAction) in /#PATH#/Authorizers/Authorizer.cs:line 65
   at Roadrunner.Authorization.Authorizers.Authorizer`1.Act() in /#PATH#/Authorizers/Authorizer.cs:line 38
   at Roadrunner.Features.PageRelations.Delete.Handler.<Handle>d__7.MoveNext() in /#PATH#/PageRelations/Delete.cs:line 60
--- End of stack trace from previous location where exception was thrown ---

This check does not do a ContainsKey before trying to access the key in the other cache entry:

foreach (var parameterValue in _parameterValues)
{
    var value = parameterValue.Value;
    var otherValue = other._parameterValues[parameterValue.Key];
...

https://github.com/aspnet/EntityFramework/blob/f386095005e46ea3aa4d677e4439cdac113dbfb1/src/EFCore.Relational/Query/Internal/ShaperCommandContext.cs#L44

Steps to reproduce

It requires a few tries to reproduce as it's non deterministic due to the cache entries needing to get in there in the right order.

I don't know how the ShapedCommandContext works, when a new one is constructed or how only this one has mismatching arguments but I hope you can shed some light on this

Further technical details

EF Core version: dev
Database Provider: Npgsql
Operating system: OSX
IDE: VSCode dotnet cli 1.0 tooling

[NinoFloris]

I'd think this is quite a problem. We don't know if we release an alteration of this code (that doesn't crash EF) that it will not show up in production under load in any other way. Basically somewhere the underlying model is broken in a way that is quite scary.

Please at least give some response, any at all is better than this complete silence.

Mind you the reproduction of this issue is 100% within a few tries so this is not some sort of heisenbug you should just ignore

/cc @ajcvickers

[ajcvickers]

@NinoFloris We triage issues multiple times per week. This issue was filed after our first triage this week and our second one for this week is today. We will assign someone to investigate at that time. However, from looking at the info above, I suspect we are going to need more information to be able to reproduce it. A simplified complete project or code listing would be ideal. If that's not possible, then at least some information about what your model looks like and what queries are being executed.

[NinoFloris]

It took us a while to pinpoint the exact cause but this is the smallest we can get :)

https://gist.github.com/NinoFloris/2c91585c047050e00e388e94edc09a98

dotnet new console with package:

<PackageReference Include="Microsoft.EntityFrameworkCore.Sqlite" Version="1.1.0-*" />

[NinoFloris]

So how does this problem actually manifest?

I definitely know it is an expression visiting problem but where I would have to look then is unknown terrain for me.

I presume there is an expression visitor that decides that the nested function should be visited.
Then another visitor goes and finds all the parameters used in the expression tree and adds them to the ShaperCommandContext? Then my question is, how and by what piece of code are these two different queries incorrectly conflated?

Is there a visitor that decides query shape equality? E.g. same where's, from, joins, projection and other expression bits but fails to take into account the argument names when they are not db parameters?

So as I've asked in my first post already:

I don't know how the ShapedCommandContext works, when a new one is constructed or how only this one has mismatching arguments but I hope you can shed some light on this

I do really like to know the answer to:
How does EF select the ShaperCommandContext to share between queries?

My educated guess is that is where it fails to take into account a nuance around nested functions that cannot be translated to sql with respect to their arguments not being db parameters, therefore failing to distinguish they are different queries.