Contains to OPENJSON translation regresses performance #32394

Suchiman · 2023-11-22T16:52:04Z

After upgrading to EFC8, we've run into several severe performance regressions where millisecond queries started timeouting.
This is due to EFC8 now generating

DECLARE @__p_1 int = 0
DECLARE @__p_2 int = 25
DECLARE @__profileIds_0 nvarchar(4000) = N'[923]'
SELECT [a].[AmsPk] AS [Id], [a].[Bearbeitet], [a].[Amsidnr]
FROM [AmsKunden] AS [a]
INNER JOIN [StorageProfiles] AS [s] ON [a].[Profile_Id] = [s].[Id]
WHERE [s].[Id] IN (
    SELECT [p].[value]
    FROM OPENJSON(@__profileIds_0) WITH ([value] int '$') AS [p]
)
ORDER BY [a].[Bearbeitet] DESC
OFFSET @__p_1 ROWS FETCH NEXT @__p_2 ROWS ONLY

where it used to generate

DECLARE @__p_1 int = 0
DECLARE @__p_2 int = 25
SELECT [a].[AmsPk] AS [Id], [a].[Bearbeitet], [a].[Amsidnr]
FROM [AmsKunden] AS [a]
INNER JOIN [StorageProfiles] AS [s] ON [a].[Profile_Id] = [s].[Id]
WHERE [s].[Id] IN (923)
ORDER BY [a].[Bearbeitet] DESC
OFFSET @__p_1 ROWS FETCH NEXT @__p_2 ROWS ONLY

from my analysis, the problem here is that the cardinality estimator flat assumes that OPENJSON will return 50 rows. If the column that you're filtering on is not very selective, that is enough to dissuade SQL Server from seeking that index. In addition, it also dissuades it from using filtered indexes which requires constants but that's orthogonal. I have a lot of queries where i do .Where(x => col.Contains(x.SomeId)) where col contains in 99% of the time just one element, with the very rare 0 or occasional 2 elements (although more are possible in theory).

Since that is absolutely blocking, i had to apply the CompatLevel 120 hack but i consider that quite the nuclear option, especially since we would like to use more of the ToJson features. The only other option i could see to get around that was to apply a FORCESEEK hint but this isn't (well) supported in EFC either.

CompatLevel 120 works for now but i don't think that's a permanent solution. Query Cache poisoning and frequent recompilations are not remotely as expensive as queries that regress from milliseconds to "can't finish in 120s", so this feature comes at a trade off that is not worth it for us. The naive better solution to workaround this would be similiar to the SplitQuery feature (that has both a global and query level switch).

Include provider and version information

EF Core version: 8.0
Database provider: Microsoft.EntityFrameworkCore.SqlServer
Target framework: .NET 8.0
Operating system: Windows 10
IDE: Visual Studio 2022 17.9P1

The text was updated successfully, but these errors were encountered:

roji · 2023-11-23T00:10:18Z

@Suchiman is it possible for you to provide a repro for this performance regression, i.e. data and a query which show the new OPENJSON-based approach performing significantly worse? That would help deciding what to do here.

stevendarby · 2023-11-23T08:19:58Z

@Suchiman are the FORCESEEKs in your queries above generated by EF?

@roji sorry if this clouds matters but a problem stemming from cardinality estimates was raised in the original issue and I don't think it was tracked in a separate issue as you requested #13617 (comment). Just a little reminder now in case it's useful to consider alongside this issue.

Suchiman · 2023-11-23T08:37:13Z

are the FORCESEEKs in your queries above generated by EF?

sorry, no, copied the wrong thing where i was tinkering if query hints would help.

data and a query which show the new OPENJSON-based approach performing significantly worse?

I'll try, the databases i'm working with are anywhere between 400GB and 3TB right now so this might need a whole lot of data to start being reproducible.

ErikEJ · 2023-11-23T10:04:59Z

@Suchiman How could EF Core determine that not using openjson is better in this case?

Suchiman · 2023-11-23T10:33:34Z

@Suchiman How could EF Core determine that not using openjson is better in this case?

Not using OPENJSON is likely always better for query runtime as SQL Server can optimize constants a lot better than an opaque input, there are tradeoffs however on query compilation times, tradeoffs that EFC cannot reasonably make assuming it will always improve things. Thus why i suggested of making it a query level option, akin to AsSplitQuery.

roji · 2023-11-24T11:08:25Z

@Suchiman @stevendarby thanks for your comments; I'm currently away but will be back next and will fully look into this in more detail.

I agree that there needs to be a way to make EF generate constants for Contains, ideally on a per-query basis. I don't think a new operator is needed - such as AsSplitQuery; what's needed is a way to force the array parameter to be interpreted by EF as a constant instead of as a parameter - that's a more general need that we know we have (there are other places where this is a problem). In other words, I think that to force constantization you'd write something like the following (see #31552):

var ids = new[] { 1, 2 };
_ = await ctx.Blogs.Where(b => EF.Constant(ids).Contains(b.Id)).ToListAsync();

In the meantime, as a pretty verbose workaround, you can use the Expression APIs to produce that exact expression tree:

var ids = new[] { 1, 2 };
ContainsMethodInfo ??= typeof(Enumerable).GetMethods()
    .Single(m => m.Name == nameof(Enumerable.Contains) && m.GetParameters().Length == 2);

var parameter = Expression.Parameter(typeof(Blog), "b");
var predicate = Expression.Lambda<Func<Blog, bool>>(
    Expression.Call(
        ContainsMethodInfo.MakeGenericMethod(typeof(int)),
        Expression.Constant(ids),
        Expression.Property(parameter, nameof(Blog.Id))),
    parameter);

var results = await ctx.Blogs.Where(predicate).ToListAsync();

This produces the same query tree as when the array is written inline in the query:

var results = await ctx.Blogs.Where(b => new[] { 1, 2 }.Contains(b.Id)).ToListAsync();

... and generates the desired SQL:

SELECT [b].[Id], [b].[Name]
FROM [Blogs] AS [b]
WHERE [b].[Id] IN (1, 2)

This is by no means a satisfactory workaround - I know - but I'm posting it here in case you want to keep the SQL Server compatibility level (for other queries) and override the behavior on a per-query basis, today with 8.0.0.

One note: if you know you have a certain number of elements in the parameterized array, you can use an inline array as follows:

var result = await ctx.Blogs.Where(b => new[] { ids[0], ids[1] }.Contains(b.Id)).ToListAsync();

This is recommended for when you know the number of elements, producing the following SQL:

SELECT [b].[Id], [b].[Name]
FROM [Blogs] AS [b]
WHERE [b].[Id] IN (@__p_0, @__p_1)

In any case, I'll take a look at how EF.Constant support would look like and whether it's even possible to consider it for an 8.0 patch.

ErikEJ · 2023-11-24T15:32:00Z

@Suchiman You might find this extension method useful, uses parameters and simple ORs https://gist.github.com/ErikEJ/6ab62e8b9c226ecacf02a5e5713ff7bd

stevendarby · 2023-11-24T19:56:25Z

@Suchiman I would just like to understand what's happening on the SQL Server side a bit more, if that's ok! Regarding this:

the problem here is that the cardinality estimator flat assumes that OPENJSON will return 50 rows. If the column that you're filtering on is not very selective, that is enough to dissuade SQL Server from seeking that index.

Is the implication here that SQL Server would also avoid seeking the index with an IN containing 50 hardcoded values (because it would know there are 50 and not just estimate that)? If so, and turns out it's really slow doing that - doesn't that seem like an odd decision for SQL Server to make? Are your statistics up to date - could that influence its decision?

Out of interest, does adding TOP 1 in the OPENJSON subquery correct its estimate and get it to use the index? I was half wondering if bucketizing a hint like that, so that the estimate is at least within the right order of magnitude, could be a solution: TOP 1, TOP 10, TOP 100 etc. Or maybe there is some other kind of hint that could be used. It would lead to multiple query plans for different sizes, but each might be better performing and the number of them still much lower than pre-EF 8. I've not investigated this though and it might be a complete non-starter.

Anyway, @roji's suggestion seems like a good way forward.

roji · 2023-11-24T21:31:45Z

I also had @stevendarby's good questions in mind, and definitely still would like to see some sort of repro for this so that we can dig deeper into exactly what's going on... That could help us decide to what extent a patch here is needed for 8.0.0.

Suchiman · 2023-11-27T09:10:38Z

Is the implication here that SQL Server would also avoid seeking the index with an IN containing 50 hardcoded values (because it would know there are 50 and not just estimate that)?

Well SQL Server is really an oddball here. If i use WHERE [s].[Id] IN (@p1,@p2) then having 2 params is already enough to make it choose badly (estimated plan but it's the same bad shape):

If using hardcoded constants like WHERE [s].[Id] IN (1,2,3,4,5,6...), then it takes 61 constants before it tips over (estimated plan but it's the same bad shape)

If so, and turns out it's really slow doing that - doesn't that seem like an odd decision for SQL Server to make? Are your statistics up to date - could that influence its decision?

Doing a statistics update with fullscan is always the first thing i try, i wish that would help all the time 😆

Out of interest, does adding TOP 1 in the OPENJSON subquery correct its estimate and get it to use the index?

That does indeed work, and it seems like for SQL Server, the tipping point is exactly 50 in my case, with TOP (49) i still get the fast plan, with TOP (50) its the slow plan. One could think to make EFC generate TOP(@p_n), that allows both query plan caching and through the magic of parameter sniffing, to get a better plan, including all the gotchas of parameter sniffing such as getting a bad reused plan as well.

roji · 2023-11-27T10:26:16Z

Thanks for all the extra info and experimentation @Suchiman, that's definitely useful. I know it isn't easy, but some sort of repro for this would allow further investigation and possibly taking this to the SQL Server people to maybe get more insights.

Otherwise, I'm generally averse to EF generating something that's super-tailored to an internal SQL Server quirk (i.e. the TOP - with what exact threshold etc.)... If we see that this is indeed some sort of general thing for all queries using IN, that might make sense, but otherwise I'm not sure what we can do here. You should at least be able to get EF to generate the TOP yourself, i.e. by composing a Take on the parameterized array.

roji · 2023-12-04T18:45:57Z

FYI everyone, the EF.Constant solution should be there for 8.0.2. I'm still definitely interested in understanding the perf characteristics here deeper, but we're going to need to see some sort of repro - I hope you can help with that.

stevendarby · 2023-12-04T23:45:30Z

Hi @roji, had a play with EF.Constant on the daily builds and it looks like EF compiles a query for each permutation of values passed to EF.Constant. I know the pre-EF 8 approach couldn't cache the SQL due to its hardcoded values, but pretty sure the query was cached in some form to avoid a full compile each time? Just thinking there may still be an argument for a per-query option to revert to the old behaviour (i.e. the suggestion OP put forward) if EF.Constant isn't the magic bullet. Still need good repros to prove the usefulness of that though...

roji · 2023-12-05T06:32:16Z

@stevendarby you're right - this is indeed a difference between the new EF.Constant and the old behavior. EF's relational layer contains two levels of caching - one very early one based on the query tree itself, and a later one based only on the nullability of parameters (since SQL varies based on that). The old behavior had the same tree for the 1st cache, but specifically prevented use of the 2nd cache; because EF.Constant integrates the constants very early on in the query tree, that causes a miss in the 1st cache, causing the entire compilation to happen again.

When working on EF.Constant, I briefly considered trying to implement it in a way which doesn't defeat EF's 1st cache.. However, that's considerably more complex/risky (and the point here was to prepare a patch for 8.0, which must be relatively simple/low-risk). In addition, EF.Constant is useful for other scenarios where one really does want to integrate constants in the tree. It's true that a specific flag for Contains (as opposed to the more general EF.Constant) would be relatively easy, but we generally try to avoid having such things unless absolutely necessary.

Note that the majority of other issues related to the new Contains translation have been fixed (or are in the process of being fixed), so I hope EF.Constant won't be needed that much (though the possible performance issues described above do remain).

In any case, users now have an efficient, global (SQL Server) option to disable OPENJSON entirely, and will have EF.Constant as an EF-expensive but per-query option via EF.Constant. I do agree that there's probably still a "gap" there, i.e. a per-query option that's more efficient. There may even be a need for a global flag to opt out of JSON subquery translation for Contains specifically, for all queries (JSON subqueries are absolutely required for most/all queries composed over primitive collections, except for Contains where there's the alternative - so that may make sense). But it seems prudent to wait and see how it all works for users - as well as get a better understanding of the actual perf impact here, with a repro - and if really needed, do another patch later on.

Does that make sense?

Nefarion · 2023-12-05T14:07:22Z

I would also like to report a massive slowdown with the new OpenJSON query style:

Query with IN: <1 sec (ssms reports 0)
Query with OpenJson: 7m 40s

In my case there are 1 218 265 rows in the table, and 609 values in the contains query.
The lookup is on the PK of the table -> .Where(x => lookup.Contains(x.PK))
The query returns 423 rows.

I am working around this right now by using .ToHashTable() instead of .ToArray() which thankfully circumvents the OpenJSON style query.

I have no experience with execution plans, but it seems to me as if the OpenJSON is executed for every row the table, which results in significant overhead of json parsing. (738 111 845 rows returned in my case, which is close to 1 218 265 * 609)

RyanONeill1970 · 2023-12-05T14:34:21Z

Here too, we've refactored to remove any usage of OpenJson as it killed a lot of database connections. Previously less than a second, afterwards we'd get timeouts at 30 seconds.

I've anonymised the generated SQL but generally, it was of the form below. Don't judge the massive parameter list, it's grown over time. I would not do it that way now. The param list has been obfuscated to prevent product codes being identified but looked like '1234567890123,1234567890124, etc...'.

exec sp_executesql N'SELECT [o].[Col1], LTRIM(RTRIM([o].[Col2])), [o].[ProductId], [p].[VariantId], [c].[ColourId], [p].[Name], LTRIM(RTRIM([o].[Col3])), [c].[Col4], [o].[ProductSize], [t0].[ImageId], [t0].[Version], [t0].[c]
FROM (
    SELECT * FROM Product WHERE Col3 IN (@p0)
) AS [o]
INNER JOIN [Variant] AS [p] ON [o].[VariantId] = [p].[VariantId]
INNER JOIN [Colour] AS [c] ON [o].[ColourId] = [c].[ColourId]
LEFT JOIN (
    SELECT [t].[ImageId], [t].[Version], [t].[c], [t].[VariantId], [t].[ColourId]
    FROM (
        SELECT [p0].[VariantImageID] AS [ImageId], [p0].[Version], 1 AS [c], [p0].[VariantId], [p0].[ColourId], ROW_NUMBER() OVER(PARTITION BY [p0].[VariantId], [p0].[ColourId] ORDER BY [p0].[PhotoTypeId]) AS [row]
        FROM [VariantImage] AS [p0]
    ) AS [t]
    WHERE [t].[row] <= 1
) AS [t0] ON [p].[VariantId] = [t0].[VariantId] AND [o].[ColourId] = [t0].[ColourId]
ORDER BY [o].[ProductId], [p].[VariantId], [c].[ColourId]',N'@p0 nvarchar(max) ',@p0=N'   xxxx 36000 chars representing a list of 10 character codes xxxx as in 1234567890123,1234567890124, etc...  '

roji · 2023-12-05T18:14:52Z

@Nefarion @RyanONeill1970 thanks for your reports - we're actively looking into OPENJSON-related issues at the moment so this is important. However, it's hard to understand the exact source of slowdown from an simple "it regressed" report, without some sort of repro. Is it possible to put together a repro for the problematic query, or at the very least, post the full SQL of the affected query (@RyanONeill1970 your query doesn't seem to contain any OPENJSON)?

RyanONeill1970 · 2023-12-05T18:38:08Z

Duh, that capture must be after I reworked it. Sorry. It's not just that one, we've replaced a few calls which were going slow.
I'll keep an eye out for any more, we've only just upgraded.

Nefarion · 2023-12-06T10:13:59Z

@roji

The OpenJSON Query looks like this (anonymized table/colum/db names and query parameters)

DECLARE @__ids_0 nvarchar(max)
SET
    @__ids_0 = '["abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345",,"abcde12345","abcde12345",,"abcde12345",,"abcde12345","abcde12345",,"abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345",null,"abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345","abcde12345"]'
SELECT
    [p].[Col1],
    [p].[Col2],
    [p].[Col3],
    [p].[Col4],
    [p].[Col5],
    [p].[Col6],
    [p].[Col7],
    [p].[Id],
    [p].[Col8],
    [p].[Col9],
    [p].[Col10],
    [p].[Col11],
    [p].[Col12]
FROM
    [Tbl] AS [p]
WHERE
    EXISTS (
        SELECT
            1
        FROM
            OPENJSON(@__ids_0) WITH ([value] nvarchar(450) '$') AS [a]
        WHERE
            [a].[value] = [p].[Id]
            OR (
                [a].[value] IS NULL
                AND [p].[Id] IS NULL
            )
    )

The IN Query looks like this:

SELECT
    [p].[Col1],
    [p].[Col2],
    [p].[Col3],
    [p].[Col4],
    [p].[Col5],
    [p].[Col6],
    [p].[Col7],
    [p].[Id],
    [p].[Col8],
    [p].[Col9],
    [p].[Col10],
    [p].[Col11],
    [p].[Col12]
FROM
    [Tbl] AS [p]
WHERE
    [p].[Id] IN ('abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', null, 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345', 'abcde12345')

QueryPlans:
IN.txt
OpenJSON.txt

Edit: I don't know if it matters, but Tbl is a View in both queries

molesinski · 2023-12-06T10:53:03Z

We have the same issue. Performance drop is caused by OPENJSON version making Azure SQL ignore non clustered indexes on large tables and forcing it to fall back to full clustered index scan. I do no have source SQL, but when Contains is replaced from IN construct to OPENJSON construct it just prevents non clustered index usage. The query is simple, just SELECT FROM WHERE with 3 single column conditions in where, 2 of them using Contains, while all 3 columns being present in clustered index.

nh43de · 2023-12-06T19:43:13Z

I'm having this same issue with the Sqlite provider - in .NET 7 it would generate WHERE [Name] IN ( <<names>> ), now in 8 it is generating a WHERE EXISTS plus json parsing. The performance went from milliseconds to now timing out. My guess is now it is doing a table scan instead of being able to rely on an index on the [Name] field. The db query optimizer should in theory be smart enough to handle optimizing with the JSON, but clearly it isn't. On the bright side, at least now it's parameterized for better plan cache utilization.

Here's part of the new Sqlite SQL output for the .Contains translation:

.param set @__searchIndexResults_0 '[ <<names>> ]'

WHERE EXISTS (
    SELECT 1
    FROM json_each(@__searchIndexResults_0) AS "s1"
    WHERE "s1"."value" = "s"."Name" )

To repro this simply use something like this


class User {
    public int UserId { get; set; }
    public string Name { get; set; }
}

Add the above to your context - now query:

            var names = new[] { "name1", "name2" };

            var rr = from user in _context.Users
                    where names.Contains(user.Name)
                    select user;

nh43de · 2023-12-06T20:02:08Z

Ok so an update/workaround - if you can re-write your .Contains() as a join instead, the performance will be almost exactly as before:

            var names = new[] { "name1", "name2" };

            var rr = from user in _context.Users
                    join name in Names
                        on user.Name equals name
                    select user;

Which gets translated to

.param set @__p_0 '[ <<values>> ]'

SELECT ...
FROM "Users" AS "u"
INNER JOIN json_each(@__p_0) AS "p" ON "u"."Name" = "p"."value"

On Sqlite. Performance is fantastic - much better than before! And now it's parameterized so should be even faster than .net7 after query compilation.

--

But now I'm going to have to re-write most of my .Contains(), and this workaround will not work with .Contains() == false. E.g. cases where I want to retreive all users except ones in my list.

roji · 2023-12-07T07:36:09Z

@nh43de thanks for the information on SQLite I'll look into this as well. Note that I considered INNER JOIN here as well when making the EF changes; but the problem is that if the array contains duplicates, the principal rows get duplicated as well. This makes INNER JOIN unsuitable for Contains unless we do apply Distinct somehow on the client, before sending the parameter to the server.

For everyone else, in the absence of any repro above, I'll try to do some additional perf experimentation on SQL Server; in my investigations last year the performance generally seemed good (slightly slower than IN+constants, but definitely nothing that prevents index usage).

But a simple, minimal repro would go a long way to help with this - this is where your help can be very valuable.

Nefarion · 2023-12-07T08:34:30Z

@roji

-- Setup Table with PK1 -> PK1000000
SELECT TOP (1000000) Id = Concat('PK', CONVERT(INT, ROW_NUMBER() OVER (ORDER BY s1.[object_id])))
INTO TestTbl
FROM sys.all_objects AS s1 CROSS JOIN sys.all_objects AS s2

-- Add Primary Key
ALTER TABLE TestTbl
ADD PRIMARY KEY (Id);


-- Fast query (<1 sec)
SELECT [Id] FROM TestTbl WHERE [Id]
IN ('PK345','PK1345','PK2345','PK3345','PK4345','PK5345','PK6345','PK7345','PK8345','PK9345','PK10345','PK11345','PK12345','PK13345','PK14345','PK15345','PK16345','PK17345','PK18345','PK19345','PK20345','PK21345','PK22345','PK23345','PK24345','PK25345','PK26345','PK27345','PK28345','PK29345','PK30345','PK31345','PK32345','PK33345','PK34345','PK35345','PK36345','PK37345','PK38345','PK39345','PK40345','PK41345','PK42345','PK43345','PK44345','PK45345','PK46345','PK47345','PK48345','PK49345','PK50345','PK51345','PK52345','PK53345','PK54345','PK55345','PK56345','PK57345','PK58345','PK59345','PK60345','PK61345','PK62345','PK63345','PK64345','PK65345','PK66345','PK67345','PK68345','PK69345','PK70345','PK71345','PK72345','PK73345','PK74345','PK75345','PK76345','PK77345','PK78345','PK79345','PK80345','PK81345','PK82345','PK83345','PK84345','PK85345','PK86345','PK87345','PK88345','PK89345','PK90345','PK91345','PK92345','PK93345','PK94345','PK95345','PK96345','PK97345','PK98345','PK99345','PK100345','PK101345','PK102345','PK103345','PK104345','PK105345','PK106345','PK107345','PK108345','PK109345','PK110345','PK111345','PK112345','PK113345','PK114345','PK115345','PK116345','PK117345','PK118345','PK119345','PK120345','PK121345','PK122345','PK123345','PK124345','PK125345','PK126345','PK127345','PK128345','PK129345','PK130345','PK131345','PK132345','PK133345','PK134345','PK135345','PK136345','PK137345','PK138345','PK139345','PK140345','PK141345','PK142345','PK143345','PK144345','PK145345','PK146345','PK147345','PK148345','PK149345','PK150345','PK151345','PK152345','PK153345','PK154345','PK155345','PK156345','PK157345','PK158345','PK159345','PK160345','PK161345','PK162345','PK163345','PK164345','PK165345','PK166345','PK167345','PK168345','PK169345','PK170345','PK171345','PK172345','PK173345','PK174345','PK175345','PK176345','PK177345','PK178345','PK179345','PK180345','PK181345','PK182345','PK183345','PK184345','PK185345','PK186345','PK187345','PK188345','PK189345','PK190345','PK191345','PK192345','PK193345','PK194345','PK195345','PK196345','PK197345','PK198345','PK199345','PK200345','PK201345','PK202345','PK203345','PK204345','PK205345','PK206345','PK207345','PK208345','PK209345','PK210345','PK211345','PK212345','PK213345','PK214345','PK215345','PK216345','PK217345','PK218345','PK219345','PK220345','PK221345','PK222345','PK223345','PK224345','PK225345','PK226345','PK227345','PK228345','PK229345','PK230345','PK231345','PK232345','PK233345','PK234345','PK235345','PK236345','PK237345','PK238345','PK239345','PK240345','PK241345','PK242345','PK243345','PK244345','PK245345','PK246345','PK247345','PK248345','PK249345','PK250345','PK251345','PK252345','PK253345','PK254345','PK255345','PK256345','PK257345','PK258345','PK259345','PK260345','PK261345','PK262345','PK263345','PK264345','PK265345','PK266345','PK267345','PK268345','PK269345','PK270345','PK271345','PK272345','PK273345','PK274345','PK275345','PK276345','PK277345','PK278345','PK279345','PK280345','PK281345','PK282345','PK283345','PK284345','PK285345','PK286345','PK287345','PK288345','PK289345','PK290345','PK291345','PK292345','PK293345','PK294345','PK295345','PK296345','PK297345','PK298345','PK299345','PK300345','PK301345','PK302345','PK303345','PK304345','PK305345','PK306345','PK307345','PK308345','PK309345','PK310345','PK311345','PK312345','PK313345','PK314345','PK315345','PK316345','PK317345','PK318345','PK319345','PK320345','PK321345','PK322345','PK323345','PK324345','PK325345','PK326345','PK327345','PK328345','PK329345','PK330345','PK331345','PK332345','PK333345','PK334345','PK335345','PK336345','PK337345','PK338345','PK339345','PK340345','PK341345','PK342345','PK343345','PK344345','PK345345','PK346345','PK347345','PK348345','PK349345','PK350345','PK351345','PK352345','PK353345','PK354345','PK355345','PK356345','PK357345','PK358345','PK359345','PK360345','PK361345','PK362345','PK363345','PK364345','PK365345','PK366345','PK367345','PK368345','PK369345','PK370345','PK371345','PK372345','PK373345','PK374345','PK375345','PK376345','PK377345','PK378345','PK379345','PK380345','PK381345','PK382345','PK383345','PK384345','PK385345','PK386345','PK387345','PK388345','PK389345','PK390345','PK391345','PK392345','PK393345','PK394345','PK395345','PK396345','PK397345','PK398345','PK399345','PK400345','PK401345','PK402345','PK403345','PK404345','PK405345','PK406345','PK407345','PK408345','PK409345','PK410345','PK411345','PK412345','PK413345','PK414345','PK415345','PK416345','PK417345','PK418345','PK419345','PK420345','PK421345','PK422345','PK423345','PK424345','PK425345','PK426345','PK427345','PK428345','PK429345','PK430345','PK431345','PK432345','PK433345','PK434345','PK435345','PK436345','PK437345','PK438345','PK439345','PK440345','PK441345','PK442345','PK443345','PK444345','PK445345','PK446345','PK447345','PK448345','PK449345','PK450345','PK451345','PK452345','PK453345','PK454345','PK455345','PK456345','PK457345','PK458345','PK459345','PK460345','PK461345','PK462345','PK463345','PK464345','PK465345','PK466345','PK467345','PK468345','PK469345','PK470345','PK471345','PK472345','PK473345','PK474345','PK475345','PK476345','PK477345','PK478345','PK479345','PK480345','PK481345','PK482345','PK483345','PK484345','PK485345','PK486345','PK487345','PK488345','PK489345','PK490345','PK491345','PK492345','PK493345','PK494345','PK495345','PK496345','PK497345','PK498345','PK499345','PK500345','PK501345','PK502345','PK503345','PK504345','PK505345','PK506345','PK507345','PK508345','PK509345','PK510345','PK511345','PK512345','PK513345','PK514345','PK515345','PK516345','PK517345','PK518345','PK519345','PK520345','PK521345','PK522345','PK523345','PK524345','PK525345','PK526345','PK527345','PK528345','PK529345','PK530345','PK531345','PK532345','PK533345','PK534345','PK535345','PK536345','PK537345','PK538345','PK539345','PK540345','PK541345','PK542345','PK543345','PK544345','PK545345','PK546345','PK547345','PK548345','PK549345','PK550345','PK551345','PK552345','PK553345','PK554345','PK555345','PK556345','PK557345','PK558345','PK559345','PK560345','PK561345','PK562345','PK563345','PK564345','PK565345','PK566345','PK567345','PK568345','PK569345','PK570345','PK571345','PK572345','PK573345','PK574345','PK575345','PK576345','PK577345','PK578345','PK579345','PK580345','PK581345','PK582345','PK583345','PK584345','PK585345','PK586345','PK587345','PK588345','PK589345','PK590345','PK591345','PK592345','PK593345','PK594345','PK595345','PK596345','PK597345','PK598345','PK599345','PK600345','PK601345','PK602345','PK603345','PK604345','PK605345','PK606345','PK607345','PK608345','PK609345','PK610345','PK611345','PK612345','PK613345','PK614345','PK615345','PK616345','PK617345','PK618345','PK619345','PK620345','PK621345','PK622345','PK623345','PK624345','PK625345','PK626345','PK627345','PK628345','PK629345','PK630345','PK631345','PK632345','PK633345','PK634345','PK635345','PK636345','PK637345','PK638345','PK639345','PK640345','PK641345','PK642345','PK643345','PK644345','PK645345','PK646345','PK647345','PK648345','PK649345','PK650345','PK651345','PK652345','PK653345','PK654345','PK655345','PK656345','PK657345','PK658345','PK659345','PK660345','PK661345','PK662345','PK663345','PK664345','PK665345','PK666345','PK667345','PK668345','PK669345','PK670345','PK671345','PK672345','PK673345','PK674345','PK675345','PK676345','PK677345','PK678345','PK679345','PK680345','PK681345','PK682345','PK683345','PK684345','PK685345','PK686345','PK687345','PK688345','PK689345','PK690345','PK691345','PK692345','PK693345','PK694345','PK695345','PK696345','PK697345','PK698345','PK699345','PK700345','PK701345','PK702345','PK703345','PK704345','PK705345','PK706345','PK707345','PK708345','PK709345','PK710345','PK711345','PK712345','PK713345','PK714345','PK715345','PK716345','PK717345','PK718345','PK719345','PK720345','PK721345','PK722345','PK723345','PK724345','PK725345','PK726345','PK727345','PK728345','PK729345','PK730345','PK731345','PK732345','PK733345','PK734345','PK735345','PK736345','PK737345','PK738345','PK739345','PK740345','PK741345','PK742345','PK743345','PK744345','PK745345','PK746345','PK747345','PK748345','PK749345','PK750345','PK751345','PK752345','PK753345','PK754345','PK755345','PK756345','PK757345','PK758345','PK759345','PK760345','PK761345','PK762345','PK763345','PK764345','PK765345','PK766345','PK767345','PK768345','PK769345','PK770345','PK771345','PK772345','PK773345','PK774345','PK775345','PK776345','PK777345','PK778345','PK779345','PK780345','PK781345','PK782345','PK783345','PK784345','PK785345','PK786345','PK787345','PK788345','PK789345','PK790345','PK791345','PK792345','PK793345','PK794345','PK795345','PK796345','PK797345','PK798345','PK799345','PK800345','PK801345','PK802345','PK803345','PK804345','PK805345','PK806345','PK807345','PK808345','PK809345','PK810345','PK811345','PK812345','PK813345','PK814345','PK815345','PK816345','PK817345','PK818345','PK819345','PK820345','PK821345','PK822345','PK823345','PK824345','PK825345','PK826345','PK827345','PK828345','PK829345','PK830345','PK831345','PK832345','PK833345','PK834345','PK835345','PK836345','PK837345','PK838345','PK839345','PK840345','PK841345','PK842345','PK843345','PK844345','PK845345','PK846345','PK847345','PK848345','PK849345','PK850345','PK851345','PK852345','PK853345','PK854345','PK855345','PK856345','PK857345','PK858345','PK859345','PK860345','PK861345','PK862345','PK863345','PK864345','PK865345','PK866345','PK867345','PK868345','PK869345','PK870345','PK871345','PK872345','PK873345','PK874345','PK875345','PK876345','PK877345','PK878345','PK879345','PK880345','PK881345','PK882345','PK883345','PK884345','PK885345','PK886345','PK887345','PK888345','PK889345','PK890345','PK891345','PK892345','PK893345','PK894345','PK895345','PK896345','PK897345','PK898345','PK899345','PK900345','PK901345','PK902345','PK903345','PK904345','PK905345','PK906345','PK907345','PK908345','PK909345','PK910345','PK911345','PK912345','PK913345','PK914345','PK915345','PK916345','PK917345','PK918345','PK919345','PK920345','PK921345','PK922345','PK923345','PK924345','PK925345','PK926345','PK927345','PK928345','PK929345','PK930345','PK931345','PK932345','PK933345','PK934345','PK935345','PK936345','PK937345','PK938345','PK939345','PK940345','PK941345','PK942345','PK943345','PK944345','PK945345','PK946345','PK947345','PK948345','PK949345','PK950345','PK951345','PK952345','PK953345','PK954345','PK955345','PK956345','PK957345','PK958345','PK959345','PK960345','PK961345','PK962345','PK963345','PK964345','PK965345','PK966345','PK967345','PK968345','PK969345','PK970345','PK971345','PK972345','PK973345','PK974345','PK975345','PK976345','PK977345','PK978345','PK979345','PK980345','PK981345','PK982345','PK983345','PK984345','PK985345','PK986345','PK987345','PK988345','PK989345','PK990345','PK991345','PK992345','PK993345','PK994345','PK995345','PK996345','PK997345','PK998345','PK999345')


-- EFCore OpenJson Query (10min 08sec)
DECLARE @__ids_0 nvarchar(max)
SET
    @__ids_0 = '["PK345","PK1345","PK2345","PK3345","PK4345","PK5345","PK6345","PK7345","PK8345","PK9345","PK10345","PK11345","PK12345","PK13345","PK14345","PK15345","PK16345","PK17345","PK18345","PK19345","PK20345","PK21345","PK22345","PK23345","PK24345","PK25345","PK26345","PK27345","PK28345","PK29345","PK30345","PK31345","PK32345","PK33345","PK34345","PK35345","PK36345","PK37345","PK38345","PK39345","PK40345","PK41345","PK42345","PK43345","PK44345","PK45345","PK46345","PK47345","PK48345","PK49345","PK50345","PK51345","PK52345","PK53345","PK54345","PK55345","PK56345","PK57345","PK58345","PK59345","PK60345","PK61345","PK62345","PK63345","PK64345","PK65345","PK66345","PK67345","PK68345","PK69345","PK70345","PK71345","PK72345","PK73345","PK74345","PK75345","PK76345","PK77345","PK78345","PK79345","PK80345","PK81345","PK82345","PK83345","PK84345","PK85345","PK86345","PK87345","PK88345","PK89345","PK90345","PK91345","PK92345","PK93345","PK94345","PK95345","PK96345","PK97345","PK98345","PK99345","PK100345","PK101345","PK102345","PK103345","PK104345","PK105345","PK106345","PK107345","PK108345","PK109345","PK110345","PK111345","PK112345","PK113345","PK114345","PK115345","PK116345","PK117345","PK118345","PK119345","PK120345","PK121345","PK122345","PK123345","PK124345","PK125345","PK126345","PK127345","PK128345","PK129345","PK130345","PK131345","PK132345","PK133345","PK134345","PK135345","PK136345","PK137345","PK138345","PK139345","PK140345","PK141345","PK142345","PK143345","PK144345","PK145345","PK146345","PK147345","PK148345","PK149345","PK150345","PK151345","PK152345","PK153345","PK154345","PK155345","PK156345","PK157345","PK158345","PK159345","PK160345","PK161345","PK162345","PK163345","PK164345","PK165345","PK166345","PK167345","PK168345","PK169345","PK170345","PK171345","PK172345","PK173345","PK174345","PK175345","PK176345","PK177345","PK178345","PK179345","PK180345","PK181345","PK182345","PK183345","PK184345","PK185345","PK186345","PK187345","PK188345","PK189345","PK190345","PK191345","PK192345","PK193345","PK194345","PK195345","PK196345","PK197345","PK198345","PK199345","PK200345","PK201345","PK202345","PK203345","PK204345","PK205345","PK206345","PK207345","PK208345","PK209345","PK210345","PK211345","PK212345","PK213345","PK214345","PK215345","PK216345","PK217345","PK218345","PK219345","PK220345","PK221345","PK222345","PK223345","PK224345","PK225345","PK226345","PK227345","PK228345","PK229345","PK230345","PK231345","PK232345","PK233345","PK234345","PK235345","PK236345","PK237345","PK238345","PK239345","PK240345","PK241345","PK242345","PK243345","PK244345","PK245345","PK246345","PK247345","PK248345","PK249345","PK250345","PK251345","PK252345","PK253345","PK254345","PK255345","PK256345","PK257345","PK258345","PK259345","PK260345","PK261345","PK262345","PK263345","PK264345","PK265345","PK266345","PK267345","PK268345","PK269345","PK270345","PK271345","PK272345","PK273345","PK274345","PK275345","PK276345","PK277345","PK278345","PK279345","PK280345","PK281345","PK282345","PK283345","PK284345","PK285345","PK286345","PK287345","PK288345","PK289345","PK290345","PK291345","PK292345","PK293345","PK294345","PK295345","PK296345","PK297345","PK298345","PK299345","PK300345","PK301345","PK302345","PK303345","PK304345","PK305345","PK306345","PK307345","PK308345","PK309345","PK310345","PK311345","PK312345","PK313345","PK314345","PK315345","PK316345","PK317345","PK318345","PK319345","PK320345","PK321345","PK322345","PK323345","PK324345","PK325345","PK326345","PK327345","PK328345","PK329345","PK330345","PK331345","PK332345","PK333345","PK334345","PK335345","PK336345","PK337345","PK338345","PK339345","PK340345","PK341345","PK342345","PK343345","PK344345","PK345345","PK346345","PK347345","PK348345","PK349345","PK350345","PK351345","PK352345","PK353345","PK354345","PK355345","PK356345","PK357345","PK358345","PK359345","PK360345","PK361345","PK362345","PK363345","PK364345","PK365345","PK366345","PK367345","PK368345","PK369345","PK370345","PK371345","PK372345","PK373345","PK374345","PK375345","PK376345","PK377345","PK378345","PK379345","PK380345","PK381345","PK382345","PK383345","PK384345","PK385345","PK386345","PK387345","PK388345","PK389345","PK390345","PK391345","PK392345","PK393345","PK394345","PK395345","PK396345","PK397345","PK398345","PK399345","PK400345","PK401345","PK402345","PK403345","PK404345","PK405345","PK406345","PK407345","PK408345","PK409345","PK410345","PK411345","PK412345","PK413345","PK414345","PK415345","PK416345","PK417345","PK418345","PK419345","PK420345","PK421345","PK422345","PK423345","PK424345","PK425345","PK426345","PK427345","PK428345","PK429345","PK430345","PK431345","PK432345","PK433345","PK434345","PK435345","PK436345","PK437345","PK438345","PK439345","PK440345","PK441345","PK442345","PK443345","PK444345","PK445345","PK446345","PK447345","PK448345","PK449345","PK450345","PK451345","PK452345","PK453345","PK454345","PK455345","PK456345","PK457345","PK458345","PK459345","PK460345","PK461345","PK462345","PK463345","PK464345","PK465345","PK466345","PK467345","PK468345","PK469345","PK470345","PK471345","PK472345","PK473345","PK474345","PK475345","PK476345","PK477345","PK478345","PK479345","PK480345","PK481345","PK482345","PK483345","PK484345","PK485345","PK486345","PK487345","PK488345","PK489345","PK490345","PK491345","PK492345","PK493345","PK494345","PK495345","PK496345","PK497345","PK498345","PK499345","PK500345","PK501345","PK502345","PK503345","PK504345","PK505345","PK506345","PK507345","PK508345","PK509345","PK510345","PK511345","PK512345","PK513345","PK514345","PK515345","PK516345","PK517345","PK518345","PK519345","PK520345","PK521345","PK522345","PK523345","PK524345","PK525345","PK526345","PK527345","PK528345","PK529345","PK530345","PK531345","PK532345","PK533345","PK534345","PK535345","PK536345","PK537345","PK538345","PK539345","PK540345","PK541345","PK542345","PK543345","PK544345","PK545345","PK546345","PK547345","PK548345","PK549345","PK550345","PK551345","PK552345","PK553345","PK554345","PK555345","PK556345","PK557345","PK558345","PK559345","PK560345","PK561345","PK562345","PK563345","PK564345","PK565345","PK566345","PK567345","PK568345","PK569345","PK570345","PK571345","PK572345","PK573345","PK574345","PK575345","PK576345","PK577345","PK578345","PK579345","PK580345","PK581345","PK582345","PK583345","PK584345","PK585345","PK586345","PK587345","PK588345","PK589345","PK590345","PK591345","PK592345","PK593345","PK594345","PK595345","PK596345","PK597345","PK598345","PK599345","PK600345","PK601345","PK602345","PK603345","PK604345","PK605345","PK606345","PK607345","PK608345","PK609345","PK610345","PK611345","PK612345","PK613345","PK614345","PK615345","PK616345","PK617345","PK618345","PK619345","PK620345","PK621345","PK622345","PK623345","PK624345","PK625345","PK626345","PK627345","PK628345","PK629345","PK630345","PK631345","PK632345","PK633345","PK634345","PK635345","PK636345","PK637345","PK638345","PK639345","PK640345","PK641345","PK642345","PK643345","PK644345","PK645345","PK646345","PK647345","PK648345","PK649345","PK650345","PK651345","PK652345","PK653345","PK654345","PK655345","PK656345","PK657345","PK658345","PK659345","PK660345","PK661345","PK662345","PK663345","PK664345","PK665345","PK666345","PK667345","PK668345","PK669345","PK670345","PK671345","PK672345","PK673345","PK674345","PK675345","PK676345","PK677345","PK678345","PK679345","PK680345","PK681345","PK682345","PK683345","PK684345","PK685345","PK686345","PK687345","PK688345","PK689345","PK690345","PK691345","PK692345","PK693345","PK694345","PK695345","PK696345","PK697345","PK698345","PK699345","PK700345","PK701345","PK702345","PK703345","PK704345","PK705345","PK706345","PK707345","PK708345","PK709345","PK710345","PK711345","PK712345","PK713345","PK714345","PK715345","PK716345","PK717345","PK718345","PK719345","PK720345","PK721345","PK722345","PK723345","PK724345","PK725345","PK726345","PK727345","PK728345","PK729345","PK730345","PK731345","PK732345","PK733345","PK734345","PK735345","PK736345","PK737345","PK738345","PK739345","PK740345","PK741345","PK742345","PK743345","PK744345","PK745345","PK746345","PK747345","PK748345","PK749345","PK750345","PK751345","PK752345","PK753345","PK754345","PK755345","PK756345","PK757345","PK758345","PK759345","PK760345","PK761345","PK762345","PK763345","PK764345","PK765345","PK766345","PK767345","PK768345","PK769345","PK770345","PK771345","PK772345","PK773345","PK774345","PK775345","PK776345","PK777345","PK778345","PK779345","PK780345","PK781345","PK782345","PK783345","PK784345","PK785345","PK786345","PK787345","PK788345","PK789345","PK790345","PK791345","PK792345","PK793345","PK794345","PK795345","PK796345","PK797345","PK798345","PK799345","PK800345","PK801345","PK802345","PK803345","PK804345","PK805345","PK806345","PK807345","PK808345","PK809345","PK810345","PK811345","PK812345","PK813345","PK814345","PK815345","PK816345","PK817345","PK818345","PK819345","PK820345","PK821345","PK822345","PK823345","PK824345","PK825345","PK826345","PK827345","PK828345","PK829345","PK830345","PK831345","PK832345","PK833345","PK834345","PK835345","PK836345","PK837345","PK838345","PK839345","PK840345","PK841345","PK842345","PK843345","PK844345","PK845345","PK846345","PK847345","PK848345","PK849345","PK850345","PK851345","PK852345","PK853345","PK854345","PK855345","PK856345","PK857345","PK858345","PK859345","PK860345","PK861345","PK862345","PK863345","PK864345","PK865345","PK866345","PK867345","PK868345","PK869345","PK870345","PK871345","PK872345","PK873345","PK874345","PK875345","PK876345","PK877345","PK878345","PK879345","PK880345","PK881345","PK882345","PK883345","PK884345","PK885345","PK886345","PK887345","PK888345","PK889345","PK890345","PK891345","PK892345","PK893345","PK894345","PK895345","PK896345","PK897345","PK898345","PK899345","PK900345","PK901345","PK902345","PK903345","PK904345","PK905345","PK906345","PK907345","PK908345","PK909345","PK910345","PK911345","PK912345","PK913345","PK914345","PK915345","PK916345","PK917345","PK918345","PK919345","PK920345","PK921345","PK922345","PK923345","PK924345","PK925345","PK926345","PK927345","PK928345","PK929345","PK930345","PK931345","PK932345","PK933345","PK934345","PK935345","PK936345","PK937345","PK938345","PK939345","PK940345","PK941345","PK942345","PK943345","PK944345","PK945345","PK946345","PK947345","PK948345","PK949345","PK950345","PK951345","PK952345","PK953345","PK954345","PK955345","PK956345","PK957345","PK958345","PK959345","PK960345","PK961345","PK962345","PK963345","PK964345","PK965345","PK966345","PK967345","PK968345","PK969345","PK970345","PK971345","PK972345","PK973345","PK974345","PK975345","PK976345","PK977345","PK978345","PK979345","PK980345","PK981345","PK982345","PK983345","PK984345","PK985345","PK986345","PK987345","PK988345","PK989345","PK990345","PK991345","PK992345","PK993345","PK994345","PK995345","PK996345","PK997345","PK998345","PK999345"]'

SELECT [Id] FROM TestTbl as p WHERE
EXISTS (SELECT 1 FROM OPENJSON(@__ids_0) WITH ([value] nvarchar(450) '$') AS [a]
        WHERE [a].[value] = [p].[Id] OR ([a].[value] IS NULL AND [p].[Id] IS NULL)
)


-- Drop Table
DROP TABLE [TestTbl]

jasenf · 2024-01-15T13:43:00Z

Agree, and thanks for the great work! Optimally this would be considered a SQL server issue, as it is something that is supported but so non-performant, it seems like it should be remedied back there. But that's probably not going to happen anytime soon 😏

DonKonfetti · 2024-01-15T13:44:26Z

I feel like people are missing the simple temporary fix, offered by @stevendarby just add this, you don't seem to lose any other .net8 or efcore 8 capabilities:

Well, I've got the feeling that this is kind of hacky. As it fixes that particular issue for the moment, it may well be that future EF releases will use the configured compatibility level for other purposes (maybe other optimizations).

roji · 2024-01-15T13:45:30Z

Well, I've got the feeling that this is kind of hacky. As it fixes that particular issue for the moment, it may well be that future EF releases will use the configured compatibility level for other purposes (maybe other optimizations).

That's true, but if you're being blocked by the new OPENJSON translations in some way, then it's recommended to do this so that you can switch to 8.0, at least as a temporary measure.

FYI we're having some internal discussions/investigations around this; I'll post a summary of the situation and the possible mitigations etc.

DonKonfetti · 2024-01-15T13:57:27Z

@roji Great to hear you're doing further investigations on this topic! Very much appreciated.

We actually use this fix to work around the performance degradation and it works well. :) Just wanted to point out that this might come back and bite us later.

asos-martinsmith · 2024-01-15T14:11:35Z

Agree, and thanks for the great work! Optimally this would be considered a SQL server issue, as it is something that is supported but so non-performant, it seems like it should be remedied back there. But that's probably not going to happen anytime soon 😏

There are going to be a variety of different cases where SQL Server will produce a better execution plan when it can see the exact values you are filtering on and compile a plan based on those (the old behaviour that people have become accustomed to).

With the new approach it is going to compile a single execution plan once and reuse it regardless of what you pass in. This will be an improvement for some people but not others.

Unless it was going to inspect the values being passed in as a string and recompile the execution plan each time it will never behave the same. (and doing so would likely cause performance regressions for a different group of people for whom the current OPENJSON approach is working fine).

Complexities like this are why I prefer hand writing SQL than using ORM written SQL TBH. SQL Server recognizes that a "one size fits all" approach doesn't always work - the availability of query hints is an implicit acknowledgement of this,

If there was to be a change in SQL Server I'd like to see it support the array type as Postgres does - and tilt the balance back towards "less opaque" and allowing sniffing of array values - and more possibilities of cardinality based recompiles (or even different concurrently cached plans for different array cardinalities)

srdex · 2024-01-15T14:13:15Z

@roji I will try to find some time to create something that you can reproduce yourself. Meanwhile, here are the execution plans:
Old:

New:

srdex · 2024-01-15T14:48:15Z

@roji Here, you can find zip file attached with the DB with just one table with only necessary columns in it (data included). Both queries are included in txt files.
openjson_issue_db.zip

And here is the script for creating indexes since they are not included in the test DB - even though they don't make difference:
create_indexes.txt

roji · 2024-01-15T14:58:13Z

Thank you @srdex - that will indeed help!

@asos-martinsmith thanks for you input - and thanks in general for your analyses and everything you've posted on this issue, it has been very helpful.

If there was to be a change in SQL Server I'd like to see it support the array type as Postgres does - and tilt the balance back towards "less opaque" and allowing sniffing of array values - and more possibilities of cardinality based recompiles (or even different concurrently cached plans for different array cardinalities)

This is something we've been discussing with the SQL Server people, and there's definitely a desire to make this better - specifically possibly sniff into OPENJSON's array argument to get an actual row estimate that would feed into the planning process. That technically doesn't require an array type like PostgreSQL, though that obviously makes it more 1st-class and less of a specific planning tweak around OPENJSON etc. But of course, all that is for future versions of SQL Server.

With the new approach it is going to compile a single execution plan once and reuse it regardless of what you pass in. This will be an improvement for some people but not others.

I agree that there's unfortunately no perfect/one-size-fits-all solution here. Out of curiosity, given all of the above, what would be your recommendation here, given that EF does need to have a single translation for Contains (at least as a default)?

Complexities like this are why I prefer hand writing SQL than using ORM written SQL TBH. SQL Server recognizes that a "one size fits all" approach doesn't always work - the availability of query hints is an implicit acknowledgement of this,

I don't disagree, although the Contains translation seems to be a relatively rare case where this is especially problematic; in other words, in most other cases there seems to be a reasonable LINQ->SQL translation that performs well in most scenarios. We do have some cases where there are two correct translations, each performing better in different scenarios, but OTOH not that many. It's also perfectly fine for users to drop down to SQL for these particularly problematic queries (EF has ample APIs/machinery to help with that as well).

FWIW PostgreSQL by-design does not support query hints, although it obviously can suffer from the same mis-planning issues (IIRC SQLite/MySQL are similar, though I may be wrong there). SQL Server seems somewhat special in the DB world in allowing such planning tweaks.

ErikEJ · 2024-01-15T15:50:53Z

Out of curiosity, given all of the above, what would be your recommendation here, given that EF does need to have a single translation for Contains (at least as a default)?

This has turned out quite interesting. Given the experience of this change, I think that usage of OPENJSON should be an opt-in, and the default behavior should be the "classic" behaviour. I know that this will be difficult to handle in terms on versioning.

asos-martinsmith · 2024-01-15T16:03:20Z

@roji Here, you can find zip file attached with the DB with just one table with only necessary columns in it (data included). Both queries are included in txt files. openjson_issue_db.zip

And here is the script for creating indexes since they are not included in the test DB - even though they don't make difference: create_indexes.txt

It may well be possible to restructure this query to improve things somewhat but as it currently stands...

There are 534,461 rows in the table,
533,849 get past the initial filtering on [ORD_DeletedOn] IS NULL AND ORD_Discriminator IN (N'Cso', N'Hcc', N'Qcc')
Each of these is tested against the IN conditions in turn.
This is a semi join so it can stop early if it finds a match but only eight of these rows match any of the six OPENJSON conditions at all so in the vast majority of cases it can't short circuit and needs to evaluate all of them.
The remaining 533,841 rows need to be tested against all of the OPENJSON conditions.
The splitting of the string to the JSON array is not just done once and the result shared between each of the OPENJSON calls. Each OPENJSON instance evaluates it independently. Worse it re-evaluates for every outer row. So the JSON is re-parsed ~3.2 million times across the 6 operators.
There are 15 elements in the array 533,841 * 15 = 8,007,615 and 533,849*15 = 8,007,735 so this is the reason the row counts from these operators are in this range.

Materialising the OPENJSON call once into a #temp table and referencing that improves things somewhat but this is still massively less performant than just the single filter operator with all the conditions baked in as in the "query_without_openjson" case.

OR logic is often problematic but for "fun" I could get reasonable performance with quite an extensive rewrite - this gives me a plan with two outer (hash) joins - that only evaluate the JSON once each - rather than a nested loops that re-evaluates it many times.


SELECT [o].[ORD_ID]
FROM   [test].[dbo].[Orders] AS [o]
WHERE  [o].[ORD_DeletedOn] IS NULL
       AND [o].[ORD_Discriminator] IN ( N'Cso', N'Qcc', N'Hcc' )
       AND CASE [o].[ORD_Discriminator]
             WHEN N'Cso' THEN [o].[CSO_CSO_ID]
             WHEN N'Qcc' THEN [o].[QCC_CSO_ID]
             WHEN N'Hcc' THEN [o].[HCC_CSO_ID]
           END IN (SELECT [o1].[value]
                   FROM   OPENJSON('[615270,614283,614282,614280,614275,614274,614273,614272,614271,614270,613271,612273,612270,612266,611266]') 
                          WITH ([value] INT '$') AS [o1])
        OR CASE [o].[ORD_Discriminator]
             WHEN N'Cso' THEN [o].[CSO_QCC_ID]
             WHEN N'Qcc' THEN [o].[QCC_QCC_ID]
             WHEN N'Hcc' THEN [o].[HCC_HCC_ID]
           END IN (SELECT [o2].[value]
                   FROM   OPENJSON('[615270,614283,614282,614280,614275,614274,614273,614272,614271,614270,613271,612273,612270,612266,611266]')
                          WITH ([value] INT '$')AS [o2])

dandros · 2024-01-15T18:02:24Z

@roji I can't share my database schema.
In my case performance pbm was caused by Table Spool (Lazy Spool) execution step.

I don't understand why this step only happened when query is executed with "exec sp_executesql" and using OPENJSON.
Once I have added covering indexes that SSMS execution plan suggested, Lazy Spool step has gone. The query is running instantly with "exec sp_executesql" and using OPENJSON syntax.

I also checked execution plan of the same query without indexes and without OPENJSON syntax, but using the regular IN with static values. Table Spool (Lazy Spool) execution step was not there. That's why it was running smoothly before EF8

IanKemp · 2024-01-17T11:52:39Z

Out of curiosity, given all of the above, what would be your recommendation here, given that EF does need to have a single translation for Contains (at least as a default)?

This has turned out quite interesting. Given the experience of this change, I think that usage of OPENJSON should be an opt-in, and the default behavior should be the "classic" behaviour. I know that this will be difficult to handle in terms on versioning.

I strongly disagree; the reason the change to use openjson was made is precisely to overcome the problem of the query processor going haywire with massive in statements, which seems to me to be a larger problem (see e.g. projects like this one) than openjson sometimes performing badly. We already have a way to opt out of openjson via the UseCompatibilityLevel(120) trick, which I agree is not optimal but at least is there. And finally, a whole bunch of people posting here have already discovered that doing the very basics of profiling their queries and adding missing indexes, has entirely resolved their issue with openjson. My suspicion is that in most cases, the in query was executing fine due to an old query plan that just happened to work, and the EFC 8 implicit change to openjson has simply forced a new query plan that's exposed a lack of basic DB maintenance.

I would definitely like to see some sort of explicit configuration switch on SqlServerDbContextOptionsBuilder, like UseOpenJsonForContains which defaults to true and can be disabled to revert to the in behaviour. Ideally this would be on a per-query level but that would be a significant API surface change that I'm not sure the team would be interested in making. If there is a need for it at the per-query level then packages like the one I linked to can always be used.

DonKonfetti · 2024-01-17T13:20:49Z

@IanKemp Your UseOpenJsonForContains proposal (which defaults to true) sounds like exactly that kind of configuration I'd like to have. So one could globally disable the OpenJSON usage for IN clauses.

Note that 8.0.2 will bring a per-query solution with EF.Constant (see #31552).

roji · 2024-01-17T15:36:05Z

@IanKemp I'm not sure you're right there; the fundamental problem is that SQL Server doesn't do proper row estimates for OPENJSON (i.e. by sniffing the parameter), leaving it to build query plan with severely reduced information; that ultimately seems to be the source of most problems here. Of course, the previous method also had its problems (differing SQLs leading to query plan pollution), but that's a very different kind of problem (@asos-martinsmith would be good to have your input on this question as well).

I unfortunately haven't yet had the time to fully dive/test the above scenarios - that'll happen at some point - so I don't have a clear opinion on things yet; that's also why these discussions are very useful - thanks for engaging.

We've definitely discussed something like UseOpenJsonForContains (or possibly something more general that would affect the Contains translation for other databases as well, as similar issues can probably occur there). Note that EF.Constant is indeed a per-query way of switching to the old syntax, but it has quite a performance impact on the EF side; since constants are integrated into the incoming LINQ query tree, every execution needs to go through full query compilation/processing as if it were a totally new query. That's much slower than the UseCompatibilityLevel() solution (and incidentally, causes cache pollution in EF's internal query cache, conceptually similar to how the SQL constants cause cache pollution in SQL Server).

In any case, we generally try to avoid behavioral knobs/tweaks such as this UseOpenJsonForContains, unless they're absolutely necessary - though that may end up being the case here.

vgallegob · 2024-01-17T16:15:01Z

I think leaving a flag like UseOpenJsonForContains up for the developer to set is risky, I wouldn't want it. It's confusing...
I think it might be easyer to set it on a per query basis.

jasenf · 2024-01-17T16:22:34Z

I appreciate the teams effort to minimize these kind of configuration options. It's made Entity Framework great to work with.

I do think it will be required for this one, and prefer a global one time configuration option. We haven't seen a single query where this was performant in our case and I don't want our developers to have to remember each time.

Just my 2 cents.

JamesHill3 · 2024-01-25T16:25:16Z

I appreciate the teams effort to minimize these kind of configuration options. It's made Entity Framework great to work with.

I do think it will be required for this one, and prefer a global one time configuration option. We haven't seen a single query where this was performant in our case and I don't want our developers to have to remember each time.

Just my 2 cents.

Totally agree with this sentiment. We have hundreds of queries in the application that this most affected. It's not reasonable to have a developer refactor hundreds of EF queries to include EF.Constant when a global flag would give us back queries that we know perform as expected. Like you, we have not found a single instance where the OPENJSON change benefited us.

roji · 2024-01-25T16:46:05Z

@JamesHill3 @jasenf note that the switch to OPENJSON for Contains was specifically to mitigate #13617, which was a very highly-voted issue before EF 8.0. In a nutshell, the previous translation simply embedded the values as constants in a SQL IN expression, causing different values to generate different SQLs, and defeating the SQL Server query cache. This notably has the effect of causing other, unrelated queries to get ejected from the cache, due to the pollution caused by all the values.

In other words, the point of switching to OPENJSON for Contains was not to make queries with Contains run faster; in fact, we knew when introducing the change that there will most probably be a small regression for those queries. The point was to prevent making every other query run slower, due to repeated planning caused by the cache pollution. This is very easy to overlook if you're simply focussing on the performance of the individual Contains query.

This is also why this is less clear cut than it is made out to be in your comments - there are advantages and disadvantages to each approach, and there's unfortunately not one approach that's superior in all cases. Having said that, the performance regressions analyzed above for Contains go beyond what we saw in the query shapes we explored, and I agree there's good reason to at least provide a way to not use OPENJSON because of that.

jasenf · 2024-01-25T16:50:53Z

Understood. We had in fact tried third party extensions that did the same thing to try and fix the query plan issues, so I get it. However at the end of the day, for us, performant queries was more important than pre compiled and cached query plans.

JamesHill3 · 2024-01-25T17:09:14Z

@roji I appreciate the additional information and context. I get that it's more complex than my initial comment made it.

I confess, however, that the fix still feels simple to me (while at the same time recognizing the armchair athlete nature of that statement).

The application most affected by this change for us has been live for 2 years, after 4 years of development and 150,000+ lines of code. Devs and users alike were quite happy with the performance. The application did exactly what we expected of it in a timely fashion - cache pollution or not. We could no longer say that after the OPENJSON change (until we changed compatibility level).

IanKemp · 2024-01-25T17:25:25Z

However at the end of the day, for us, performant queries was more important than pre compiled and cached query plans.

This statement is bizarre to me... the query plan cache is the thing that allows queries to be fast by virtue of the fact that the optimiser doesn't have to start from scratch with each one.

stevendarby · 2024-01-25T17:28:54Z

However at the end of the day, for us, performant queries was more important than pre compiled and cached query plans.

This statement is bizarre to me... the query plan cache is the thing that allows queries to be fast by virtue of the fact that the optimiser doesn't have to start from scratch with each one.

The time to compile the query plan can be a fraction of the total execution time if it's an expensive query

MelGrubb · 2024-02-02T15:48:37Z

I have a rather large EF "Contains" query that has absolutely tanked the performance as soon as we moved to v8. Rolling back isn't an option because reasons. This isn't a one-off query hidden in a dark corner somewhere. This is a highly-used query, and the OPENJSON performance has been an absolute killer for us. I'm implementing the compatibility level fix now, but telling the whole database to go back to 2014 seems like we'd be missing out on a lot of other optimizations. We'll see how it goes, but opting out of the new behavior on a per-query basis is going to be pretty important to my team.

ErikEJ · 2024-02-02T15:54:39Z

@MelGrubb You are not "telling the database" anything but adjusting EF Core behaviour for only OPENJSON

MelGrubb · 2024-02-02T15:56:38Z

If the compatibility level is set at the context-level, then it seems to me that it would affect every single advancement that distinguished 120 from 130, 140, and 150. I'm not just picking and choosing for this one query, I'm setting a general level for everything, right?

Okay... replace "database" with "context", and I think it's clearer. But EF is asking the database to act as if it was still 2014.

roji · 2024-02-02T17:39:43Z

@MelGrubb no - with UseCompatibilityLevel() (breaking change note), EF isn't asking the database anything. You're just telling EF to issue queries as if it's talking to an old database. In practice, the only thing this currently affects is the Contains translation - so it should be a reasonable workaround.

Other than that we're definitely thinking about this and intend to do something better for 9.0.

MelGrubb · 2024-02-02T17:54:28Z

I get the distinction, and if you're saying that the contains behavior is the only thing that EF would phrase differently between compatibility level 120 and 150, then that's great. It just seemed to me like there was surely something else in newer SQL versions that EF was taking advantage of. I'll keep my eye out for the new version and use whatever the new, official fix is.

ErikEJ · 2024-02-02T17:58:24Z

@MelGrubb this is actually the first time EF Core uses a SQL Servet feature that is not available in all supported versions.

Suchiman added the customer-reported label Nov 22, 2023

This was referenced Nov 24, 2023

EF8 - Regression on Contains query when column collation is not default DB collation #32147

Open

Implement EF.Constant #32412

Merged

Montago mentioned this issue Dec 6, 2023

EF should probe compatibility_level (for MS SQL) and translate appropriately #32528

Closed

jaxidian mentioned this issue Jan 17, 2024

In what scenarios will the new .Contains functionality using OPENJSON change the data being returned by a query? #32852

Closed

ajcvickers assigned roji Jan 31, 2024

Contains to OPENJSON translation regresses performance #32394

Contains to OPENJSON translation regresses performance #32394

Comments

Suchiman commented Nov 22, 2023 • edited

Include provider and version information

roji commented Nov 23, 2023

stevendarby commented Nov 23, 2023

Suchiman commented Nov 23, 2023

ErikEJ commented Nov 23, 2023

Suchiman commented Nov 23, 2023

roji commented Nov 24, 2023

ErikEJ commented Nov 24, 2023

stevendarby commented Nov 24, 2023

roji commented Nov 24, 2023

Suchiman commented Nov 27, 2023 • edited

roji commented Nov 27, 2023

roji commented Dec 4, 2023

stevendarby commented Dec 4, 2023

roji commented Dec 5, 2023 • edited

Nefarion commented Dec 5, 2023 • edited

RyanONeill1970 commented Dec 5, 2023 • edited

roji commented Dec 5, 2023

RyanONeill1970 commented Dec 5, 2023

Nefarion commented Dec 6, 2023 • edited

molesinski commented Dec 6, 2023

nh43de commented Dec 6, 2023 • edited

nh43de commented Dec 6, 2023 • edited

roji commented Dec 7, 2023

Nefarion commented Dec 7, 2023 • edited

jasenf commented Jan 15, 2024

DonKonfetti commented Jan 15, 2024

roji commented Jan 15, 2024

DonKonfetti commented Jan 15, 2024

asos-martinsmith commented Jan 15, 2024 • edited

srdex commented Jan 15, 2024

srdex commented Jan 15, 2024 • edited

roji commented Jan 15, 2024

ErikEJ commented Jan 15, 2024

asos-martinsmith commented Jan 15, 2024 • edited

dandros commented Jan 15, 2024

IanKemp commented Jan 17, 2024

DonKonfetti commented Jan 17, 2024

roji commented Jan 17, 2024

vgallegob commented Jan 17, 2024

jasenf commented Jan 17, 2024

JamesHill3 commented Jan 25, 2024

roji commented Jan 25, 2024

jasenf commented Jan 25, 2024

JamesHill3 commented Jan 25, 2024

IanKemp commented Jan 25, 2024

stevendarby commented Jan 25, 2024

MelGrubb commented Feb 2, 2024

ErikEJ commented Feb 2, 2024

MelGrubb commented Feb 2, 2024 • edited

roji commented Feb 2, 2024

MelGrubb commented Feb 2, 2024

ErikEJ commented Feb 2, 2024

Suchiman commented Nov 22, 2023 •

edited

Suchiman commented Nov 27, 2023 •

edited

roji commented Dec 5, 2023 •

edited

Nefarion commented Dec 5, 2023 •

edited

RyanONeill1970 commented Dec 5, 2023 •

edited

Nefarion commented Dec 6, 2023 •

edited

nh43de commented Dec 6, 2023 •

edited

nh43de commented Dec 6, 2023 •

edited

Nefarion commented Dec 7, 2023 •

edited

asos-martinsmith commented Jan 15, 2024 •

edited

srdex commented Jan 15, 2024 •

edited

asos-martinsmith commented Jan 15, 2024 •

edited

MelGrubb commented Feb 2, 2024 •

edited