Query: Improve query test matrix

[divega]

Historically we have a large number of regressions in our LINQ implementation on every release. A significant number of regressions are introduced when we improve the stack to recognize more expressions that they can be push down to the server, but some of them are not translated correctly.

There is an ongoing brainstorming on how to improve this, and the resulting plan will be part of the focus in 2.2.

[AndriySvyryd]

LINQ features

Query 'root'

• DbSet/Keyless DbSet
  • Same/different context
• Navigation
• FromSQL
  • Composable/not composable
• Mapped function

Where

• Special consideration: null semantics
• Boolean columns (as opposed to search conditions)
• Equality
  • Scalar
  • Structural
  • Composite
  • Entity (navigation, QSRE, PK to FK)
• Expressions (method calls, etc.)

OrderBy

• Complex (not on a property)
• Multiple (ThenBy)
• Duplicate (same property twice)
• Repeated (first one ignored)

Select

• Single element X
• Collection of X
• Where X in
  • Anonymous
  • DTO
  • Tuple/ValueTuple
  • Entity/Keyless type
  • Scalar
  • And possibly recursively...

SelectMany

• Anonymous
• DTO
• Tuple/ValueTuple
• Entity/Keyless type
• Scalar
• With DefaultIfEmpty
• With explicit result selector
• And possibly recursively...

Join/GroupJoin

• Special consideration: combine with projection
• Property
• Entity
• Navigation (left/right key selector)
• LOJ pattern (GroupJoin, SelectMany, DefaultIfEmpty)

GroupBy

• Naked
• With aggregate (scalar/expression/sub-query/VB)
• With/without result selector
• Different group keys
  • Constant
  • Scalar
  • Entity
  • Composite
• With/without element selector

Result operators

• Count/LongCount
• Min/Max
• Sum/Average
• Skip/Take
• Distinct
• First/FirstOrDefault/Single/SingleOrDefault
• Last*
• OfType/Cast (inheritance)
• All/Any
• DefaultIfEmpty
• Set (Union, etc.)
• SkipWhile/TakeWhile
• Contains
• Reverse

Expressions

• Contains
• DateAdd
• DateDiff
• StartsWith/EndsWith
• Like
• Math.*
• ToArray/ToList/etc.
• AsEnumerable/AsQueryable
• String manipulation
  • Concatenation
  • IndexOf
  • IsNullOrEmpty/WhiteSpace
  • Length
  • Replace
  • Substring
  • ToLower/ToUpper
  • Trim/TrimEnd/TrimStart
  • Contains/StartsWith/EndsWith
  • String comparison modifiers

Other operations

• Coalesce
• Ternary conditional
• Type conversion
  • Implicit
  • Explicit cast
  • as
  • is
  • VB converts

‘let’ keyword

• Funcletization
  • Different complexity (literal/expression/recursive/skip/take)
  • Functions always sent to the server Guid.NewID
• Closure variables
  • Scalar
  • Entity
  • Collection (in case of Contains)

Navigation expressions

• Reference
• Collection
• Skip collection
• Multi level
• PK -> FK optimization (order.Customer.Id -> order.CustomerId)
• Special consideration:
• Owned
• Inside Join key selector
• Inside SelectMany
• Optional/required

Include

• Reference/Collection/Skip collection
• Split/Single
• Nested
• On a derived type
• String
• Filtered

Sync/Async

AsTracking/AsNoTracking

Scenarios

Correlated Collection

• Variation of all basic query concepts: Where, OrderBy, Select
• Nested,
• Collection returning only one scalar (not currently handled well)

Complex binding

• Reference element from outside of a subquery

Null semantics

• Simple
• Negated
• Complex (coalesce, conditional)

Streaming/buffering

SelectExpression

• Type inference
• RowNumberPaging

Client evalution

EF.Property

MARS

Model features

Entity Types

• Simple
• Derived
• Owned
  • Weak
  • Derived
• Shared type
• Keyless

Functions

• Built-in
• Provider specific

Properties

• Optional/Required
• Type Conversion
  • Simple (int -> string)
  • Structural (byte[] -> long, JSON -> string)
• Shadow/non-shadow

Keys

• string/GUID/int
• byte[]
  • Composite
  • Alternate
• int[] and other types with value converters
• Composite over discriminator

Foreign keys

• Optional/Required
• Unique/Nonunique
• Self-referencing
• 0/1/2 navigations

Mapping

• TPH/Complete TPH/TPT/TPC/Entity splitting
• Table sharing
• Shared columns
• View/Defining Query/TVF

Global filters

For the entity class features see #12014

[divega]

Just making sure these are covered explicitly:

• equality in general, e.g. scalar vs. structural vs. composite vs. entity (this is somewhat covered in groupby and join keys , but it applies to other parts of the query)
• model-defined (global) filters
• defining query
• query types
• usage of DTOs, anonymous, Tuples inside the query (although I assume these are implicitly covered as just a combination of projection plus some other operator applied afterwards)

Do we need to call out specific LINQ query operators in this list or should we assume that all query operators in Queryable and Enumerable are fair game?

[divega]

Also, I wonder if type mapping and value conversions is something we should add to the mix here (or in #12014) or keep as separate. I believe there are types that are particularly interesting, like binary.

[bricelam]

The Visual Basic Aggregate Clause can produce some interesting expressions that may be worth adding coverage for.

[NetTecture]

As a user that sis bitten by 2.1.1 basically blowing every non trivial query we have in ef6 - can you please put a focus on a struture/frameowkr that allows end users to submit test cases? And then actually put a focus on fixing those, not just ignoring them for a grand larger plan?

We moved from ef6 to 2.1.1 a month ago, and I am stuck with half my queries literally blowing in my face with all kinds of issues, from bad sql generated to parsing exceptions. I have test cases for most thigns that I could put up as repro, but that is way too much ramp up and no documentation how to do that easily.

As long as you focus on some individual isolated expressions there is always the risk of overlooking complex interactions - and those are coming from users.

I applause the attempt to get LINQ into a less error prone scenario, but this also must include an easy ramp for end user cases that then get fixed - and the fix published, preferably not in the "next marjor release which, btw., you can use possibly in 4 months because hey, we don't really want you to get your work into your production".

If this sounds frustrated - it is. Stuck on 2.1.1 with basics not working and no release date. Heck, today i found one (#12951) but I lack the documentaiton and guideliens how to put up a test case for it. Stuff like that can make end users provide really complex text cases. Oh, yeah - you likely try aout a lot of permutations. My cases possibly involve 8-10 navigations and each of it hsa up to 2 global filters that possibly add another 4-5 table joins. This is where you sometimes find really interesting things.

[ajcvickers]

Moving to the 3.0 milestone for tracking; it's not really punted, just ongoing work.

[maumar]

Initial work and infrastructure added in cb27031

Verification will be added in the future - mutators can be in 2 modes: generating query and generating baseline. When ran with the same seed the resulting query should be the same, unless the baseline query was different in the first place (need to compensate for Maybe but appart from that, most baselines are identical.

We also need to create a better comparer for untyped results, The one we currently have simply calls to underlying Equals method which won't work for collections and other more complicated structures.

[roji]

Removing from milestone to discuss the relevance after we've disabled client eval.