WIP - Add support for IAsyncEnumerable

[maca88]

Limitations: The added Linq extension method AsAsyncEnumerable do not support PostExecuteTransformer as the transformer operates with IEnumerable<>.

Fixes: #2267

Added WIP as the async generator has some issues generating obsolete overloads and tests with IAsyncEnumerable.

[hazzik]

Not sure how this will behave....

[bahusoid]

What are you talking about? Is it about using (SuspendAutoFlush())? Yeah this code seems suspicious - if this suspend is really required ( as this code looks broken to me even for sync case - it's surely not scoped for items iterations) it needs to be done inside iterator.

[hazzik]

Yes, I'm talking about that.

[maca88]

As now PerformAsyncIterate and PerformIterate do not execute any query, the auto flush logic was moved into the initialization of the enumerator, so that the auto flush logic will be triggered before the query is executed.
I checked how hibernate does that and they are also suspending only for the first query that is executed. I didn't found any scenario that would need to perform this logic inside the iteration.

[hazzik]

Looks good so far.

[bahusoid]

What are you talking about? Is it about using (SuspendAutoFlush())? Yeah this code seems suspicious - if this suspend is really required ( as this code looks broken to me even for sync case - it's surely not scoped for items iterations) it needs to be done inside iterator.

[bahusoid]

Does it really make sense to do it for each processed item? Maybe it should be done once for iterations (so this logic should be moved to GetAsyncEnumerables or even to JoinedAsyncEnumerable)

[maca88]

Does it really make sense to do it for each processed item?

The user may change the DefaultReadOnly value while iterating, which may cause a readonly query to load entites as writable, if DefaultReadOnly is not set for each item.

[hazzik]

So, with the latest iteration of changes this has become a close resemblance of "Future" feature, right?

[maca88]

The last iteration of changes applied the same logic in terms of when the query is executed to the current Enumerable methods, which is similar how "Future" feature work, right.

[maca88]

Rebased and re-generated with AsyncGenerator 0.18.1

[maca88]

Only now I realized that PerformIterate and now PerformAsyncIterate trigger N+1 queries. For example the following query:

s.CreateQuery("from Simple").Enumerable<Simple>()

will trigger one query that will retrieve all ids and then on each MoveNext call a separate query will be triggered for the current id, if the entity is not in the cache (hibenate does the same). Hibernate has also scroll method that does what I thought Enumerable does, which executes only one query. For me, the behavior of scroll method is more suitable for Enumerable and AsyncEnumerable but it would require to port the feature from hibernate, which is a topic for a separate PR.

[maca88]

Rebased.

[bahusoid]

Only now I realized that PerformIterate and now PerformAsyncIterate trigger N+1 queries.

That sucks :( It's not very useful for entities but I guess it's still can be useful for scalar values...

For me, the behavior of scroll method is more suitable for Enumerable and AsyncEnumerable but it would require to port the feature from hibernate, which is a topic for a separate PR.

Yeah. And I would just replace Enumerable implementation with scroll. Current enumerable entity handling is not intuitive at all.

[maca88]

Yeah. And I would just replace Enumerable implementation with scroll

One issue that I didn't checked, is how scroll behave when the query has collection fetches. Hibernate uses FetchingScrollableResultsImpl, but I didn't tested it yet to understand how it works.

[fredericDelaporte]

So, with the latest iteration of changes this has become a close resemblance of "Future" feature, right?

The last iteration of changes applied the same logic in terms of when the query is executed to the current Enumerable methods, which is similar how "Future" feature work, right.

The main purpose of the Future feature is not deferred execution but batching together queries. (Future also features a very loose coupling between code parts defining the queries, contrary to the recent query batcher which requires referring to the same batcher reference for adding queries into it, but as such also helps knowing exactly what is batched together.)
Deferred execution is just a side effect of how Future works.

This IAsyncEnumerable feature does nothing to batch queries together, so it is still two quite different features.

Anyway, it appears this "enumerable" feature is not very attractive with its current performances issues.

Is it the reason for this PR to be stalled? Or should its conflicts be resolved for allowing merging it?

[maca88]

Is it the reason for this PR to be stalled?

Yes, the scroll feature from Hibernate should be ported first, so that it can be used in this PR to avoid N+1 issue. I haven't started working on it yet as my current priority is the Include method. Changed to WIP until the scroll feature is ported.

[zlsmith86]

Is there an issue to track the porting of the scroll feature?

[Harpush]

Any progress here with the scroll feature?