Run the selector on Skipped items.

[VSadov]

It seems that while we can do certain optimization based on the known size of the resulting sequence after Skip is applied, but not running selectors on skipped items during enumeration is observable and could be unexpected.

The following queries seem to have subtly different meaning -
list.Select(s).Skip(10)
list.Skip(10).Select(s)

[karelz]

Out of curiosity, what is the difference between the 2?

BTW: Given this is Design Discussion, I will remove the "up for grabs"

[VSadov]

The fix is likely to be applied in ListPartition where it would need to run the selector for the skipped elements before yielding the first result (similarly to the EnumerablePartition)

[VSadov]

@karelz - the list.Select(s).Skip(10) seems to promise to run s as many times as elements in the list.
It would, if list was some random enumerable.

Does the fact that we have a "seek-able" source allow us to optimize away running selectors on the elements that we skip?

[VSadov]

Related to: https://github.com/dotnet/corefx/issues/13910

[karelz]

The only difference wold be if Select causes observable side-effects, right? Otherwise there should not be a concern, right?
Do we even guarantee that we will enumerate list just once? I thought that is entirely up to Linq to decide and that it can change -- which kind of also rules out side-effects.

Why is there any difference between seekable and non-seekable enumerable? In theory, we should be able to make the optimization for either both cases, or none.

(my apologies if my questions are too newbie for the area - feel free to call me out on that ;-))

[karelz]

OK, I read more in #19455 and now I see some of your points. I didn't realize that throwing (as part of Validate() method) is potentially reasonable side-effect (which can be executed repeatedly).

Maybe doing less magic on behalf of developer is not that bad ... if developers care about perf, they should be able to optimize their code and not expect auto-magic from a compiler like C# to pay for their mistakes/laziness in thinking things through.

[VSadov]

On the other hand 'Take(0)' would short-cirquit everything...

And what happens with 'ElementAt(10)'?

Also, if it is ok to skip selectors, is it ok to run them out of order or run twice?

With actual indexing of arrays and lists - yes. We assume no sudeeffects and indexing can be skipped, reordered, repeated. But what about selectors?

Sideeffecting selectors are evil, but we need some rational principle when dealing with them when skipping.
Is skipping the same as counting?

[stephentoub]

@VSadov, should we close this at this point?

[VSadov]

Yes. I think the question was settled at:

• we do run selectors/predicates as a part of aggregating queries. Ex: .Count runs all the preceding selectors on all elements.
• GetCount(bool onlyIfCheap) is used only for sizing of internal buffers, not for skipping sideeffects.
• we draw the line at operators that do not imply a particular implementation - Ex: OrderBy, GroupBy can elide/reorder/duplicate predicate calls.
• A special case of above is streaming/short-circuiting behavior of operators like Single. It is permitted to return with a failure once there is a second match. I think we took a compat break here. Considering that sequences could be large or infinite, "streaming" behavior is preferable, when possible, over "buffer all the elements" approach. Also consistent with other, more obvious cases like Any not counting the whole sequence.
• it is permitted to elide/reorder/duplicate operators not passed as lambdas (opEquals, GetHashCode, ordinary math, conversions).

[NetMage]

@VSadov What does your comment imply about the actual question here, namely Skip(10) or Take(0)?