Skip to content
This repository was archived by the owner on Jan 23, 2023. It is now read-only.

Added Enumerable.Append and Enumerable.Prepend#5947

Merged
VSadov merged 2 commits intodotnet:masterfrom
VSadov:prepend
Feb 9, 2016
Merged

Added Enumerable.Append and Enumerable.Prepend#5947
VSadov merged 2 commits intodotnet:masterfrom
VSadov:prepend

Conversation

@VSadov
Copy link
Copy Markdown
Member

@VSadov VSadov commented Feb 6, 2016

New public API in System.Linq

Closes #2075

@VSadov
Copy link
Copy Markdown
Member Author

VSadov commented Feb 6, 2016

@stephentoub @terrajobst
as agreed in https://github.com/dotnet/corefx/issues/2075

NOTE: "element" vs. "value" difference. It seems that "element" would be more consistent with other Enumerable methods.

@VSadov
Copy link
Copy Markdown
Member Author

VSadov commented Feb 6, 2016

@weshaggard
updated refs per the new public API.
this may have effect on legacy FX builds

}

[Fact]
public void SameResultsRepeatCallsIntQueryPrePend()
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: Prepend instead of PrePend

@JonHanna
Copy link
Copy Markdown
Contributor

JonHanna commented Feb 7, 2016

LGTM.

Maybe consider adding the tests:

[Fact]
public void ForcedToEnumeratorDoesntEnumeratePrepend()
{
    var iterator = NumberRangeGuaranteedNotCollectionType(0, 3).Prepend(4);
    // Don't insist on this behaviour, but check it's correct if it happens
    var en = iterator as IEnumerator<int>;
    Assert.False(en != null && en.MoveNext());
}

[Fact]
public void ForcedToEnumeratorDoesntEnumerateAppend()
{
    var iterator = NumberRangeGuaranteedNotCollectionType(0, 3).Append(4);
    // Don't insist on this behaviour, but check it's correct if it happens
    var en = iterator as IEnumerator<int>;
    Assert.False(en != null && en.MoveNext());
}

While the regression this would catch is far from likely, considering the number of times custom iterators are used in Enumerable, and the number of tweaks done in the name of performance, I added this to all the existing enumerable types, both custom and compiler-produced. Otherwise it could be tempting to those of us who like shaving off nanoseconds in this class to cut out a few checks and state changes that would trigger this.

@VSadov
Copy link
Copy Markdown
Member Author

VSadov commented Feb 7, 2016

@JonHanna Thanks!
Made suggested changes.
Also added a test for Prepend to make sure we are not iterating the source before yielding the prepended element. I think this behavior could be expected and taken dependency on.

@JonHanna
Copy link
Copy Markdown
Contributor

JonHanna commented Feb 7, 2016

Why is System.Linq.Expressions.Tests.Queryable_Tests.MatchSequencePattern at https://github.com/dotnet/corefx/blob/master/src/System.Linq.Expressions/tests/SequenceTests/SequenceTests.cs#L1660 not failing? I don't see a change to it, but it's supposed to fail if Enumerable has extension methods that Queryable doesn't unless they're mentioned in the list in that test.


ie.Add(42);

Assert.Equal(prepended, ie.Prepend(4));
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this test should assert that GetEnumerator() wasn't called prior to the second element being iterated. I can't see someone finding a way that iterating it had any advantage (quite the opposite), but eagerly calling GetEnumerator() might possibly profile as a slight improvement (or simply an implementation short-cut with less typing!), would have worse potential problems than this, and would cover everything this covers too.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea was primarily to catch attempts to "probe" the source for emptiness or something like that before yielding the prepended element. It could be a problem if the whole thing is used in TakeWhile and if fetching elements from the source is somehow observable.

Considering how simple this operator is, this kind of changes are unlikely to happen unnoticed. I just added a testcase since I thought about the scenario while making changes.
Overall, I think we should have enough tests here already.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's throwing out ideas, not an objection. I'm still confused by MatchSequencePattern not failing, though.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm still confused by MatchSequencePattern not failing, though.

Probably because System.Linq.Expressions doesn't have a P2P reference to System.Linq and is building against the bits in the package rather than what's live in the repo.
cc: @weshaggard

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah. So it won't fail until after this is merged? I was pretty sure it failed before under similar circumstances.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will add the new operators to the exclusion list.
We can consider adding them to IQueryable, but not in this change. It is not 100% match anyways.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, I don't think they're a match, being akin to Repeat and Range in producing elements.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Modulo the query operators that are not extension methods on IEnumerable<T>, as far as I know, the IQueryable<T> surface is complete, so we should likely bring it on par for these operators (which are extension methods) in a separate change as well.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably open an issue to discuss that, but I'm inclined to think that there's no good way to universally do that across providers except by casting back to IEnumerable<T> and calling into the methods here anyway, which this already provides for given the inheritance order.

@stephentoub
Copy link
Copy Markdown
Member

Assuming we're ok with new surface area in corefx's System.Linq that's not in the full framework's (@weshaggard), LGTM.

New public API in System.Linq

Closes #2075
@VSadov VSadov force-pushed the prepend branch 3 times, most recently from 477fa7a to 9d7317b Compare February 9, 2016 05:52
@VSadov
Copy link
Copy Markdown
Member Author

VSadov commented Feb 9, 2016

test this please

* Bumped up version of System.Linq
* Added Append/Prepend to the MatchSequencePattern test in System.Linq.Expressions  since the new operators are not avaialble on IQueryable.
* added a test for combinations of Append, Prepend, Concat
@VSadov
Copy link
Copy Markdown
Member Author

VSadov commented Feb 9, 2016

test this please

VSadov added a commit that referenced this pull request Feb 9, 2016
Added   Enumerable.Append and Enumerable.Prepend
@VSadov VSadov merged commit 998d957 into dotnet:master Feb 9, 2016
@ericstj
Copy link
Copy Markdown
Member

ericstj commented Feb 10, 2016

This PR actually failed but jenkins didn't catch it:

00:15:53 d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\Tools\Packaging.targets(843,5): error : System.Linq should be supported on .NETCore,Version=v4.5/win8-x86 but has no compile assets. [d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\src\System.Linq\pkg\System.Linq.pkgproj]
00:15:53 d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\Tools\Packaging.targets(843,5): error : System.Linq should be supported on .NETCore,Version=v4.5/win8-x86 but has no runtime assets. [d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\src\System.Linq\pkg\System.Linq.pkgproj]
00:15:53 d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\Tools\Packaging.targets(843,5): error : System.Linq should be supported on .NETCore,Version=v4.5/win8-x64 but has no compile assets. [d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\src\System.Linq\pkg\System.Linq.pkgproj]
00:15:53 d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\Tools\Packaging.targets(843,5): error : System.Linq should be supported on .NETCore,Version=v4.5/win8-x64 but has no runtime assets. [d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\src\System.Linq\pkg\System.Linq.pkgproj]
00:15:53 d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\Tools\Packaging.targets(843,5): error : System.Linq should be supported on .NETCore,Version=v4.5/win8-arm but has no compile assets. [d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\src\System.Linq\pkg\System.Linq.pkgproj]
00:15:53 d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\Tools\Packaging.targets(843,5): error : System.Linq should be supported on .NETCore,Version=v4.5/win8-arm but has no runtime assets. [d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\src\System.Linq\pkg\System.Linq.pkgproj]
00:15:53 d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\Tools\Packaging.targets(843,5): error : System.Linq should be supported on .NETCore,Version=v4.5.1/win81-x86 but has no compile assets. [d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\src\System.Linq\pkg\System.Linq.pkgproj]
00:15:53 d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\Tools\Packaging.targets(843,5): error : System.Linq should be supported on .NETCore,Version=v4.5.1/win81-x86 but has no runtime assets. [d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\src\System.Linq\pkg\System.Linq.pkgproj]
00:15:53 d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\Tools\Packaging.targets(843,5): error : System.Linq should be supported on .NETCore,Version=v4.5.1/win81-x64 but has no compile assets. [d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\src\System.Linq\pkg\System.Linq.pkgproj]
00:15:53 d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\Tools\Packaging.targets(843,5): error : System.Linq should be supported on .NETCore,Version=v4.5.1/win81-x64 but has no runtime assets. [d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\src\System.Linq\pkg\System.Linq.pkgproj]
00:15:53 d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\Tools\Packaging.targets(843,5): error : System.Linq should be supported on .NETCore,Version=v4.5.1/win81-arm but has no compile assets. [d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\src\System.Linq\pkg\System.Linq.pkgproj]
00:15:53 d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\Tools\Packaging.targets(843,5): error : System.Linq should be supported on .NETCore,Version=v4.5.1/win81-arm but has no runtime assets. [d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\src\System.Linq\pkg\System.Linq.pkgproj]
00:15:53 d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\Tools\Packaging.targets(843,5): error : System.Linq should be supported on .NETFramework,Version=v4.5/win-x86 but has no compile assets. [d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\src\System.Linq\pkg\System.Linq.pkgproj]
00:15:53 d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\Tools\Packaging.targets(843,5): error : System.Linq should be supported on .NETFramework,Version=v4.5/win-x86 but has no runtime assets. [d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\src\System.Linq\pkg\System.Linq.pkgproj]
00:15:53 d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\Tools\Packaging.targets(843,5): error : System.Linq should be supported on .NETFramework,Version=v4.5/win-x64 but has no compile assets. [d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\src\System.Linq\pkg\System.Linq.pkgproj]
00:15:53 d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\Tools\Packaging.targets(843,5): error : System.Linq should be supported on .NETFramework,Version=v4.5/win-x64 but has no runtime assets. [d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\src\System.Linq\pkg\System.Linq.pkgproj]
00:15:53 d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\Tools\Packaging.targets(843,5): error : System.Linq should be supported on .NETFramework,Version=v4.5.1/win-x86 but has no compile assets. [d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\src\System.Linq\pkg\System.Linq.pkgproj]
00:15:53 d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\Tools\Packaging.targets(843,5): error : System.Linq should be supported on .NETFramework,Version=v4.5.1/win-x86 but has no runtime assets. [d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\src\System.Linq\pkg\System.Linq.pkgproj]
00:15:53 d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\Tools\Packaging.targets(843,5): error : System.Linq should be supported on .NETFramework,Version=v4.5.1/win-x64 but has no compile assets. [d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\src\System.Linq\pkg\System.Linq.pkgproj]
00:15:53 d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\Tools\Packaging.targets(843,5): error : System.Linq should be supported on .NETFramework,Version=v4.5.1/win-x64 but has no runtime assets. [d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\src\System.Linq\pkg\System.Linq.pkgproj]
00:15:53 d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\Tools\Packaging.targets(843,5): error : System.Linq should be supported on .NETFramework,Version=v4.6/win-x86 but has no compile assets. [d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\src\System.Linq\pkg\System.Linq.pkgproj]
00:15:53 d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\Tools\Packaging.targets(843,5): error : System.Linq should be supported on .NETFramework,Version=v4.6/win-x86 but has no runtime assets. [d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\src\System.Linq\pkg\System.Linq.pkgproj]
00:15:53 d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\Tools\Packaging.targets(843,5): error : System.Linq should be supported on .NETFramework,Version=v4.6/win-x64 but has no compile assets. [d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\src\System.Linq\pkg\System.Linq.pkgproj]
00:15:53 d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\Tools\Packaging.targets(843,5): error : System.Linq should be supported on .NETFramework,Version=v4.6/win-x64 but has no runtime assets. [d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\src\System.Linq\pkg\System.Linq.pkgproj]
00:15:53 d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\Tools\Packaging.targets(843,5): error : System.Linq should be supported on .NETFramework,Version=v4.6/win7-x86 but has no compile assets. [d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\src\System.Linq\pkg\System.Linq.pkgproj]
00:15:53 d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\Tools\Packaging.targets(843,5): error : System.Linq should be supported on .NETFramework,Version=v4.6/win7-x86 but has no runtime assets. [d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\src\System.Linq\pkg\System.Linq.pkgproj]
00:15:53 d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\Tools\Packaging.targets(843,5): error : System.Linq should be supported on .NETFramework,Version=v4.6/win7-x64 but has no compile assets. [d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\src\System.Linq\pkg\System.Linq.pkgproj]
00:15:53 d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\Tools\Packaging.targets(843,5): error : System.Linq should be supported on .NETFramework,Version=v4.6/win7-x64 but has no runtime assets. [d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\src\System.Linq\pkg\System.Linq.pkgproj]
00:15:53 d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\Tools\Packaging.targets(843,5): error : System.Linq should be supported on .NETFramework,Version=v4.6.1/win-x86 but has no compile assets. [d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\src\System.Linq\pkg\System.Linq.pkgproj]
00:15:53 d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\Tools\Packaging.targets(843,5): error : System.Linq should be supported on .NETFramework,Version=v4.6.1/win-x86 but has no runtime assets. [d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\src\System.Linq\pkg\System.Linq.pkgproj]
00:15:53 d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\Tools\Packaging.targets(843,5): error : System.Linq should be supported on .NETFramework,Version=v4.6.1/win-x64 but has no compile assets. [d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\src\System.Linq\pkg\System.Linq.pkgproj]
00:15:53 d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\Tools\Packaging.targets(843,5): error : System.Linq should be supported on .NETFramework,Version=v4.6.1/win-x64 but has no runtime assets. [d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\src\System.Linq\pkg\System.Linq.pkgproj]
00:15:53 d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\Tools\Packaging.targets(843,5): error : System.Linq should be supported on .NETFramework,Version=v4.6.1/win7-x86 but has no compile assets. [d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\src\System.Linq\pkg\System.Linq.pkgproj]
00:15:53 d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\Tools\Packaging.targets(843,5): error : System.Linq should be supported on .NETFramework,Version=v4.6.1/win7-x86 but has no runtime assets. [d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\src\System.Linq\pkg\System.Linq.pkgproj]
00:15:53 d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\Tools\Packaging.targets(843,5): error : System.Linq should be supported on .NETFramework,Version=v4.6.1/win7-x64 but has no compile assets. [d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\src\System.Linq\pkg\System.Linq.pkgproj]
00:15:53 d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\Tools\Packaging.targets(843,5): error : System.Linq should be supported on .NETFramework,Version=v4.6.1/win7-x64 but has no runtime assets. [d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\src\System.Linq\pkg\System.Linq.pkgproj]
00:15:53 d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\Tools\Packaging.targets(843,5): error : System.Linq should be supported on WindowsPhoneApp,Version=v8.1 but has no compile assets. [d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\src\System.Linq\pkg\System.Linq.pkgproj]
00:15:53 d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\Tools\Packaging.targets(843,5): error : System.Linq should be supported on WindowsPhoneApp,Version=v8.1 but has no runtime assets. [d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\src\System.Linq\pkg\System.Linq.pkgproj]
00:15:53 d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\Tools\Packaging.targets(843,5): error : System.Linq should be supported on WindowsPhone,Version=v8.0 but has no compile assets. [d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\src\System.Linq\pkg\System.Linq.pkgproj]
00:15:53 d:\j\workspace\dotnet_corefx\windows_nt_debug_prtest\Tools\Packaging.targets(843,5): error : System.Linq should be supported on WindowsPhone,Version=v8.0 but has no runtime assets. 

@mmitche how are we evaluating build success/failure?

@mmitche
Copy link
Copy Markdown
Member

mmitche commented Feb 11, 2016

@ericstj The return code from build.cmd (or whatever the build step was). 0 = pass, anything else = fail

@ericstj
Copy link
Copy Markdown
Member

ericstj commented Feb 12, 2016

@mmitche that's not consistent with MSBuild's error reporting apparently.

@bartonjs
Copy link
Copy Markdown
Member

@ChadNedzlek I feel like you said that you saw where this particular error got downgraded from "big red text that scares developers" to "don't fail the build". Or did I misunderstand?

@ChadNedzlek
Copy link
Copy Markdown

I just recall seeing that in the PR that fixes the build break, there was a change from ContinueOnError="WarnAndContinue" to ContinueOnError="ErrorAndContinue". #6018, the changes to file "build.proj". We had previously explicitly been ignoring these failures

@ericstj
Copy link
Copy Markdown
Member

ericstj commented Feb 12, 2016

@ChadNedzlek that was a different task for package restore https://github.com/dotnet/corefx/pull/6018/files#diff-3ad318a39efd856540e8cf813f20ec09R63. The PR also had my real fix for this issue by updating the package authoring. dotnet-bot@0487996

This one definitely has big red text that scares developers and shows up in the build summary. I suspect msbuild is still returning 0 because we don't return false from the task. We do this specifically because we don't want to halt the build. We can switch the task to return false, but then put ErrorAndContinue on the task to see if that results in an error and a non-zero return value without halting the build on the error.

@weshaggard
Copy link
Copy Markdown
Member

It sounds like we are hitting dotnet/buildtools#401 which I created and assigned to @ericstj a while ago when I hit something similar.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.