Skip to content
This repository has been archived by the owner on Jan 23, 2023. It is now read-only.

Add span-based CopyTo and CopyToAsync methods #27639

Merged
merged 15 commits into from Nov 13, 2019

Conversation

manandre
Copy link

@manandre manandre commented Nov 3, 2019

@stephentoub
Copy link
Member

Thanks for working on this.

A large part of the value of such APIs is derived types providing a better implementation. Can you please do so in this PR for the Streams in CoreLib, e.g.MemoryStream?

src/System.Private.CoreLib/shared/System/IO/Stream.cs Outdated Show resolved Hide resolved
src/System.Private.CoreLib/shared/System/IO/Stream.cs Outdated Show resolved Hide resolved
src/System.Private.CoreLib/shared/System/IO/Stream.cs Outdated Show resolved Hide resolved
@stephentoub
Copy link
Member

ps MemoryStream was one example. Others include NullStream, FileStream, UnmanagedMemoryStream, and probably another one or more I'm forgetting. Thanks.

@stephentoub
Copy link
Member

Thanks for continuing to work on this, @manandre.

Still some open issues that looks like may have been missed:

  • Why is the validation helper method necessary? Since the overload is just going to delegate to the Stream-based overload that already does these same checks, isn't the only check we need in this overload whether the callback is null?
  • What about other streams in corelib, like NullStream, FileStream, UnmanagedMemoryStream, etc.?

Thanks.

@manandre
Copy link
Author

manandre commented Nov 4, 2019

Why is the validation helper method necessary? Since the overload is just going to delegate to the Stream-based overload that already does these same checks, isn't the only check we need in this overload whether the callback is null?

Can we safely rely on CopyTo/CopyToAsync(Stream) to ensure arguments validation? even if they are overriden by a custom stream implementation?
If yes, I agree to check only whether the callback is null.

What about other streams in corelib, like NullStream, FileStream, UnmanagedMemoryStream, etc.?

Let's discuss it:

@maryamariyan
Copy link
Member

Thank you for your contribution. As announced in dotnet/coreclr#27549 this repository will be moving to dotnet/runtime on November 13. If you would like to continue working on this PR after this date, the easiest way to move the change to dotnet/runtime is:

  1. In your coreclr repository clone, create patch by running git format-patch origin
  2. In your runtime repository clone, apply the patch by running git apply --directory src/coreclr <path to the patch created in step 1>

@stephentoub
Copy link
Member

stephentoub commented Nov 7, 2019

Can we safely rely on CopyTo/CopyToAsync(Stream) to ensure arguments validation? even if they are overriden by a custom stream implementation?

Yes. What validation an override chooses to perform is up to that override, based on the logic the override is going to use in its implementation. When this overload delegates to it, it's up to the override to provide the validation needed for the arguments provided to it, and we just need to validate the others.

NullStream: The override only saves the creation of the WriteCallbackStream instance. Does it worth it?

Yes.

UnmanagedMemoryStream: Can we optimize the copy here?

For the sync overload, it can just create a span from its memory and call the action with that.

UnmanagedMemoryStreamWrapper
PinnedBufferMemoryStream
ManifestResourceStream

Don't worry about these; they're very specialized streams and it's unlikely this method is ever going to be used on them, and if it they happen to be, the overhead of using the base shouldn't be impactful.

FileStream

There's a lot of work happening in FileStream's existing CopyToAsync. Probably not worth overriding.

@manandre
Copy link
Author

@stephentoub Can you please do a final review in order to give us a chance to get this pull request merged before the migration to dotnet/runtime? 🤞

Copy link
Member

@stephentoub stephentoub left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Several remaining comments to be addressed. Otherwise, LGTM. Thanks.

Co-Authored-By: Stephen Toub <stoub@microsoft.com>
@stephentoub
Copy link
Member

/azp run coreclr-ci

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

Copy link
Member

@stephentoub stephentoub left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@manandre
Copy link
Author

/azp run coreclr-ci

@azure-pipelines
Copy link

Commenter does not have sufficient privileges for PR 27639 in repo dotnet/coreclr

@manandre
Copy link
Author

@stephentoub Help 🆘

@stephentoub stephentoub merged commit 6abc099 into dotnet:master Nov 13, 2019
@stephentoub
Copy link
Member

Failures were unrelated.

Thanks!

@manandre manandre deleted the stream-copyto-span branch November 14, 2019 12:02
Dotnet-GitSync-Bot pushed a commit to Dotnet-GitSync-Bot/corefx that referenced this pull request Nov 14, 2019
* Add span-based CopyTo and CopyToAsync methods

* Update according to feedback

* Add span-based CopyTo overrides for MemoryStream

* Improve span-based CopyTo arguments validation

To avoid code duplication

* Update according to second review

Stream API is changed

* Resolve InternalReadSpan/Memory inlining

* Refactor ValidateCopyToArgs

* Update according to third review

* Update after fourth review

* Override span CopyTo for UnmanagedMemoryStream

* Apply suggestions from code review

Co-Authored-By: Stephen Toub <stoub@microsoft.com>

* Update after fifth review

* Add cross sync/async support for span-based CopyTo

* Call sync action directly in async context

* Rework cross sync/async support for span-based CopyTo

Co-Authored-By: Stephen Toub <stoub@microsoft.com>

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>
Dotnet-GitSync-Bot pushed a commit to Dotnet-GitSync-Bot/corert that referenced this pull request Nov 14, 2019
* Add span-based CopyTo and CopyToAsync methods

* Update according to feedback

* Add span-based CopyTo overrides for MemoryStream

* Improve span-based CopyTo arguments validation

To avoid code duplication

* Update according to second review

Stream API is changed

* Resolve InternalReadSpan/Memory inlining

* Refactor ValidateCopyToArgs

* Update according to third review

* Update after fourth review

* Override span CopyTo for UnmanagedMemoryStream

* Apply suggestions from code review

Co-Authored-By: Stephen Toub <stoub@microsoft.com>

* Update after fifth review

* Add cross sync/async support for span-based CopyTo

* Call sync action directly in async context

* Rework cross sync/async support for span-based CopyTo

Co-Authored-By: Stephen Toub <stoub@microsoft.com>

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>
Dotnet-GitSync-Bot pushed a commit to Dotnet-GitSync-Bot/mono that referenced this pull request Nov 14, 2019
* Add span-based CopyTo and CopyToAsync methods

* Update according to feedback

* Add span-based CopyTo overrides for MemoryStream

* Improve span-based CopyTo arguments validation

To avoid code duplication

* Update according to second review

Stream API is changed

* Resolve InternalReadSpan/Memory inlining

* Refactor ValidateCopyToArgs

* Update according to third review

* Update after fourth review

* Override span CopyTo for UnmanagedMemoryStream

* Apply suggestions from code review

Co-Authored-By: Stephen Toub <stoub@microsoft.com>

* Update after fifth review

* Add cross sync/async support for span-based CopyTo

* Call sync action directly in async context

* Rework cross sync/async support for span-based CopyTo

Co-Authored-By: Stephen Toub <stoub@microsoft.com>

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>
jkotas pushed a commit to dotnet/corert that referenced this pull request Nov 15, 2019
* Add span-based CopyTo and CopyToAsync methods

* Update according to feedback

* Add span-based CopyTo overrides for MemoryStream

* Improve span-based CopyTo arguments validation

To avoid code duplication

* Update according to second review

Stream API is changed

* Resolve InternalReadSpan/Memory inlining

* Refactor ValidateCopyToArgs

* Update according to third review

* Update after fourth review

* Override span CopyTo for UnmanagedMemoryStream

* Apply suggestions from code review

Co-Authored-By: Stephen Toub <stoub@microsoft.com>

* Update after fifth review

* Add cross sync/async support for span-based CopyTo

* Call sync action directly in async context

* Rework cross sync/async support for span-based CopyTo

Co-Authored-By: Stephen Toub <stoub@microsoft.com>

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>
akoeplinger pushed a commit to mono/mono that referenced this pull request Nov 15, 2019
* Add span-based CopyTo and CopyToAsync methods

* Update according to feedback

* Add span-based CopyTo overrides for MemoryStream

* Improve span-based CopyTo arguments validation

To avoid code duplication

* Update according to second review

Stream API is changed

* Resolve InternalReadSpan/Memory inlining

* Refactor ValidateCopyToArgs

* Update according to third review

* Update after fourth review

* Override span CopyTo for UnmanagedMemoryStream

* Apply suggestions from code review

Co-Authored-By: Stephen Toub <stoub@microsoft.com>

* Update after fifth review

* Add cross sync/async support for span-based CopyTo

* Call sync action directly in async context

* Rework cross sync/async support for span-based CopyTo

Co-Authored-By: Stephen Toub <stoub@microsoft.com>

Signed-off-by: dotnet-bot <dotnet-bot@microsoft.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
6 participants