Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[API Proposal] API for interpretting Span<byte> as struct without copying #26587

Closed
jkotas opened this issue Jun 22, 2018 · 11 comments
Closed

[API Proposal] API for interpretting Span<byte> as struct without copying #26587

jkotas opened this issue Jun 22, 2018 · 11 comments
Labels
api-approved API was approved in API review, it can be implemented area-System.Memory
Milestone

Comments

@jkotas
Copy link
Member

jkotas commented Jun 22, 2018

MemoryMarshal contains several APIs that allow reinterpreting Span<byte>, but none of them are a good fit for interpreting Span as struct without copying:

  • MemoryMarshal.Read allows interpreting Span<byte> as arbitrary struct. It creates a local copy that is fine and beneficial for small structs, but becomes expensive for large structs.
  • MemoryMarshal.Cast allows interpreting Span<byte> as sequence of arbitrary structs. It does not create a local copy, but comes with extra overhead required to compute the length of the resulting span.
  • MemoryMarshal.GetReference + Unsafe.As does not create a local copy, but also skips any bounds checks that is not good for security defense in depth.

Proposal

Add methods that allow interpreting Span<byte> as arbitrary large struct without copying to MemoryMarshal.

static class MemoryMarshal
{
    public static ref readonly T AsRef<T>(System.ReadOnlySpan<byte> span) where T : struct { throw null; }
    public static ref T AsRef(System.Span<byte> span) where T : struct { throw null; }
}

Implementation of these methods is going to be faster functional equivalent of ref MemoryMarshal.Cast<byte, T>(span)[0].

These APIs come with the same set of portability caveats as Memory.Cast. They can be only used when the memory block is sufficiently aligned by other means; or when the target platform does not require strong memory alignment.

@jkotas
Copy link
Member Author

jkotas commented Jun 22, 2018

Related proposal for other direction #24656

dotnet/corefx#30614 has multiple examples of how these methods can be used in real-world code.

@benaadams
Copy link
Member

A bit like Marshal.PtrToStructure<T> Method (IntPtr) but generally safer, including a >= sizeof(T) check; and ref returning rather than copying?

@jkotas
Copy link
Member Author

jkotas commented Jun 22, 2018

Right.

@morganbr
Copy link
Contributor

Could this use the unmanaged constraint instead of the struct constraint? GC references wouldn't be a good thing with this API.

@jkotas
Copy link
Member Author

jkotas commented Jun 26, 2018

The implementation will enforce no-GC references at runtime just like MemoryMarshal.Cast and other similar APIs. unmanaged constrain is enforcing more than no-GC references. It would make this API less useful and inconsistent with the rest of the MemoryMarshal and Span methods.

@terrajobst
Copy link
Member

terrajobst commented Jun 26, 2018

Video

This API and scenarios make sense but we shouldn't use Struct in the name. It seems to make more sense to align this with Unsafe.AsRef(), thus:

static class MemoryMarshal
{
    public static ref readonly T AsRef<T>(ReadOnlySpan<byte> span) where T : struct;
    public static ref T AsRef(Span<byte> span) where T : struct;
}

@mgravell
Copy link
Member

@terrajobst where T : unmanaged ?

@jkotas
Copy link
Member Author

jkotas commented Jun 29, 2018

@mgravell
Copy link
Member

mgravell commented Jun 29, 2018

@jkotas apparently I fail at reading today, thx

dotnet-maestro-bot referenced this issue in dotnet-maestro-bot/corefx Jul 19, 2018
dotnet-maestro-bot referenced this issue in dotnet-maestro-bot/corert Jul 19, 2018
jkotas referenced this issue in dotnet/corert Jul 19, 2018
jkotas referenced this issue in dotnet/corefx Jul 19, 2018
jkotas referenced this issue in jkotas/corefx Jul 20, 2018
- Add tests
- Take advantage of the new API in CoreFX

Fixes #30613
jkotas referenced this issue in jkotas/corefx Jul 21, 2018
- Add tests
- Take advantage of the new API in CoreFX

Fixes #30613
jkotas referenced this issue in dotnet/corefx Jul 23, 2018
* Expose MemoryMarshal.AsRef

- Add tests
- Take advantage of the new API in CoreFX

Fixes #30613

* Auto-generated System.Memory ref
@virzak
Copy link
Contributor

virzak commented Sep 5, 2019

Is it possible to use AsRef in .NET Standard 2.1? I was only able to use it targeting netcoreapp3.0

@danmoseley
Copy link
Member

@virzak best to ask this in the dotnet/standard repo

@msftgits msftgits transferred this issue from dotnet/corefx Jan 31, 2020
@msftgits msftgits added this to the 3.0 milestone Jan 31, 2020
@ghost ghost locked as resolved and limited conversation to collaborators Dec 16, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
api-approved API was approved in API review, it can be implemented area-System.Memory
Projects
None yet
Development

No branches or pull requests

8 participants