Span<T>

[jkotas]

Turn https://github.com/dotnet/corefxlab/tree/master/src/System.Slices into efficient first class feature.

• Initial implementation: Merge SpanOfT branch into master coreclr#7886
• Design document: https://github.com/dotnet/corefxlab/blob/master/docs/specs/span.md
• JIT - Correctness – proper GC reporting of the special byref field
  • Re-enable GC layout reporting for Span https://github.com/dotnet/coreclr/blob/f42984160f5f9407f70ebc2e86edfcf226fccd81/src/vm/jitinterface.cpp#L2306. JIT chokes on it - fragile assumption introduced by recent JIT throughput fix.
  • Intrinsic for reading/writing byrefs https://github.com/dotnet/coreclr/blob/f42984160f5f9407f70ebc2e86edfcf226fccd81/src/mscorlib/src/System/Span.cs#L39, multiple places
• VM - Correctness
  • proper GC reporting: https://github.com/dotnet/coreclr/issues/8517
  • enforcement of byreflike constrains https://github.com/dotnet/coreclr/issues/8516
  • https://github.com/dotnet/coreclr/issues/9032
• VM - Stress
  • GC reporting for tailcalls - https://github.com/dotnet/coreclr/issues/9032
  • GC reporting for struct passed in regs on Unix - https://github.com/dotnet/coreclr/issues/9033
• JIT - Optimizations – apply same sort of optimizations to Span as it does for Strings or Arrays (treat indexers as intrinsics, range check elimination, etc.)
  • Span methods may not inline when called from shared methods dotnet/coreclr#10031
  • Improve aliasing model for implicit-by-ref struct passing dotnet/coreclr#10049
  • Recognize/optimize Span bounds checks more like array bounds checks dotnet/coreclr#10050
  • Better optimization for CORINFO_HELP_RUNTIMEHANDLE_METHOD dotnet/coreclr#10051 [moved to future]
• Update buildtools with Roslyn that has local byref and locals C# 7 feature
• Implement the full public surface dotnet/coreclr#5857
• Add Span<T> to framework contracts
• Add Span<T> tests
• Add Span<T> and ReadOnlySpan<T> as yet another ByRef-like type in Roslyn, so that unsupported operations on Span<T> like boxing or creating Span<T>[] are compile errors: https://github.com/dotnet/csharplang/blob/master/proposals/csharp-7.2/span-safety.md
• Make sure there is a way to pin Span<T> for interop
• Debugging https://github.com/dotnet/coreclr/issues/8518 / https://github.com/dotnet/corefx/issues/14376
• Reflection - https://github.com/dotnet/coreclr/issues/17296
• Performance optimizations
  • Optimize Copy, Clear, Fill - https://github.com/dotnet/coreclr/issues/9161
• Marshaler support (as with arrays) https://github.com/dotnet/coreclr/issues/14460
• APIs across corefx that accept spans (https://github.com/dotnet/corefx/issues/21281)
• APIs for working with specialty spans (https://github.com/dotnet/corefx/issues/21395)

[stephentoub]

• Marshaler support (as with arrays)?

[adamsitnik]

I just love it, hope for some up-for-grabs issues!

[benaadams]

Does "Add Span to framework contracts" include items like adding to Stream Read/Write etc? Or is it making Span available as a type? (Assuming latter, and others will be new issues after)

[omariom]

The CoreFxLab's prototype has useful extensions methods for Span<byte> and Span<char> like Read/Write and StartsWith/EndsWith. Will CoreCLR's Span get them?

[dotnetchris]

Would there be any sane method to allow Span<> to integrate with List<> that it could operate on the underlying array?

[KrzysztofCwalina]

@omariom, we need the corfxlab API and the corlcr one to be the same (portable). The reason for it is that I think we need to use the corfxlab implementation for previously shipped runtime/frameworks, e.g. .NET Framework 4.5. Otherwise, the addoption of Span will be very much inhibited by inability to write portable code. Of course, the out-of-band span will be slower, but as long as the API is available in portable code, we should be good.

[KrzysztofCwalina]

@benaadams, we need some existing APIs, e.g. Stream.Read/Write to accept spans, but I think it's a separate workitem form the one @jkotas listed.

[KrzysztofCwalina]

@dotnetchris, what do you consider "sane" in this context? If you mean wrapping Span into List then no (Span is stack only, and does not always have "underlying array").

[KrzysztofCwalina]

@adamsitnik, I am not sure we want completely open up for grabs so early (as there is lots of tricky work here) but you have done lots of good work in corfxlab and so if you (presonally) would like to take dotnet/coreclr#5857, it would be great! Same for @omariom :-)

[adamsitnik]

@KrzysztofCwalina Thanks! consider it done ;)

[dotnetchris]

@KrzysztofCwalina i meant for Span to access List's internal array

[KrzysztofCwalina]

I think we might also need to constrain T to primitive type (i.e. type with no managed object references). Otherwise, it will be possible to create Span over native memory without GC being aware of the roots.

[omariom]

@KrzysztofCwalina
Only certain operations seem unsafe in this sense: Cast, BlockEquals and Span<byte>.Read/Write.

[KrzysztofCwalina]

@dotnetchris, yes, it would be great to add List<T>.Slice that returns Span<T>. Same with other slicable types, e.g. String.Slice -> Span<char>.

[KrzysztofCwalina]

@omariom, it seems like accessors and copy operations too, e.g.
var nativeSpan = new Span(nativeMemory, len);
nativeSpan[0] = new object();
// will GC know that the object is rooted? will it change the native memory when the object is rooted and moves?

Similarly when an array backed span of object sis copied to native span.

[omariom]

@KrzysztofCwalina
Then I think it is worth adding a new constraint to CLR and C# - blittable(or other name). And ability to apply it at members level.
With such constraint we could have Spans of any type but some operations would be only allowed on Spans of types having no references.

[jkotas]

For these kind of operations, I think the blittability should be enforced via a dynamic check if needed, that the JIT can eliminate by treating it as intrinsic; or by Roslyn analyzer. I do not think it makes sense to have first class blittable constrain - it is too much complexity, for the value that it provides.

Also, the casting between different blitable Span types is pretty questionable operation. It has portability problems because of it lets you create misaligned Spans. I do not think it should be in the "safe" Span API set. (Having it in "unsafe" span API set is fine.)

[jkotas]

var nativeSpan = new Span(nativeMemory, len);
nativeSpan[0] = new object();
// will GC know that the object is rooted? will it change the native memory when the object is rooted and moves?

When you are doing interop with unmanaged pointers, you have to know what you are doing and you have to get it right. If you do not get it right, your program will crash in a very bad way.

In your example, the GC will not track the native memory and your program will crash in a very bad way. It is no different from if you pass wrong len to Span(void * p, int len) that will crash your program in a very bad way as well.

I know that we have historically tried to prevent some of the bad uses, but I always had mixed feelings about it.

Maybe we should have the dynamic check for blitability in the regular Span(void * p, int len), but also have extension method without any checks in "unsafe" span API set for power-users?

[nietras]

I think it would be really good to have a "unsafe" span API set for high perf scenarios. As long as there will be a code path without unnecessary checks in cases where you know what you are doing.

Also I hope it will still be possible to query the span whether it is based on native or managed memory and have access to native pointer or managed array/pinning depending on which memory is used.

[omariom]

@jkotas

If Type had ContainsReferences property treated by JIT as a constant then it would be no-op in the dangerous methods when T is blittable and throw otherwise.

For CoreLab's Span another solution should be found. May be Reflection? Though it will be slow.

[nietras]

For corefx one might use the solution shown here: https://github.com/AndreyAkinshin/BlittableStructs/blob/master/BlittableStructs/BlittableHelper.cs

It is probably pretty slow (haven't benchmarked it) on first call, but should then be fast perhaps even inlineable. The helper is also discussed in a blog post: http://aakinshin.net/en/blog/dotnet/blittable/

[nblumhardt]

Using ReadOnlySpan<char> as the span type for substrings seems to preclude the addition of a lot of interesting/useful text-specific members (intuitive ToString(), StartsWith(string), etc.). Is there room for a separate StringSpan type for this case?

[jaredpar]

I do not think it makes sense to have first class blittable constrain - it is too much complexity, for the value that it provides.

The ability to convert from say Span<int> to Span<byte> proved really useful in Midori. It's something I'd love to preserve with CLR as well.

The language already has this exact concept in the spec under the guise unmanaged type. Essentially a struct that transitively contains no reference types. The compiler is already making this check for other scenarios. Why not expose it as a real language feature and generic constraint?

struct Point { int x; int y }
void M<T>() where T : blittable // implies struct
M<Point>() // Okay 
M<object>() // Nope, not blittable 

[benaadams]

The ability to convert from say Span to Span proved really useful in Midori.

Could even have a safe convert where it checks (length * sizeof(T1)) % sizeof(T2) == 0 (might want also an unsafe convert where it doesn't - as may be pre-checking and re-slicing).

You can already have explicit layout overlapping arrays; but the length property gets confused as its based on the instantiation type (which makes sense); but the span approach could have the appropriate length for each sized datatype.

Example use reading from stream/disk/network from byte[] => Vector3[] or saving Vector3[] => byte[]

[KrzysztofCwalina]

@nblumhardt, the operations you listed could be extension methods over ROS, couldn't they?

[nblumhardt]

@KrzysztofCwalina yes, though the ergonomics of extension methods are still second-rate because of the need to add a using statement before they're discoverable.

Tooling helps, and is improving, but at present it's still easier to find members on a type than it is to find extensions. The prevalence of "remove unused usings" as an automated refactoring means using System; is not reliably present in every code file. Perhaps a minor issue.

Even if the extension method route is preferred, could the character-oriented extensions like StartWith() be included in this PR, just in case the experience of implementing them reveals any opportunities.

(Stretching a little further, would implementing and optimizing the most common string-like methods on Span<char> open the door to providing them in the future on Span<byte> with UTF-8 or user-specified encoding?)

Thanks for the reply, apologies if I missed earlier discussions around this BTW :-)

[ilexp]

I've been (somewhat) following development of the C# Slicing issue, but I'm having trouble understanding how exactly these two issues are related. A few questions to clear things up:

1. Is this the CLR-part of the actual array slicing feature in C#, or considered something entirely different?
2. As far as I followed slicing, a core feature of an array slice was that it could be treated the same as an array. Otherwise - it would be impossible to integrate with existing libraries and APIs and essentially turn into another ArraySegment<T> that exists but is barely used. Will this be possible with a Span<T>?
3. It seems that Span<T> is currently considered stack-only. That does rule out a few use cases. Is this a temporary state or considered a final decision?

Overall, I'm in huge favor of having a mechanism (like Span<T>?) to provide differently typed, offset or truncated views on a memory block, mostly for performance reasons. However, I'm also worried that it might stop halfway to its full potential, see also this CoreCLR issue.

Sorry to interrupt - as a mostly passive observer, I might have missed some details.

[jaredpar]

@ilexp it's a bit confusing because the ability to have efficient slices requires both runtime and language work.

1. Yes. The CLR portion is making the necessary runtime changes to properly handle the "ref" field used in Span<T>.
2. No. Having Span<T> == T[] is not a goal of this particular approach. This approach is solving a slightly different problem: uniform representation for contiguous memory allocations. Whether that memory be arrays, native memory or strings. Unifying T[] and Span<T> is possible and has been explored in great detail. But it has a number of negative tradeoffs associated with it.
3. This particular type is indeed stack only. The design includes mechanisms for heap storage though when needed (so yes you will be able to use it in async methods).

[buybackoff]

@KrzysztofCwalina Memory<T> has a public property of a type Span<T>. If Span<T> is not supposed to work (at least initially) from F# (as @davidfowl said above), how Memory<T> will work? Or it will work just fine if I do not touch that property from code?

[KrzysztofCwalina]

@davidfowl did not say that Span will not work in F#. Today, C# does not implement these checks either, and Span works in C#. The checks just make it easier for the programmer using it to use it correctly. In the absence of the checks, the runtime will throw at runtime.

[ahsonkhan]

Performance optimizations - dotnet/coreclr#9161 completed.

[HFadeel]

When Span will land in .NET Framework?

[WillSullivan]

Spec docs link is wrong, actual is https://github.com/dotnet/corefxlab/blob/master/docs/specs/span.md

[jkotas]

@WillSullivan Updated - thanks.

[KrzysztofCwalina]

@HFadeel, Span<T> is in System.Memory nuget package. This package works on .NET Framework.

[jkotas]

Updated the milestone to 2.1. We need to go through the list above and make sure that there is nothing major falling through the cracks for Span<T> end-to-end experience for .NET Core 2.1.

cc @ahsonkhan @KrzysztofCwalina @stephentoub @joshfree

[joshfree]

thanks for giving this a nudge @jkotas

[KrzysztofCwalina]

Besides the VM correctness and stress workitems, I still worry that debug-ability of code using Spans is a pretty big issue. I will turn lots of people off spans.

[colombod]

Why so negative? This api is for scenarios that are not exact everyone’s cup of tea. Don’t see anything bad about the api and design.

[ahsonkhan]

Reflection?

Is this work item related to IsReferenceOrContainsReferences?

If so, we already have the implementation of that check in place (here and here). Is there any additional context for this?

[jkotas]

It is about calling methods on Span or that take Span arguments via reflection:

• It is not possible to do it via existing reflection methods. We should have test to verify that e.g. typeof(SpanExtensions).GetMethod("AsReadOnlySpan", new Type[] { typeof(string) }).Invoke(null, new object[] { "Hello" }); fails gracefully.
• We may want to look into adding new reflection APIs that allow calling these methods via reflection. This does not need to be for 2.1.

[ahsonkhan]

Marking "Debugging" and "APIs for working with specialty spans" as complete.

Also "VM - Correctness" and "VM - Stress" are done.

What's left:

• JIT - Optimizations – apply same sort of optimizations to Span as it does for Strings or Arrays (treat indexers as intrinsics, range check elimination, etc.)
  • Better optimization for CORINFO_HELP_RUNTIMEHANDLE_METHOD dotnet/coreclr#10051
• Reflection?
• Marshaler support (as with arrays) dotnet/coreclr#14460

[kstewart83]

What is the possibility of adding a Span/Memory constructor for working with memory mapped files? Currently, it looks like I have to have unsafe code in order to do this:

var dbPath = "test.txt";
var initialSize = 1024;
var mmf = MemoryMappedFile.CreateFromFile(dbPath);
var mma = mmf.CreateViewAccessor(0, initialSize).SafeMemoryMappedViewHandle;
Span<byte> bytes;
unsafe
{
    byte* ptrMemMap = (byte*)0;
    mma.AcquirePointer(ref ptrMemMap);
    bytes = new Span<byte>(ptrMemMap, (int)mma.ByteLength);
}

Also, it seems like I can only create Spans, as there aren't public constructors for Memory that take a pointer (maybe I'm missing the reason for this). But since the view accessors have safe memory handles that implement System.Runtime.InteropServices.SafeBuffer (i.e., they have a pointer and a length)...it seems natural to be able to leverage this for Span/Memory. So what would be nice is something like this:

var dbPath = "test.txt";
var initialSize = 1024;
var mmf = MemoryMappedFile.CreateFromFile(dbPath);
var mma = mmf.CreateViewAccessor(0, initialSize).SafeMemoryMappedViewHandle;
var mem = new Memory(mma);
var span = mem.Span.Slice(0, 512);

I also noticed that the indexer and internal length of Span uses int. With memory mapped files (especially for database scenarios) it is reasonable that the target file will exceed the upper limit for int. I'm not sure about the performance impact of long based indexing or if there is some magic way to have it both ways, but it would be convenient for certain scenarios.

[jkotas]

https://github.com/dotnet/corefx/issues/26603 has the discussion about Span & memory mapped files.

[kstewart83]

Unfortunately, looking at https://github.com/dotnet/corefx/issues/26603 along with the referenced code in the benchmarks didn't clear things up for me. It seems like that particular use case is geared to copying small bits of the memory mapped files into Spans and ReadOnlySegments. It looks like the solution still involves unsafe code with OwnedMemory<T>, which is what I'd like to avoid. I don't have experience with manual memory management in C#, so some of this is a little difficult to grasp. That's what I found appealing about Span/Memory is that I could now access additional performance and reduce/eliminate copying data around without the headache of manual memory management and the issues that come with it. It seems memory mapped files fit into target paradigm of Span/Memory (unifying the APIs around contiguous random access memory), so hopefully some type of integration of memory mapped files and Span/Memory makes it in at some point.

[davidfowl]

@KrzysztofCwalina I think we should create something first class with Memory mapped files and the new buffer primitives (ReadOnlySequence).

@kstewart83 all we have right now are extremely low level primitives that you have to string together to make something work. That specific issue was about the performance gap between using Span directly and using the ReadOnlySequence (the gap has been reduced for that specific scenario).

Dealing with anything bigger than an int you'll need to use ReadOnlySequence<T> which is just a view over a linked list of ReadOnlyMemory<T>.

[ahsonkhan]

I think we should create something first class with Memory mapped files and the new buffer primitives (ReadOnlySequence).

I have re-opened the dotnet/corefx#26603 issue. Feel free to move the discussion there (See https://github.com/dotnet/corefx/issues/26603#issuecomment-370306008).

[RussKeldorph]

@ahsonkhan Is there more work on this issue for 2.1?

[ahsonkhan]

Is there more work on this issue for 2.1?

There is no other work for 2.1.

There are two items left from the original list, but both are marked as future. They can be tracked separately (and outside of 2.1):

• Marshaler support (as with arrays) dotnet/coreclr#14460
• Reflection dotnet/coreclr#17296

Closing this issue!