CL-2023-08-03.md

Collection literals working group meeting for August 3, 2023

Agenda

• Collection expression escape scope for ref-structs (like ReadOnlySpan<T>).
• Confirm behavior of collection expressions and ReadOnlySpans of blittable data.

Discussion

The purpose of the meeting was to discuss what semantics we would have in the language for a collection expression target-typed to a ref-struct.

First, for simplicity and to aid by giving examples, we'll use ReadOnlySpan<T> to demonstrate the general problem space, the varying options, and the pros/cons of each option. Second, all these cases deal with collection expression elements of non-blittable-constant data (e.g. [a, b, c] vs [1, 2, 3]). Rules around the latter are covered at the end.

Today, a user can write code like so:

ReadOnlySpan<int> x1 = stackalloc int[] { a, b, c }; // stack allocated, local scope.
ReadOnlySpan<int> x2 = new int[] { a, b, c };        // heap allocated, global scope.

Based on how the variable is initialized, the language then dictates rules about how it can be used. For example, with the above code it would be illegal to return x1; from the method (since it has local scope), while it would be fine to return x2;. Variables without an initializer are assumed to have global scope. So the following, for example, would be illegal:

ReadOnlySpan<int> x3;              // global scope
x3 = stackalloc int[] { a, b, c }; // illegal.  global scope now points at local data.

Unlike existing constructs, which clearly delineate whether they are on the stack or heap, a decision needs to be made as to what the following means:

ReadOnlySpan<int> x4 = [a, b, c];

The working group considered several options.

Option 1. Stack-alloc'ed only if target has local-scope.

The first option considered was that a collection expression used with a ReadOnlySpan<T> would itself only be stack-alloc'ed if the span itself was known to be local scoped based on its declaration. In other words, the following:

ReadOnlySpan<int> x1 = [a, b, c];        // heap allocated, global scope.
scoped ReadonlySpan<int> x2 = [a, b, c]; // stack allocated, local scope.

// Less likely, but shown for completeness:
ReadOnlySpan<int> x3 = stackalloc int[] { d, e, f }; // stack allocated, local scope.
x3 = [g, h, i];                                      // stack allocated.

This has the positive that both heap and local allocations are possible, and choosing between them is as simple as adding scoped or not. Unfortunately it has several major negatives the working group was quite dissatisfied with:

1. We believe the common case for using spans with collection expressions will be for people wanting stack allocated data. Having the default (non-scoped) syntax basically flips that and makes the common case more cluttered and less pleasant to work with.
2. We believe the standard intuition for nearly all users (including experts in the low-level space) would be that the first line produces stack-allocated data. After all, that's the most common reason to create locals of span types. Their belief would be that if they initialize with a collection expression, it will do the right thing.
3. We believe that the first line would actually be a large "pit of fail" for users. It would be very easy and attractive for customers to just take existing code, hear about collection expressions (including that it works for spans) and translate stackalloc code over to just using the collection expression, resulting in allocations that didn't exist before.
4. We believe if we had this behavior, we would have to have a diagnostic (likely on by default) that warned the user of the heap allocation on 'x1' and to either:
   1. suppress the diagnostic if they really wanted that, or
   2. put an explicit cast (e.g. (int[])[a, b, c]) to indicate that they indeed did want this behavior, or
   3. add the scoped modifier.

All in all, while this allowed for both cases to easily be distinguished with a simple modifier, everything about the defaults and behavior was flipped from where we wanted it to be. This then led us to option 2:

Option 2. Determine what scoping a target has based on usage.

A discussion has been going on about allowing the user to just write:

ReadOnlySpan<int> x1 = [a, b, c]; // Could be local or global scoped
scoped ReadOnlySpan<int> x2 = [a, b, c]; // Always local scoped.

and then determining the scoping based on how the variable is used. For example, if 'x1' does not escape, then it could be stack allocated. If it does end up escaping though (for example with return x1;), the collection expression would be heap allocated. 'x2' would always be local scoped of course and would be prevented from escaping.

The benefit of this approach would be allowing the user to always write the simple form with a collection expression, and then get the semantics they needed to make the code legal.

This was attractive because of the simplicity of the user technically writing simple code, and then not having any stumbling blocks. However, we felt this was actually a net negative. A significant problem we foresee with this approach is that users will have code that is fast and nicely stack-allocating as they expect, and then some innocuous looking change ends up causing the compiler to conservatively think it may escape, causing the collection to now be heap allocated. This could happen silently, easily tanking program performance. Or, it could come with a warning. But in the latter case, users would then have to suppress or introduce explicit casts.

Similar to option 1, it felt very strange for us to introduce behavior where the compiler would heap allocate for the user only to warn, since in the majority of times it was believed users would not want that. After all, in the case of heap allocating, one might as well just choose a heap allocated type to begin with.

These concerns led us to our final option:

Option 3. Always stack-alloc for ref-struct target type.

We then moved to a discussion around our belief that when using ref-structs, and collection expressions, users would expect that data to be put on the stack. Exploring this led to the following semantics:

ReadOnlySpan<int> x1 = [a, b, c];        // stack allocated, local scope.
scoped ReadonlySpan<int> x2 = [a, b, c]; // stack allocated, local scope.

In other words, the above two have the same meaning. With the latter just being explicit about the local scoping. Returning either of those from a method would be illegal. This has the benefit that the simple form matches the intuition nearly all users will want this to have, and will provide the best performance by default. This heavily aligns with the stated goal of collection expressions (which we will continue to evangelize it with) that it makes the right default choices and by using it you should always (ideally) be getting the best performance results.

This does have some downsides though.

First, if the user did want to return the ref struct, they would now be blocked. The workaround for this would be to instead be explicit that you wanted heap allocation for this span (e.g. ReadOnlySpan<int> x1 = (int[])[1, 2, 3];). However, we see this explicitness as a good thing. When working with spans and collection-expressions, we feel that having code be explicit when it is on the heap is actually a desirable documentary property for it to have. And, while the cast syntax is not attractive, we could consider a simpler way of expressing that in the future (e.g. new [1, 2, 3]), where the syntax was light, but clear about intent.

Second, the aforementioned redundancy if you actually include scoped. We could consider warning or blocking that to prevent confusion and to help convey that the data is always scoped. That said, we already have this redundancy with scoped ReadOnlySpan<int> x = stackalloc int[] { ... }; and we give no warning. So we likely can just do the same here.

This approach also has the benefit that no diagnostic need ever be created for a span-targeted collection expression actually being heap allocated. As we will always stack-allocate that simply is never a concern.

Decision:

Working group goes with option 3. Collection expressions targeted to ref-struct types are always stack allocated. Note that this covers more than just Span<T>/ReadOnlySpan<T>. For example, if one had the following:

[CollectionBuilder(FrugalListBuilder)]
ref struct FrugalList<T>
{
}

static class FrugalListBuilder
{
    public static FrugalList<T> Create(ReadOnlySpan<T> values);
}

// ...
FrugalList<int> f = [a, b, c]; // Stack-allocated, local scope.

We believe that this is the semantics ref-struct-collections will want. And, similar to spans, the semantics users would expect to get without having to annotate with scoped on the local variable.

Blittable Data

The above discussion applied to collection expressions, target typed to ref-structs, without constant, blittable data in it. Blittable data is effectively defined as C# builtin of 1-8 bytes in size. Intuitively, the primitive integers, floating points, and characters.

Today, the language already allows optimizing the following:

private static ReadOnlySpan<char> s_chars = new char[] { 'a', 'b', 'c' };

Despite this explicitly having a new in the code, and explicitly referencing the char[] type, the above will be compiled without any allocations or array types. Instead, the data will be embedded directly into the data segment of the compiled assembly, and the span will be created pointing directly at that raw data.

We will have those rules apply when using a collection expression as well. This means that the following will be legal and will have the same behavior at runtime:

private static ReadOnlySpan<char> s_chars = ['a', 'b', 'c'];

In other words, if a collection literal has nothing but constant blittable data, and it is target typed to a ReadOnlySpan<BlittableType> only, then it will actually be considered to have global not local scope. This means that the following would be allowed:

ReadOnlySpan<int> x = [1, 2, 3]; // pointer to data segment, global scope.
return x;

No decisions were needed on this. Working group just affirmed these are the desired semantics and these rules and optimizations will be maintained with collection expressions.