Test Plan for Span<T>, aka interior pointer, aka stackonly struct

[gafter]

This is a test plan for the feature set variously known as "stackonly structs", "interior pointer", and "Span" targeting milestone 15.6 (C# 7.2), which is related to (but different from) slicing.

See also

• Championed feature at Champion "Span<T>, aka interior pointer, aka stackonly struct" (C# 7.2) csharplang#666
• Proposed safety rules at https://github.com/dotnet/csharplang/blob/master/proposals/csharp-7.2/span-safety.md
• Test plan for ref readonly at Test plan for "ref readonly" and related features (7.2) #19216
• Test plan suggestions at https://github.com/dotnet/roslyn/blob/master/docs/contributing/Compiler%20Test%20Plan.md

Declarations and types

ref struct declarations

• The parser accepts ref as a modifier on a struct declaration
• The ref modifier must appear immediately before the struct keyword (no intervening public)
• A ref struct type cannot be partial (neither ref partial struct nor partial ref struct are accepted)
• A ref struct type may be generic
• A ref struct type may contain an instance field of another ref struct type
• Neither a non-ref struct, nor a class, may contain a field of a ref struct type
• A ref struct type declaration may not contain a base clause. It therefore cannot be declared to extend any interface type.
• It is not legal to use base in the body of any method, property, etc in a ref struct type.
• A ref struct type declaration may not contain an iterator instance method.
• A ref struct type declaration may not contain an async instance method.
• A ref struct type declaration may contain a static iterator method, and a static async method.
• (NoPia) A ref struct type may not be embedded.

use of ref struct types

• A ref struct type may not be used as a type argument to a class, struct, delegate, or method.
  • It is an error to construct System.Nullable with a type argument of a ref struct type, even implicitly.
  • If an instance method e.M returns a value of a ref struct type, e?.M() is an error.
  • If type inference infers a ref struct type as a type argument, an error is given.
• An instance method declared in a ref struct type may not be converted to a delegate
• An instance method inherited into a ref struct type (e.g. GetHashCode) may not be converted to a delegate
• A static field may not be of a ref struct type (even if the container is a ref struct type)
• It is an error to use a ref struct type as the element type of an array type
• An anonymous type may not contain an element of a ref struct type.
• A tuple type may not contain an element of a ref struct type.
• It is an error if a tuple literal's natural type contains an element of a ref struct type, e.g. (1, span)
• It is not an error if a tuple literal, e.g., (null, span) has no natural type and is converted to a tuple type that does not contain an element of a ref struct type (e.g. by the use of a user-defined conversion), e.g. (string, int) x = (null, span); where the span's type contains a conversion to int.
• A Deconstruct method may output one or more values of a ref struct type.
  • Such a value can be stored in a local variable via a deconstruction statement.
  • However, it is an error to use such a deconstruction expression elsewhere.
• A set of ref struct types can be written to implement the foreach pattern, in which case the foreach statement should work with an iterator variable of a ref struct type.
• A set of ref struct types can be written to implement the foreach pattern with an element type of a ref struct type that contains a Deconstruct method, in which case the foreach statement should work with a pair of iterator variables of ref struct types.
• Neither a parameter nor local variable of ref struct type may be captured in a lambda or local function.
• Neither a parameter nor local variable of ref struct type may be declared in an iterator or async method.
• A value of ref struct type may not require spilling in async code, e.g. it is an error (possibly reported during emit) to compile an expression such as M(span1, await e).
• No boxing conversion exists from a ref struct type to object or ValueType.
• No unboxing conversion exists to a ref struct type
• No instance method declared in object or in System.ValueType but not overridden in a ref struct type may be called with a receiver of that ref struct type.
• A user-declared conversion may convert to or from a ref struct type.
  • Such a user-declared conversion does not have a lifted form in which the ref struct type is nullable.
• A user-declared operator may accept a parameter of ref struct type.
  • Such a user-declared operator does not have a lifted form in which the ref struct type is nullable.
    • test with one operand a ref struct type (left, and right)
    • test with both operands a ref struct type
• A local function may contain parameters and local variables of ref struct types.
  • Unless the local function is async or an iterator method.
• (extra credit) A set of ref struct types, delegate types, and methods can be declared which permit the use of Linq with a sequence of value of a ref struct type represented by, e.g., Span<T>.
• A ref struct type cannot be awaited, even if it is otherwise task-like. (is this right?)

Restrictions on stackalloc

Tests should demonstrate that stackalloc can only be used

1. As the initializer of a local variable declaration; or
2. As either the second or third operand of a ternary operator which is in one of these contexts.

• stackalloc works in an initializer
• stackalloc works in a ternary in an initializer
• stackalloc works in a nested ternary in an initializer
• stackalloc does not work as an operand to a method invocation, e.g. M(stackalloc int[1])
• there may be a user-defined conversion between the stackalloc result and the variable's type.
  • when the converted-from type is a pointer type
  • when the converted-from type is a ref struct type
• stackalloc is an error if parenthesized
• stackalloc is an error if either operand of ??
• stackalloc is an error if subject to an explicit cast, e.g. var x = (myspan)stackalloc int[10];
  • Alternatively, change the spec to permit it and test it, including beneath a ternary operator (allowed and tested)
• The statement var x = stackalloc int[10]; is an error unless in an unsafe context, because x is of type int*.
• The statement Span<int> x = stackalloc int[10]; is not an error outside an unsafe context.
• The statement T x = stackalloc int[10]; is an error outside an unsafe context if it required a user-defined conversion from int* to T
• The statement T x = stackalloc int[10]; is permitted outside an unsafe context if it required a user-defined conversion from Span<int> to T
• TODO: We should have LDM confirmation of the above stackalloc restrictions regarding where it may appear.

Tests should demonstrate that the compiler rejects an attempt to use stackalloc (#21918)

• In a catch block (with or without a catch parameter).
• In an exception filter (due to constraints on the localloc instruction). [This cannot occur if stackalloc is restricted to local variable initializers]
• In a finally block.

Miscellaneous

• The C# 7 compiler prevents the use of a ref struct type from metadata
• The C# 5 compiler prevents the use of a ref struct type from metadata (hand verify)
• A warning is given when a ref struct type is declared with an explicit [Obsolete] attribute
• Demonstrate that a ref struct type acts as such in separate compilation scenarios
  • through a metadata reference
  • through a compilation reference
• The use of a ref struct type from VB produces a compile-time error.
• There should be some public API (may be an extension method) that can be used to determine that a given ITypeSymbol or TypeSymbol is a ref struct type.
• A ref struct value cannot be used in an expression tree, even as an intermediate result. (?)
• There is no conversion involving a ref struct type, either to or from dynamic.
• A value of ref struct type may be used in an object, collection, or dictionary initializer for a type whose API was designed to permit this.
• An extension method may operate on a this parameter of a ref struct type.
• A value of ref struct type may not be used as a fill-in in string interpolation due to the need to box the value.

APIs

• In a statement of the form T1 x = stackalloc int[10];, the semantic model should report that the stackalloc expression has the type int* and a converted type of T1
  • When T1 is int*
  • When there is a user-defined conversion from int* to T1
  • When there is a user-defined conversion from Span<int> to T1.
  • For each of these, the conversion-from-expression from stackalloc int[10] appears as some appropriate kind of conversion.
• A stackalloc expression in any other syntactic context than a local variable initializer context has the type Span<T>.
  • under a ternary operator
  • under a nested ternary operator
  • when subjected to a cast expression, if that is permitted

Escape safety rules

Each test bullet below of the form "Show that x is (ref-)safe-to-escape y but/and no further.", this is intended to require a test that demonstrates that x is (ref-)safe-to-escape to y, and a separate test that demonstrates that x is not (ref-)safe-to-escape the enclosing scope of y.

Parameters

• Show that a parameter of a ref struct type is ref-safe-to-escape to the top level of a method, but no further, no matter the ref mode of the parameter.
• Show that a ref, in, or out parameter that is not of a ref struct type is ref-safe-to-escape from the entire method.
• Show that the this parameter of a struct type that is not a ref struct type is ref-safe-to-escape to the top level of a method, but no further.
• Show that a value parameter that is not a ref struct type is ref-safe-to-escape to the top level of a method, but no further.
• Show that a (reference to a) parameter of ref struct type is safe-to-escape (by value) from the entire method. Show this for the this parameter as well.

Locals

• Show that a non-ref local is ref-safe-to-escape to the scope in which it was declared, but no further (for a ref struct type and any other type).
• Show that a ref local variable declaration requires an initializer.
• Show that a local variable of a ref struct type does not require an initializer, in which case it is safe to return. TODO: The spec needs to say this
• Show that a ref local is ref-safe-to-escape to the same ref-safe-to-escape as its initializer, but no further.
• Show that an rvalue that is a use of a local whose type is a ref struct type that is declared as the iterator variable of a foreach loop is safe-to-escape the same as the loop's expression, and no further.
• Show that an rvalue that is a use of a local whose type is a ref struct type that is declared in a local variable declaration is safe-to-escape the same as the variable's initializer, and no further.
• It is an error if a ref struct type is explicitly the type of a pattern variable.
• It is an error if a ref struct type is implicitly the type of a pattern variable.
• Show that out variables are safe to return.
• Show that out variables participate in the no mixing rule.
• Show that a top-level variable in a script may not be of a ref struct type.
  • Even when the result of deconstruction or pattern-matching.
• Show that a nested variable in a script may be of a ref struct type.
• TODO: Are there other interesting contexts in which local variables can be declared? What about deconstruction?

2017-09-15 Review of the test plan stopped here

Fields

• Show that if e is a reference type, that e.F is ref-safe-to-escape from the entire method.
• Show that if e is a value type, its ref-safe-to-escape is the same as the ref-safe-to-escape of e, but no further.
• Show that if e.F is a ref struct type, it is safe-to-escape the same as the safe-to-escape of e, but no further.

Multi-operand expression forms

• Show that given an expression of the form c ? e1 : e2 where the result is a ref struct type, that the safe-to-escape of the result is the narrowest among the safe-to-escape of e1 and e2.
  • When e1 is a ref struct type but e2 is not (the safe-to-escape is taken from e1)
  • When e2 is a ref struct type but e1 is not (the safe-to-escape is taken from e2)
  • When e1 and e2 are ref struct types (the safe-to-escape is the smallest of the two)
  • When neither e1 nor e2 are ref struct types (the result is safe to return)
• For an expression of the form c ? ref e1 : ref e2,
  • Show that the compiler requires that the ref-safe-to-escape of e1 and e2 agree
  • Show that the ref-safe-to-escape* of the result is the same as the *ref-safe-to-escape* of e1`, but no further.

Method invocation

• Show that an lvalue resulting from a ref-returning method invocation e1.M(e2, ...) is ref-safe-to-escape the smallest of the following scopes (but no further) (a pair of positive/negative tests for each of these)
  • The entire method, if no other rule below applies
  • The ref-safe-to-escape of all ref and out argument expressions
    • excluding the receiver
    • excluding arguments of ref struct types
  • For an in parameter for which there is a corresponding expression that is an lvalue, its ref-safe-to-escape
  • For an in parameter for which there is a no corresponding expression, the immediately enclosing scope
  • The safe-to-escape of all argument expressions
    • including the receiver
• Show that a value resulting from a ref struct returning method invocation e1.M(e2, ...) is safe-to-escape the smallest of the following scopes (but no further) (a pair of positive/negative tests for each of these)
  • The entire method, if no other rule below applies
  • The ref-safe-to-escape of all ref and out argument expressions
    • excluding the receiver
    • excluding arguments of ref struct types
  • For an in parameter for which there is a corresponding expression that is an lvalue, its ref-safe-to-escape
  • For an in parameter for which there is a no corresponding expression, the immediately enclosing scope
  • The safe-to-escape of all argument expressions
    • including the receiver
• TODO: we should have a rule and tests for an argument expression of the form ref d.F where d is of type dynamic
• The same rules apply to a user-defined operator invocation. Specifically, show that given an expression of the form e1 + e2 where the result is a ref struct type, that the safe-to-escape of the result is the narrowest among the safe-to-escape of e1 and e2.
  • When e1 is a ref struct type but e2 is not (the safe-to-escape is taken from e1)
  • When e2 is a ref struct type but e1 is not (the safe-to-escape is taken from e2)
  • When e1 and e2 are ref struct types (the safe-to-escape is the smallest of the two)
  • When neither e1 nor e2 are ref struct types (the result is safe to return)
• A property invocation that returns a ref result follows the method invocation rules: both the ref-safe-to-escape is taken from the safe-to-escape of the receiver.
• A property invocation that returns a result of ref struct type follows the method invocation rules: the safe-to-escape is taken from the safe-to-escape of the receiver.
• A constructor invocation acts as a method invocation without a receiver; for a constructor of a ref struct type, its result is safe-to-escape the smallest of the following scopes (but no further) (a pair of positive/negative tests for each of these)
  • The entire method, if no other rule below applies
  • The ref-safe-to-escape of all ref and out argument expressions
    • excluding the receiver
    • excluding arguments of ref struct types
  • For an in parameter for which there is a corresponding expression that is an lvalue, its ref-safe-to-escape
  • For an in parameter for which there is a no corresponding expression, the immediately enclosing scope
  • The safe-to-escape of all argument expressions
• In a statement such as return ref await e;, where the type of e is a custom value task whose GetResult() method is ref-returning, we should be returning the returned ref, not a ref to a copy of its value. In particular, if the returned ref is ref-safe-to-return, then there should be no error.

stackalloc

• a stackalloc expression is safe-to-escape to the top level of the method, but no further.

default

• A default or default(T) expression is safe-to-escape from the entire enclosing method (i.e. it is safe to return)

Other

• A default value of a ref struct type is safe to return (i.e. escape from the whole method).

Constraints in expressions

• (Only if ref reassignment is supported) For a ref reassignment ref e1 = ref e2, the ref-safe-to-escape of e2 must be at least as wide a scope as the ref-safe-to-escape of e1.
  • Alternately, demonstrate that ref reassignment is not supported.
• For a ref return statement return ref e1, the ref-safe-to-escape of e1 must be ref-safe-to-escape from the entire method.
• For a return statement return e1, the safe-to-escape of e1 must be safe-to-escape from the entire method.
• For an assignment e1 = e2, if the type of e1 is a ref struct type, then the safe-to-escape of e2 must be at least as wide a scope as the safe-to-escape of e1.
• In a method invocation, the following constraints apply (negative tests required):
  • If there is a ref or out argument to a ref struct type (including the receiver), with safe-to-escape E1, then
    • no ref or out argument (excluding the receiver and arguments of ref struct types) may have a narrower ref-safe-to-escape than E1; and
    • no argument (including the receiver) may have a narrower safe-to-escape than E1.

[jcouv]

• Should deconstruction with Span be allowed (void Deconstruct(out Span<int> x, out Span<int> y))? If yes, we need to solve Surprising System.ValueTuple requirement for deconstruction #18629 first, since all deconstructions try to make a return type, but ValueTuple<Span, Span> is not allowed.

[Note from @gafter: Issue #18629 has been solved]

[gafter]

@jaredpar I see that you are the test and review resource for this feature. It wasn't clear to me how much of that you've delegated. Please let me know if I am on the hook for further work aside from the open issues in the draft spec.

[Note from @gafter: I have been assigned the test resource for this.]

[benaadams]

For ref reassignment will this work with non ref structs; e.g. array elements

Pattern I've wanted to express (poor pseudo code):

ref var e1 = ref _array[0];
while(el.NotCorrect)
{
    if ((uint)e1.Next >= (uint)_array.Length)
    {
       break;
    }

    ref el = ref _array[e1.Next];
}

if (el.NotCorrect)
{
   var oldLength = _array.Length;
    _array = new var[oldLength  * 2];
   // *** reassignment here
    ref e1 = ref _array[oldLength];
}

// Do something with el
// Rather than create a second array lookup here
// ref var e2 = ref _array[index];

You can do the general reassignment in the loop body (by the ref being local to loop); however then you need to pass the int indexer out and do a second array lookup to work with it; when you have already found the ref in the loop.

[gafter]

@benaadams Yes, once we add support for ref reassignment that would work. But we are not adding ref reassignment in 7.2.

[gafter]

I've added a championed proposal for ref reassignment. dotnet/csharplang#933

[jcouv]

@gafter @VSadov Can you two sync up on updating the status on this test plan? Thanks

[gafter]

The test plan is a gate on integrating a feature. Since this feature has been integrated, closing this test plan.

[jcouv]

That doesn't seem right. Checking 18 out of 161 defeats the purpose and seems like a joke.

If I were to choose two items to push on, I'd ask to consolidate the docs for this feature and bullets for the test plan (to act as a reminder when working on next feature).

[gafter]

@jcouv While I agree, we integrated before the developer identified tests for these bullet items, and no tests have ever been identified for those bullets. We did not follow the defined process for feature development (i.e. test plan checked off before feature integrated). But that is, in practice, what we did.

I'll reopen and assign to the developer to enact your suggestion.

[jcouv]

Vlad updated the docs: https://github.com/dotnet/csharplang/blob/master/proposals/csharp-7.2/readonly-ref.md

We still need to update the test plan.

[Nukepayload2]

The use of a ref struct type from VB produces a compile-time error.

@gafter
Could you please explain why we should not use ref struct in VB? I'd like to suggest modify this item to "The use of a ref struct type from VB should be checked with the same rules as verifying the use of ref struct in c#".

[gafter]

This feature has long since shipped.