New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal: Ref Returns and Locals #118

Closed
stephentoub opened this Issue Jan 28, 2015 · 167 comments

Comments

@stephentoub
Member

stephentoub commented Jan 28, 2015

(Note: this proposal was briefly discussed in #98, the C# design notes for Jan 21, 2015. It has not been updated based on the discussion that's already occurred on that thread.)

Background

Since the first release of C#, the language has supported passing parameters by reference using the 'ref' keyword, This is built on top of direct support in the runtime for passing parameters by reference.

Problem

Interestingly, that support in the CLR is actually a more general mechanism for passing around safe references to heap memory and stack locations; that could be used to implement support for ref return values and ref locals, but C# historically has not provided any mechanism for doing this in safe code. Instead, developers that want to pass around structured blocks of memory are often forced to do so with pointers to pinned memory, which is both unsafe and often inefficient.

Solution: ref returns

The language should support the ability to declare ref locals and ref return values. We could, for example, now declare a function like the following, which not only accepts 'ref' parameters but which also has a ref return value:

public static ref TValue Choose<TValue>(
    Func<bool> condition, ref TValue left, ref TValue right)
{
    return condition() ? ref left : ref right;
}

With a method like that, one can now write code that passes two values by reference, with one of them being returned based on some condition:

Matrix3D left = …, right = …;
Choose(chooser, ref left, ref right).M20 = 1.0;

Based on the function that gets passed in here, a reference to either 'left' or 'right' will be returned, and the M20 field of it will be set. Since we’re trading in references, the value contained in either 'left' or 'right' is updated, rather than a temporary copy being updated, and rather than needing to pass around big structures, necessitating big copies.

If we don't want the returned reference to be writable, we could apply 'readonly' just as we were able to do earlier with ‘ref’ on parameters (extending the proposal mentioned in #115 to also support return refs):

public static readonly ref TValue Choose<TValue>(
    Func<bool> condition, ref TValue left, ref TValue right)
{
    return condition() ? ref left : ref right;
}
…
Matrix3D left = …, right = …;
Choose(chooser, ref left, ref right) = new Matrix3D(...); // Error: returned reference is read-only

Note that when referencing the 'left' and 'right' ref arguments in the Choose method’s implementation, we used the 'ref' keyword. This would be required by the language, just as it’s required to use the ‘ref’ keyword when passing a value to a 'ref' parameter.

Solution: ref locals

Once you have the ability to receive 'ref' parameters and to return ‘ref’ return values, it’s very handy to be able to define 'ref' locals as well. A 'ref' local can be set to anything that’s safe to return as a 'ref' return, which includes references to variables on the heap, 'ref' parameters, 'ref' values returned from a call to another method where all 'ref' arguments to that method were safe to return, and other 'ref' locals.

public static ref int Max(ref int first, ref int second, ref int third)
{
    ref int max = first > second ? ref first : ref second;
    return max > third ? ref max : ref third;
}
…
int a = 1, b = 2, c = 3;
Max(ref a, ref b, ref c) = 4;
Debug.Assert(a == 1); // true
Debug.Assert(b == 2); // true
Debug.Assert(c == 4); // true

We could also use ‘readonly’ with ref on locals (again, see #115), to ensure that the ref variables don’t change. This would work not only with ref parameters, but also with ref locals and ref returns:

public static readonly ref int Max(
    readonly ref int first, readonly ref int second, readonly ref int third)
{
    readonly ref int max = first > second ? ref first : ref second;
    return max > third ? ref max : ref third;
}

@stephentoub stephentoub changed the title from Proposal: ref returns and locals to Proposal: Ref Returns and Locals Jan 28, 2015

@theoy theoy added the Language-C# label Jan 28, 2015

@gafter gafter added the 1 - Planning label Feb 2, 2015

@MgSam

This comment has been minimized.

Show comment
Hide comment
@MgSam

MgSam Feb 4, 2015

If I recall, Eric Lippert blogged about this some years back and the response in the comments was largely negative.

I do not like this feature for C#. The resulting code is like an uglier version of C++, and code written with it takes longer to reason about and understand. The use-cases are not particularly compelling, and I have never run into a situation where I wished I had ref locals or return values.

MgSam commented Feb 4, 2015

If I recall, Eric Lippert blogged about this some years back and the response in the comments was largely negative.

I do not like this feature for C#. The resulting code is like an uglier version of C++, and code written with it takes longer to reason about and understand. The use-cases are not particularly compelling, and I have never run into a situation where I wished I had ref locals or return values.

@axel-habermaier

This comment has been minimized.

Show comment
Hide comment
@axel-habermaier

axel-habermaier Feb 4, 2015

Contributor

Yes, I know very well that mutable structs should be avoided. Still, one interesting use case would be lists of mutable structs. Consider:

struct MutableStruct { public int X { get; set; } }
MutableStruct[] a = ...
List<MutableStruct> l = ..
a[3].X = 5; // changes the value of X of the struct in the array
l[3].X = 5; // compile time error

If the indexer of the List<T> class would return the value stored in the list by reference, the code above would compile, making the use of mutable structs less surprising. It is probably even more efficient as the (potentially large) struct no longer has to be copied out from the list.

Unfortunately, I doubt that the return type of List<T>'s indexer can be changed for backwards compatibility reasons.

Contributor

axel-habermaier commented Feb 4, 2015

Yes, I know very well that mutable structs should be avoided. Still, one interesting use case would be lists of mutable structs. Consider:

struct MutableStruct { public int X { get; set; } }
MutableStruct[] a = ...
List<MutableStruct> l = ..
a[3].X = 5; // changes the value of X of the struct in the array
l[3].X = 5; // compile time error

If the indexer of the List<T> class would return the value stored in the list by reference, the code above would compile, making the use of mutable structs less surprising. It is probably even more efficient as the (potentially large) struct no longer has to be copied out from the list.

Unfortunately, I doubt that the return type of List<T>'s indexer can be changed for backwards compatibility reasons.

@xen2

This comment has been minimized.

Show comment
Hide comment
@xen2

xen2 Feb 7, 2015

Disclaimer: I work on game engine, so I am probably not the typical user.

One use case this could really help us is this one:

MyHugeStruct[] data; // we use a struct to improve data locality and reduce GC pressure
// Ideally, we would like to be able to use List<T>, but we can't take ref then
for (int i = 0; i < data.Length; ++i)
{
   // Option 1: make a local copy (slow)
   var item = data[i];

   // Option2: To avoid making a stack copy of MyHugeStruct,
   // we have to defer to a inner loop function
   MyLoopBody(ref data[i]);

   // Option3: using new proposal, that would be much better:
   ref MyHugeStruct = data[i];
}

We end up making separate function for loop body, and in case of tight loop this can end up being quite bad:

  • Have to forward all parameters
  • Sometimes we found out with VTune that inner loop stack "initlocals" was taking up most (80%+) of the time if inner loop body happened to have a several locals (even if only 0 or 1 was used due to branching). This would not happened if the locals were contained and memzeroed once in the function containing the "for" loop.
  • not inlined in simple cases

Nice to have:

  • ref this[] operator(?) so that List<> and other collections can be used (vs being forced to use arrays)
  • a ++ operator on ref to be able to loop by incrementing pointer instead of indice multiplication (but probably unsafe).

Extra (probably impossible without changing BCL):

  • Lot of struct copy could also be avoided in EqualityComparer (Dictionary) if ref could be used when large structs are being used as key.

xen2 commented Feb 7, 2015

Disclaimer: I work on game engine, so I am probably not the typical user.

One use case this could really help us is this one:

MyHugeStruct[] data; // we use a struct to improve data locality and reduce GC pressure
// Ideally, we would like to be able to use List<T>, but we can't take ref then
for (int i = 0; i < data.Length; ++i)
{
   // Option 1: make a local copy (slow)
   var item = data[i];

   // Option2: To avoid making a stack copy of MyHugeStruct,
   // we have to defer to a inner loop function
   MyLoopBody(ref data[i]);

   // Option3: using new proposal, that would be much better:
   ref MyHugeStruct = data[i];
}

We end up making separate function for loop body, and in case of tight loop this can end up being quite bad:

  • Have to forward all parameters
  • Sometimes we found out with VTune that inner loop stack "initlocals" was taking up most (80%+) of the time if inner loop body happened to have a several locals (even if only 0 or 1 was used due to branching). This would not happened if the locals were contained and memzeroed once in the function containing the "for" loop.
  • not inlined in simple cases

Nice to have:

  • ref this[] operator(?) so that List<> and other collections can be used (vs being forced to use arrays)
  • a ++ operator on ref to be able to loop by incrementing pointer instead of indice multiplication (but probably unsafe).

Extra (probably impossible without changing BCL):

  • Lot of struct copy could also be avoided in EqualityComparer (Dictionary) if ref could be used when large structs are being used as key.
@paulomorgado

This comment has been minimized.

Show comment
Hide comment
@paulomorgado

paulomorgado Mar 6, 2015

What happens with this?

var data = GetData();
...
ref SomeStruct GetData()
{
    var ss1 = new SomeStruct();
    var ss2 = new SomeStruct();

    return ref Choose(ref ss1, ref ss2);
}
ref SomeStruct Choose(ref SomeStruct ss1, ref SomeStruct ss2)
{
    return whatever ? ref ss1 : ref ss2;
}

GetData might not be aware that Choose is returning one of its variables and returns to the caller a reference to it.

Does the value still exist after exiting GetData?

What happens with this?

var data = GetData();
...
ref SomeStruct GetData()
{
    var ss1 = new SomeStruct();
    var ss2 = new SomeStruct();

    return ref Choose(ref ss1, ref ss2);
}
ref SomeStruct Choose(ref SomeStruct ss1, ref SomeStruct ss2)
{
    return whatever ? ref ss1 : ref ss2;
}

GetData might not be aware that Choose is returning one of its variables and returns to the caller a reference to it.

Does the value still exist after exiting GetData?

@gafter

This comment has been minimized.

Show comment
Hide comment
@gafter

gafter Mar 6, 2015

Member

@paulomorgado You would not be allowed to return a ref to a local variable or parameter.

Member

gafter commented Mar 6, 2015

@paulomorgado You would not be allowed to return a ref to a local variable or parameter.

@paulomorgado

This comment has been minimized.

Show comment
Hide comment
@paulomorgado

paulomorgado Mar 6, 2015

@gafter, the only difference between my Choose method and @stephentoub's one is that mine does not have the selector passed as a delegate. Did I miss something here?

@gafter, the only difference between my Choose method and @stephentoub's one is that mine does not have the selector passed as a delegate. Did I miss something here?

@stephentoub

This comment has been minimized.

Show comment
Hide comment
@stephentoub

stephentoub Mar 6, 2015

Member

@paulomorgado, the compiler would only let you return a ref to something that it knew was either on the heap or that came from the caller. In my example, the ref inputs to the Choose method were all from ref parameters (or ref locals to ref parameters), so the compiler would conclude that the result of the Choose method met the criteria and would allow its returned ref to be returned. But in your example, the refs passed to Choose were not from the caller nor from the heap, such that the compiler couldn't be sure that the result of Choose was allowed to be returned, and it would error out.

Member

stephentoub commented Mar 6, 2015

@paulomorgado, the compiler would only let you return a ref to something that it knew was either on the heap or that came from the caller. In my example, the ref inputs to the Choose method were all from ref parameters (or ref locals to ref parameters), so the compiler would conclude that the result of the Choose method met the criteria and would allow its returned ref to be returned. But in your example, the refs passed to Choose were not from the caller nor from the heap, such that the compiler couldn't be sure that the result of Choose was allowed to be returned, and it would error out.

@paulomorgado

This comment has been minimized.

Show comment
Hide comment
@paulomorgado

paulomorgado Mar 6, 2015

@stephentoub, forget my Choose method. Your's is the best that can be done and you just published it to NuGet and I added it to my project. How can the compiler know where the return valur of Choose is coming from? My GetData is just complying to the contract of Choose to get its result and pass along as all the code written so far and to be written in the future does.

What you're saying is that publicly exposed methods can't return refs, which reduce the usage to only private methods.

@stephentoub, forget my Choose method. Your's is the best that can be done and you just published it to NuGet and I added it to my project. How can the compiler know where the return valur of Choose is coming from? My GetData is just complying to the contract of Choose to get its result and pass along as all the code written so far and to be written in the future does.

What you're saying is that publicly exposed methods can't return refs, which reduce the usage to only private methods.

@stephentoub

This comment has been minimized.

Show comment
Hide comment
@stephentoub

stephentoub Mar 6, 2015

Member

@paulomorgado, I understand the confusion, but that's not what I'm saying.

There would be some rules about what it would be safe to return, e.g.

  • refs to variables on the heap are safe to return
  • ref and out parameters are safe to return
  • a ref returned from another method is safe to return if all refs passed to that method were safe to return (by this same set of rules)

Forget the implementation of Choose here. Assuming Choose abides by these rules (which the compilation of Choose would enforce), in my example all of the inputs to Choose were valid to be returned, therefore the result of Choose could be returned. In your example, at least one of the inputs to Choose wasn't valid to be returned, therefore the result of Choose could not be returned. The compiler can validate that.

Member

stephentoub commented Mar 6, 2015

@paulomorgado, I understand the confusion, but that's not what I'm saying.

There would be some rules about what it would be safe to return, e.g.

  • refs to variables on the heap are safe to return
  • ref and out parameters are safe to return
  • a ref returned from another method is safe to return if all refs passed to that method were safe to return (by this same set of rules)

Forget the implementation of Choose here. Assuming Choose abides by these rules (which the compilation of Choose would enforce), in my example all of the inputs to Choose were valid to be returned, therefore the result of Choose could be returned. In your example, at least one of the inputs to Choose wasn't valid to be returned, therefore the result of Choose could not be returned. The compiler can validate that.

@paulomorgado

This comment has been minimized.

Show comment
Hide comment
@paulomorgado

paulomorgado Mar 8, 2015

@stephentoub, what I'm having trouble with is understanding how those rules can be effectively enforced.

And a proposal should have an example that works under the proposal.

@stephentoub, what I'm having trouble with is understanding how those rules can be effectively enforced.

And a proposal should have an example that works under the proposal.

@stephentoub

This comment has been minimized.

Show comment
Hide comment
@stephentoub

stephentoub Mar 8, 2015

Member

@paulomorgado, how does my example not work under the proposal? And why do you believe the rules can't be enforced?

Member

stephentoub commented Mar 8, 2015

@paulomorgado, how does my example not work under the proposal? And why do you believe the rules can't be enforced?

@paulomorgado

This comment has been minimized.

Show comment
Hide comment
@paulomorgado

paulomorgado Mar 8, 2015

@stephentoub, either that or I totally missed everything.

My understanding is that there's no way the caller can take the result of your Choose method as safe to return as reference. Is there? If so, how?

@stephentoub, either that or I totally missed everything.

My understanding is that there's no way the caller can take the result of your Choose method as safe to return as reference. Is there? If so, how?

@stephentoub

This comment has been minimized.

Show comment
Hide comment
@stephentoub

stephentoub Mar 8, 2015

Member

@paulomorgado, in this example:

public static ref TValue Choose<TValue>(
    Func<bool> condition, ref TValue left, ref TValue right)
{
    return condition() ? ref left : ref right;
}

left and right are both safe to return because they came from the caller.

In this example:

public static ref int Max(ref int first, ref int second, ref int third)
{
    ref int max = first > second ? ref first : ref second;
    return max > third ? ref max : ref third;
}

first, second, and third are all safe to return because they all came from the caller. max is safe to return because the only refs it's possibly assigned to are those which are safe to return.

If I as a caller wanted to use Choose, e.g.

public static ref TValue ChooseByTime<TValue>(
    ref TValue left, ref TValue right)
{
    return Choose(() => DateTime.UtcNow.Seconds % 2 == 0, ref left, ref right);
}

Both left and right are safe to return because they came from the caller. Therefore all of the ref inputs to Choose are safe to return. Therefore the resulting ref from Choose is also safe to return. I don't need to worry about the implementation of Choose, because the compiler is enforcing all of these same rules on the implementation of Choose.

Member

stephentoub commented Mar 8, 2015

@paulomorgado, in this example:

public static ref TValue Choose<TValue>(
    Func<bool> condition, ref TValue left, ref TValue right)
{
    return condition() ? ref left : ref right;
}

left and right are both safe to return because they came from the caller.

In this example:

public static ref int Max(ref int first, ref int second, ref int third)
{
    ref int max = first > second ? ref first : ref second;
    return max > third ? ref max : ref third;
}

first, second, and third are all safe to return because they all came from the caller. max is safe to return because the only refs it's possibly assigned to are those which are safe to return.

If I as a caller wanted to use Choose, e.g.

public static ref TValue ChooseByTime<TValue>(
    ref TValue left, ref TValue right)
{
    return Choose(() => DateTime.UtcNow.Seconds % 2 == 0, ref left, ref right);
}

Both left and right are safe to return because they came from the caller. Therefore all of the ref inputs to Choose are safe to return. Therefore the resulting ref from Choose is also safe to return. I don't need to worry about the implementation of Choose, because the compiler is enforcing all of these same rules on the implementation of Choose.

@paulomorgado

This comment has been minimized.

Show comment
Hide comment
@paulomorgado

paulomorgado Mar 8, 2015

Both left and right are safe to return because they came from the caller. Therefore all of the ref inputs to Choose are safe to return. Therefore the resulting ref from Choose is also safe to return. I don't need to worry about the implementation of Choose, because the compiler is enforcing all of these same rules on the implementation of Choose.

But ChooseByTime isn't returning neither left nor right. It's returning the return value of Choose. Noting but the implementation details of Choose is saying its return value is the same as one of its parameters. What if Choose is an implementation of an interface?

You're restricting the use of Choose to cases where it works without any safeguards or proof that it's safe.

My example shows the opposite.

Both left and right are safe to return because they came from the caller. Therefore all of the ref inputs to Choose are safe to return. Therefore the resulting ref from Choose is also safe to return. I don't need to worry about the implementation of Choose, because the compiler is enforcing all of these same rules on the implementation of Choose.

But ChooseByTime isn't returning neither left nor right. It's returning the return value of Choose. Noting but the implementation details of Choose is saying its return value is the same as one of its parameters. What if Choose is an implementation of an interface?

You're restricting the use of Choose to cases where it works without any safeguards or proof that it's safe.

My example shows the opposite.

@stephentoub

This comment has been minimized.

Show comment
Hide comment
@stephentoub

stephentoub Mar 8, 2015

Member

@paulomorgado, your example wouldn't compile... the compiler would error out exactly because it doesn't abide by the rules: your call to Choose is passed ref values that are not safe to return, therefore the result of your call to Choose is not safe to return. I'm sorry if I'm not explaining this well; not sure how to convey it differently.

Member

stephentoub commented Mar 8, 2015

@paulomorgado, your example wouldn't compile... the compiler would error out exactly because it doesn't abide by the rules: your call to Choose is passed ref values that are not safe to return, therefore the result of your call to Choose is not safe to return. I'm sorry if I'm not explaining this well; not sure how to convey it differently.

@stephentoub

This comment has been minimized.

Show comment
Hide comment
@stephentoub

stephentoub Mar 9, 2015

Member

Noting but the implementation details of Choose is saying its return value is the same as one of its parameters.

Ah, maybe this is the point of confusion. The implementation doesn't matter because the compiler assumes the worst: regardless of how a parameter is actually used, if any argument isn't safe to return, then the result of the call isn't safe to return. The compiler is conservative in that regard.

Member

stephentoub commented Mar 9, 2015

Noting but the implementation details of Choose is saying its return value is the same as one of its parameters.

Ah, maybe this is the point of confusion. The implementation doesn't matter because the compiler assumes the worst: regardless of how a parameter is actually used, if any argument isn't safe to return, then the result of the call isn't safe to return. The compiler is conservative in that regard.

@paulomorgado

This comment has been minimized.

Show comment
Hide comment
@paulomorgado

paulomorgado Mar 9, 2015

A conservative compiler that assumes the worst cannot assume the return value of Choose is safe to return.

Is this what you're proposing?

public static ref TValue ChooseByTime<TValue>(
    ref TValue left, ref TValue right)
{
    TValue result = Choose(() => DateTime.UtcNow.Seconds % 2 == 0, ref left, ref right);
    if (result == left) reurn ref left;
    else if (result == right) return ref right;
    else throw new Exception("Invalid value.");
}

A conservative compiler that assumes the worst cannot assume the return value of Choose is safe to return.

Is this what you're proposing?

public static ref TValue ChooseByTime<TValue>(
    ref TValue left, ref TValue right)
{
    TValue result = Choose(() => DateTime.UtcNow.Seconds % 2 == 0, ref left, ref right);
    if (result == left) reurn ref left;
    else if (result == right) return ref right;
    else throw new Exception("Invalid value.");
}
@stephentoub

This comment has been minimized.

Show comment
Hide comment
@stephentoub

stephentoub Mar 9, 2015

Member

Why do you say that? What specifically about this example do you believe is problematic?

Member

stephentoub commented Mar 9, 2015

Why do you say that? What specifically about this example do you believe is problematic?

@stephentoub

This comment has been minimized.

Show comment
Hide comment
@stephentoub

stephentoub Mar 9, 2015

Member

Let's try something else: can you construct an implementation of Choose that will compile based on the aforementioned rules/explanations but where the caller of the method could not assume its return value was safe to return?

Member

stephentoub commented Mar 9, 2015

Let's try something else: can you construct an implementation of Choose that will compile based on the aforementioned rules/explanations but where the caller of the method could not assume its return value was safe to return?

@paulomorgado

This comment has been minimized.

Show comment
Hide comment
@paulomorgado

paulomorgado Mar 9, 2015

No I can't. Because I haven't been able to understand how this would work.

I can understand how, in your implementation of Choose, it is safe to return that reference.

What I can't understand is why its callers can safely return the same reference without intimately knowing its internals..

No I can't. Because I haven't been able to understand how this would work.

I can understand how, in your implementation of Choose, it is safe to return that reference.

What I can't understand is why its callers can safely return the same reference without intimately knowing its internals..

@stephentoub

This comment has been minimized.

Show comment
Hide comment
@stephentoub

stephentoub Mar 9, 2015

Member

Because it wouldn't be allowed to return anything that's not safe in the case where the caller assumes it is safe. If the only thing the caller passes in are refs that are safe to return, then what could this method return?

  • one of those refs: that's safe.
  • a ref to an object it allocates on the heap: that's safe
  • a ref to some other local or parameter: that's not safe, but it's also not allowed, so it can't actually do this
  • a ref it got back from another call, but only if it passed in safe to return refs; if it passed in any non safe refs, then the returned ref would also not be safe to return, the compiler wouldn't allow it. Effectively the rules apply recursively here.

Etc.

Member

stephentoub commented Mar 9, 2015

Because it wouldn't be allowed to return anything that's not safe in the case where the caller assumes it is safe. If the only thing the caller passes in are refs that are safe to return, then what could this method return?

  • one of those refs: that's safe.
  • a ref to an object it allocates on the heap: that's safe
  • a ref to some other local or parameter: that's not safe, but it's also not allowed, so it can't actually do this
  • a ref it got back from another call, but only if it passed in safe to return refs; if it passed in any non safe refs, then the returned ref would also not be safe to return, the compiler wouldn't allow it. Effectively the rules apply recursively here.

Etc.

@paulomorgado

This comment has been minimized.

Show comment
Hide comment
@paulomorgado

paulomorgado Mar 9, 2015

So, this wouldn't be safe, right?

public static ref TValue ChooseByTime<TValue>(
    ref TValue left)
{
    ref TValue right = default(TValue);
    return Choose(() => DateTime.UtcNow.Seconds % 2 == 0, ref left, ref right);
}

So, this wouldn't be safe, right?

public static ref TValue ChooseByTime<TValue>(
    ref TValue left)
{
    ref TValue right = default(TValue);
    return Choose(() => DateTime.UtcNow.Seconds % 2 == 0, ref left, ref right);
}
@stephentoub

This comment has been minimized.

Show comment
Hide comment
@stephentoub

stephentoub Mar 9, 2015

Member

Correct, that would not compile.

Member

stephentoub commented Mar 9, 2015

Correct, that would not compile.

@copernicus365

This comment has been minimized.

Show comment
Hide comment
@copernicus365

copernicus365 Jul 6, 2015

Beautiful solution, I've wondered why this couldn't be done before.

@MgSam [The resulting code is like an uglier version of C++]
Because of sentiments like this (i.e. 'anything I don't personally use should never be part of the language for anybody else either, even though the CLR itself has this capability'), it means our language is needlessly crippled in places where a very easy and beautiful solution like this gives us such a capability. As the gamer showed in the comment above, this can be a big performance win in some cases.

Beautiful solution, I've wondered why this couldn't be done before.

@MgSam [The resulting code is like an uglier version of C++]
Because of sentiments like this (i.e. 'anything I don't personally use should never be part of the language for anybody else either, even though the CLR itself has this capability'), it means our language is needlessly crippled in places where a very easy and beautiful solution like this gives us such a capability. As the gamer showed in the comment above, this can be a big performance win in some cases.

@whoisj

This comment has been minimized.

Show comment
Hide comment
@whoisj

whoisj Jul 7, 2015

👍

Anytime I can pass a pointer instead of performing a value copy, I'm all for it. Are there good reasons to pass memory by value-copy? Yes. Should it always be the case? Absolutely not.

The resulting code is like an uglier version of C++

I agree, it is not pretty but it is very descriptive. It would be nice if the ref keyword could be replaced with syntax we're all used to. Perhaps we could use * in place of ref because int* foo; is "cleaner" and "easier" to read than ref int foo;. I put "cleaner" and "easier" in quotes because it is incredibly subjective.

Yes, I know that * is generally reserved for unsafe but there's no reason the symbol cannot be reused, so long as one is reserved for a "safe" contexts and the other for an "unsafe" context.

whoisj commented Jul 7, 2015

👍

Anytime I can pass a pointer instead of performing a value copy, I'm all for it. Are there good reasons to pass memory by value-copy? Yes. Should it always be the case? Absolutely not.

The resulting code is like an uglier version of C++

I agree, it is not pretty but it is very descriptive. It would be nice if the ref keyword could be replaced with syntax we're all used to. Perhaps we could use * in place of ref because int* foo; is "cleaner" and "easier" to read than ref int foo;. I put "cleaner" and "easier" in quotes because it is incredibly subjective.

Yes, I know that * is generally reserved for unsafe but there's no reason the symbol cannot be reused, so long as one is reserved for a "safe" contexts and the other for an "unsafe" context.

@HaloFour

This comment has been minimized.

Show comment
Hide comment
@HaloFour

HaloFour Jul 7, 2015

Given the limitations listed above imposed to maintain a safe context I'm having a hard time envisioning the use cases for this feature. The real gains would seem to be in how structs can be used throughout the BCL with arrays, lists or other collection types.

HaloFour commented Jul 7, 2015

Given the limitations listed above imposed to maintain a safe context I'm having a hard time envisioning the use cases for this feature. The real gains would seem to be in how structs can be used throughout the BCL with arrays, lists or other collection types.

@whoisj

This comment has been minimized.

Show comment
Hide comment
@whoisj

whoisj Jul 7, 2015

Given the limitations listed above imposed to maintain a safe context I'm having a hard time envisioning the use cases for this feature. The real gains would seem to be in how structs can be used throughout the BCL with arrays, lists or other collection types.

Agreed. This is, in my opinion, a small step in the right direction though.

whoisj commented Jul 7, 2015

Given the limitations listed above imposed to maintain a safe context I'm having a hard time envisioning the use cases for this feature. The real gains would seem to be in how structs can be used throughout the BCL with arrays, lists or other collection types.

Agreed. This is, in my opinion, a small step in the right direction though.

@whoisj

This comment has been minimized.

Show comment
Hide comment
@whoisj

whoisj Jul 8, 2015

Would this implementaion allow for ref int[] intRefs = new ref int[512];?

If it doesn't, then I am less excited than I originally was. If it does, read ref struct[] is difficult. Is it a reference to an array of structures or an array of structure references?

Better to use struct*[] in my opinion.

whoisj commented Jul 8, 2015

Would this implementaion allow for ref int[] intRefs = new ref int[512];?

If it doesn't, then I am less excited than I originally was. If it does, read ref struct[] is difficult. Is it a reference to an array of structures or an array of structure references?

Better to use struct*[] in my opinion.

@jaredpar

This comment has been minimized.

Show comment
Hide comment
@jaredpar

jaredpar Sep 23, 2016

Member

@Ziflin

However, how does a readonly struct prevent the encapsulation issue of:
transform.GetPositionByRef() = position;

No. The proposal about readonly struct only refers to the ability of the struct to modify itself via instance methods. The mechanism for doing so is changing this to be typed as readonly ref T instead of the normal ref T.

The GetPositionByRef method though can control whether or not callers can assign into the returned value. Using readonly ref as the return prevents assignment irrespective of whether or not the struct itself is readonly.

Member

jaredpar commented Sep 23, 2016

@Ziflin

However, how does a readonly struct prevent the encapsulation issue of:
transform.GetPositionByRef() = position;

No. The proposal about readonly struct only refers to the ability of the struct to modify itself via instance methods. The mechanism for doing so is changing this to be typed as readonly ref T instead of the normal ref T.

The GetPositionByRef method though can control whether or not callers can assign into the returned value. Using readonly ref as the return prevents assignment irrespective of whether or not the struct itself is readonly.

@Ziflin

This comment has been minimized.

Show comment
Hide comment
@Ziflin

Ziflin Sep 23, 2016

The GetPositionByRef method though can control whether or not callers can assign into the returned value. Using readonly ref as the return prevents assignment irrespective of whether or not the struct itself is readonly.

Ok, so then there is a proposal / feature planned for returning a readonly ref? That's mostly what I've been trying to figure out. And can this be used by properties?

Ziflin commented Sep 23, 2016

The GetPositionByRef method though can control whether or not callers can assign into the returned value. Using readonly ref as the return prevents assignment irrespective of whether or not the struct itself is readonly.

Ok, so then there is a proposal / feature planned for returning a readonly ref? That's mostly what I've been trying to figure out. And can this be used by properties?

@benaadams

This comment has been minimized.

Show comment
Hide comment
@benaadams

benaadams Sep 23, 2016

Contributor

@Ziflin I think its captured in #115 though it deals with ref parameters so may need to be extended for ref returns, now they are a thing.

Contributor

benaadams commented Sep 23, 2016

@Ziflin I think its captured in #115 though it deals with ref parameters so may need to be extended for ref returns, now they are a thing.

@jaredpar

This comment has been minimized.

Show comment
Hide comment
@jaredpar

jaredpar Sep 23, 2016

Member

@Ziflin

As @benaadams pointed out, #115 has it a bit. But it does need to be extended for readonly structs and refs to be complete. It's on my list of items to write up.

Member

jaredpar commented Sep 23, 2016

@Ziflin

As @benaadams pointed out, #115 has it a bit. But it does need to be extended for readonly structs and refs to be complete. It's on my list of items to write up.

@Ziflin

This comment has been minimized.

Show comment
Hide comment
@Ziflin

Ziflin Sep 23, 2016

@jaredpar @benaadams Ok great. I'm definitely with @joeante (in #115) in his desire to see C# perform as well as it can and this seems like one of the last issues I've had with moving to C# from C++ for game engine development. I guess keeping C# as clean a language as possible with that is the hard part.

Ziflin commented Sep 23, 2016

@jaredpar @benaadams Ok great. I'm definitely with @joeante (in #115) in his desire to see C# perform as well as it can and this seems like one of the last issues I've had with moving to C# from C++ for game engine development. I guess keeping C# as clean a language as possible with that is the hard part.

@xoofx

This comment has been minimized.

Show comment
Hide comment
@xoofx

xoofx Oct 13, 2016

Member

In this case the argument is taken by ref presumably to avoid copying a large struct. However in order to execute the ToString call the compiler will fully copy the value to the stack. Oops

@jaredpar, not sure to understand why readonly ref should be interpreted as a ref readonly... Couldn't we have the ability to have a:

  • readonly ref, i.e. you can do whatever you want on the struct behind the ref, but you can't modify the ref
  • ref readonly, i.e. you cannot modify the struct behind the ref and you can't modify the ref (this would be different from a C++ const). This would allow typically to be able to pass a ref to a readonly field (something we can't do today)

I'm really missing the readonly ref behavior there...

Member

xoofx commented Oct 13, 2016

In this case the argument is taken by ref presumably to avoid copying a large struct. However in order to execute the ToString call the compiler will fully copy the value to the stack. Oops

@jaredpar, not sure to understand why readonly ref should be interpreted as a ref readonly... Couldn't we have the ability to have a:

  • readonly ref, i.e. you can do whatever you want on the struct behind the ref, but you can't modify the ref
  • ref readonly, i.e. you cannot modify the struct behind the ref and you can't modify the ref (this would be different from a C++ const). This would allow typically to be able to pass a ref to a readonly field (something we can't do today)

I'm really missing the readonly ref behavior there...

@whoisj

This comment has been minimized.

Show comment
Hide comment
@whoisj

whoisj Oct 13, 2016

@xoofx please can avoid the taint of C here with its const int const *ptr non-sense.

I'm having a difficult time thinking of a real scenario where any would want an mutable pointer to an immutable object. The function could too easily just replace the object with its own, mutate as it sees fit, then return control to the caller who would then have a new struct not realizing it. Seems rife with misuse and danger.

whoisj commented Oct 13, 2016

@xoofx please can avoid the taint of C here with its const int const *ptr non-sense.

I'm having a difficult time thinking of a real scenario where any would want an mutable pointer to an immutable object. The function could too easily just replace the object with its own, mutate as it sees fit, then return control to the caller who would then have a new struct not realizing it. Seems rife with misuse and danger.

@xoofx

This comment has been minimized.

Show comment
Hide comment
@xoofx

xoofx Oct 13, 2016

Member

a real scenario where any would want an mutable pointer to an immutable object

No scenario, I don't propose this (as I said above, the pointer ref readonly is not mutable).

Member

xoofx commented Oct 13, 2016

a real scenario where any would want an mutable pointer to an immutable object

No scenario, I don't propose this (as I said above, the pointer ref readonly is not mutable).

@Thaina

This comment has been minimized.

Show comment
Hide comment
@Thaina

Thaina Oct 13, 2016

In C# world ref of struct is not pointer. It is the object itself. It can only be immutable pointer for mutable object

And by the standard of static internal protected is the same as protected internal static. ref readonly and readonly ref must be the same

Thaina commented Oct 13, 2016

In C# world ref of struct is not pointer. It is the object itself. It can only be immutable pointer for mutable object

And by the standard of static internal protected is the same as protected internal static. ref readonly and readonly ref must be the same

@jaredpar

This comment has been minimized.

Show comment
Hide comment
@jaredpar

jaredpar Oct 13, 2016

Member

@xoofx

I'm really missing the readonly ref behavior there...

Reading your comment I think there may be a bit of a terminology difference. Let me elaborate a bit on the operations for a ref that could be affected by readonly:

  • re-pointing: a ref is really just a pointer that is safe. Hence just like you can change the address a pointer refers to, you could also change the location a ref points to.
  • mutating the target: this is modifying the memory a ref points to. In the case of a class it would be changing it to refer to a new instance (or null). In the case of a struct though it's mutating the contents directly.

Attaching readonly semantics to a ref could choose to affect one, or both of these operations.

When I say readonly ref I'm referring to protecting against mutating the target. I definitely understand the inclination to say that syntactically the readonly modifies the ref so perhaps it should be guarding against re-pointing.

At this time though the language doesn't allow for re-pointing of ref values. I have a lot of skepticism that it would ever be allowed. Mostly because it is of fairly limited use. Midori made heavy use of ref locals / returns and there was only one case in our extremely large code base where we ever wanted to allow for a re-point operation. Additionally allowing for re-pointing complicates the lifetime rules around ref locals significantly. Hence it's low use, extra complication ... less likely to happen.

My skepticism aside though, assume we did desire both re-pointing and the ability to guard against it. That would be in addition to guarding against mutating the target (a very good case can be made for this feature). That means logically variables can now be defined as readonly ref readonly. While that is logically correct and meaningful it probably makes most developers go "huh?".

But if we did go with this feature I'm sure we'll spend plenty of time debating ref readonly vs. readonly ref. Hard to pass up a good naming / syntax debate 😄

Member

jaredpar commented Oct 13, 2016

@xoofx

I'm really missing the readonly ref behavior there...

Reading your comment I think there may be a bit of a terminology difference. Let me elaborate a bit on the operations for a ref that could be affected by readonly:

  • re-pointing: a ref is really just a pointer that is safe. Hence just like you can change the address a pointer refers to, you could also change the location a ref points to.
  • mutating the target: this is modifying the memory a ref points to. In the case of a class it would be changing it to refer to a new instance (or null). In the case of a struct though it's mutating the contents directly.

Attaching readonly semantics to a ref could choose to affect one, or both of these operations.

When I say readonly ref I'm referring to protecting against mutating the target. I definitely understand the inclination to say that syntactically the readonly modifies the ref so perhaps it should be guarding against re-pointing.

At this time though the language doesn't allow for re-pointing of ref values. I have a lot of skepticism that it would ever be allowed. Mostly because it is of fairly limited use. Midori made heavy use of ref locals / returns and there was only one case in our extremely large code base where we ever wanted to allow for a re-point operation. Additionally allowing for re-pointing complicates the lifetime rules around ref locals significantly. Hence it's low use, extra complication ... less likely to happen.

My skepticism aside though, assume we did desire both re-pointing and the ability to guard against it. That would be in addition to guarding against mutating the target (a very good case can be made for this feature). That means logically variables can now be defined as readonly ref readonly. While that is logically correct and meaningful it probably makes most developers go "huh?".

But if we did go with this feature I'm sure we'll spend plenty of time debating ref readonly vs. readonly ref. Hard to pass up a good naming / syntax debate 😄

@xoofx

This comment has been minimized.

Show comment
Hide comment
@xoofx

xoofx Oct 13, 2016

Member

@jaredpar Ah, sorry, may be I have not been enough clear. I'm not proposing the idea to re-pointer the ref (though, I have never had a need for this, but hey, the idea could grow on me 😄 ) , but to disallow the variable (and the struct behind of course) to be re-assigned entirely.

Let me take an example for a readonly ref scenario:

struct MyStruct
{
   public readonly int X;
   public int Y; 
}

public void Process(readonly ref MyStruct val)
{
   // This would not compile
   // In this case, we also disallow the field X to be modified
   // while with a regular ref, we could modify it indirectly with the following code
   val = new MyStruct();  
   // We cannot do this
   val.X++;
   // But we can do this:
   val.Y++;
   ....
}

It allows typically to protect the variable + protect readonly fields behind, which is a nice behavior as It allows partial immutability of a ref struct. If the caller of the method is passing this struct, It can ensure that the callee will not be able to modify its readonly fields (or even private ones).

On the other hand ref readonly would allow to pass a readonly field or variable to another method:

class MyClass
{
     public static readonly MyStruct MyField;
}

public static void Process(ref readonly MyStruct val)
{
    // We cannot do this:
    val = new MyStruct();
    // And also we cannot do this:
    val.Y++;
}

Process(ref MyClass.MyField); // It would be possible

Hope it makes more sense 😅

Member

xoofx commented Oct 13, 2016

@jaredpar Ah, sorry, may be I have not been enough clear. I'm not proposing the idea to re-pointer the ref (though, I have never had a need for this, but hey, the idea could grow on me 😄 ) , but to disallow the variable (and the struct behind of course) to be re-assigned entirely.

Let me take an example for a readonly ref scenario:

struct MyStruct
{
   public readonly int X;
   public int Y; 
}

public void Process(readonly ref MyStruct val)
{
   // This would not compile
   // In this case, we also disallow the field X to be modified
   // while with a regular ref, we could modify it indirectly with the following code
   val = new MyStruct();  
   // We cannot do this
   val.X++;
   // But we can do this:
   val.Y++;
   ....
}

It allows typically to protect the variable + protect readonly fields behind, which is a nice behavior as It allows partial immutability of a ref struct. If the caller of the method is passing this struct, It can ensure that the callee will not be able to modify its readonly fields (or even private ones).

On the other hand ref readonly would allow to pass a readonly field or variable to another method:

class MyClass
{
     public static readonly MyStruct MyField;
}

public static void Process(ref readonly MyStruct val)
{
    // We cannot do this:
    val = new MyStruct();
    // And also we cannot do this:
    val.Y++;
}

Process(ref MyClass.MyField); // It would be possible

Hope it makes more sense 😅

@whoisj

This comment has been minimized.

Show comment
Hide comment
@whoisj

whoisj Oct 13, 2016

It'll be difficult to make a solid case for why we need "immutable references to mutable structs", "mutable references to immutable structs", "immutable references to immutable structs", and "mutable references to mutable structs".

Seems to be (ref stuct) and (readonly ref stuct) is all we need. One allows for mutable the other is immutable. This is a far simpler set of things to understand and the lost "flexibility" closes a lot of holes for bugs to sneak in through.

IMO (readonly ref struct) should be the same as (ref readonly struct), given C# laziness in keyword order enforcement historically.

whoisj commented Oct 13, 2016

It'll be difficult to make a solid case for why we need "immutable references to mutable structs", "mutable references to immutable structs", "immutable references to immutable structs", and "mutable references to mutable structs".

Seems to be (ref stuct) and (readonly ref stuct) is all we need. One allows for mutable the other is immutable. This is a far simpler set of things to understand and the lost "flexibility" closes a lot of holes for bugs to sneak in through.

IMO (readonly ref struct) should be the same as (ref readonly struct), given C# laziness in keyword order enforcement historically.

@xoofx

This comment has been minimized.

Show comment
Hide comment
@xoofx

xoofx Oct 13, 2016

Member

It'll be difficult to make a solid case for why we need "immutable references to mutable structs", "mutable references to immutable structs", "immutable references to immutable structs", and "mutable references to mutable structs".

@whoisj, I have been abusing structs for years in C#, because they are lightweight objects, interop nicely with native code and allow to lower substantially pressure on the GC. And while using them a lot, I have been facing many problems, not only related to performance but also about their safety-ness. Being a strong users of structs makes me looking forward to more powerful abilities (e.g ref locals/returns... but I have so many other stuffs that would probably roll your eyes 😋 ) and stronger options for safety (readonly, more control on immutability). So yes, the cases you are listing like they are small side things (e.g who cares about safety or immutability?), are for me primordial. I'm not talking from a "nice to have place" but from a "real-world usage" place, as yours, but with a different "is all we need" world if you prefer... 😉

Member

xoofx commented Oct 13, 2016

It'll be difficult to make a solid case for why we need "immutable references to mutable structs", "mutable references to immutable structs", "immutable references to immutable structs", and "mutable references to mutable structs".

@whoisj, I have been abusing structs for years in C#, because they are lightweight objects, interop nicely with native code and allow to lower substantially pressure on the GC. And while using them a lot, I have been facing many problems, not only related to performance but also about their safety-ness. Being a strong users of structs makes me looking forward to more powerful abilities (e.g ref locals/returns... but I have so many other stuffs that would probably roll your eyes 😋 ) and stronger options for safety (readonly, more control on immutability). So yes, the cases you are listing like they are small side things (e.g who cares about safety or immutability?), are for me primordial. I'm not talking from a "nice to have place" but from a "real-world usage" place, as yours, but with a different "is all we need" world if you prefer... 😉

@benaadams

This comment has been minimized.

Show comment
Hide comment
@benaadams

benaadams Oct 13, 2016

Contributor

@whoisj I see two variants which @xoofx covers

Pass byval semantics with pass byref cost which I think was the ref readonly example so (good for large structs and read only use):

public static void Process(ref readonly MyStruct val)
{
    // We cannot do this:
    val = new MyStruct();
    // And also we cannot do this:
    val.Y++;

   // However we can do this as it creates a copy; though introduces a byval cost
   var newVal = val;
   newVal.Y++;
}

For byref where you want to allow modifications to the original but not allow overriding of properties which is the readonly ref first example (semi-mutable structs)

public void Process(readonly ref MyStruct val)
{
   // This would not compile
   // In this case, we also disallow the field X to be modified
   // while with a regular ref, we could modify it indirectly with the following code
   val = new MyStruct();  
   // We cannot do this
   val.X++;
   // But we can do this:
   val.Y++;
}

As if you can do the new MyStruct(); then you can override the readonly properties on it with the .ctor

Contributor

benaadams commented Oct 13, 2016

@whoisj I see two variants which @xoofx covers

Pass byval semantics with pass byref cost which I think was the ref readonly example so (good for large structs and read only use):

public static void Process(ref readonly MyStruct val)
{
    // We cannot do this:
    val = new MyStruct();
    // And also we cannot do this:
    val.Y++;

   // However we can do this as it creates a copy; though introduces a byval cost
   var newVal = val;
   newVal.Y++;
}

For byref where you want to allow modifications to the original but not allow overriding of properties which is the readonly ref first example (semi-mutable structs)

public void Process(readonly ref MyStruct val)
{
   // This would not compile
   // In this case, we also disallow the field X to be modified
   // while with a regular ref, we could modify it indirectly with the following code
   val = new MyStruct();  
   // We cannot do this
   val.X++;
   // But we can do this:
   val.Y++;
}

As if you can do the new MyStruct(); then you can override the readonly properties on it with the .ctor

@xen2

This comment has been minimized.

Show comment
Hide comment
@xen2

xen2 Oct 14, 2016

+1 to readonly ref!

When you deal with large struct and want to avoid copies (think Matrix), ref makes a lot of sense.
And of course, we want to have predefined values as static readonly (i.e. Matrix.Identity).

The problem is we can't use any of the Matrix methods that take a ref with those static readonly (i.e. Matrix.Multiply(ref Matrix.Identity, ref matrix2)). The only way is to make a full copy beforehand, or getting rid of the readonly (bringing lot of safety issues).

xen2 commented Oct 14, 2016

+1 to readonly ref!

When you deal with large struct and want to avoid copies (think Matrix), ref makes a lot of sense.
And of course, we want to have predefined values as static readonly (i.e. Matrix.Identity).

The problem is we can't use any of the Matrix methods that take a ref with those static readonly (i.e. Matrix.Multiply(ref Matrix.Identity, ref matrix2)). The only way is to make a full copy beforehand, or getting rid of the readonly (bringing lot of safety issues).

@Thaina

This comment has been minimized.

Show comment
Hide comment
@Thaina

Thaina Oct 14, 2016

Well, I have remember there is an argument about readonly ref. It is about the problem that struct with readonly may not be able to call method (also property). Because, internally, method of struct could modify its value. So it cannot call any method at all

Even get only property can modified struct too

Maybe we also need readonly function and readonly get/set to make it compatible?

Thaina commented Oct 14, 2016

Well, I have remember there is an argument about readonly ref. It is about the problem that struct with readonly may not be able to call method (also property). Because, internally, method of struct could modify its value. So it cannot call any method at all

Even get only property can modified struct too

Maybe we also need readonly function and readonly get/set to make it compatible?

@jaredpar

This comment has been minimized.

Show comment
Hide comment
@jaredpar

jaredpar Oct 14, 2016

Member

@xoofx

but to disallow the variable (and the struct behind of course) to be re-assigned entirely.

Gotcha. That is absolutely the intent of readonly ref. It's a way to safely take a ref to a struct that lives in a readonly location. It effectively disallows all mutations, including assignments.

Member

jaredpar commented Oct 14, 2016

@xoofx

but to disallow the variable (and the struct behind of course) to be re-assigned entirely.

Gotcha. That is absolutely the intent of readonly ref. It's a way to safely take a ref to a struct that lives in a readonly location. It effectively disallows all mutations, including assignments.

@xoofx

This comment has been minimized.

Show comment
Hide comment
@xoofx

xoofx Oct 14, 2016

Member

the intent of readonly ref [...] It effectively disallows all mutations, including assignments.

That's a ref readonly in my terminology. 😅 A readonly ref would allow partial/controlled mutation but not full assignment (see my example above where val.Y++; is possible for a readonly ref), which is important when you want to make sure that a callee cannot modify the private/readonly fields/state of the mutable struct but only through its public mutable API.

Member

xoofx commented Oct 14, 2016

the intent of readonly ref [...] It effectively disallows all mutations, including assignments.

That's a ref readonly in my terminology. 😅 A readonly ref would allow partial/controlled mutation but not full assignment (see my example above where val.Y++; is possible for a readonly ref), which is important when you want to make sure that a callee cannot modify the private/readonly fields/state of the mutable struct but only through its public mutable API.

@jaredpar

This comment has been minimized.

Show comment
Hide comment
@jaredpar

jaredpar Oct 14, 2016

Member

@xoofx

I feel like you're trying to draw a distinction that doesn't really exist though. Their is no real difference between mutating the public and non-public / readonly portions of a struct. A struct is either mutable in it's entirety or not mutable at all.

This example is clearer if you consider method calls. Take for example the following, completely legal, method:

struct S
{
  public readonly int X;
  public int Y;
  private int Z;

  public void M()
  {
    this = new S();
  }
}

In your design would a readonly ref be able to call M without introducing a copy? In order to maintain the proposed semantics of readonly ref the answer must be no. This means then that readonly ref is only a useful distinction for accessible, mutable fields of a struct. I don't think that's enough of a benefit for the extra complexity.

Member

jaredpar commented Oct 14, 2016

@xoofx

I feel like you're trying to draw a distinction that doesn't really exist though. Their is no real difference between mutating the public and non-public / readonly portions of a struct. A struct is either mutable in it's entirety or not mutable at all.

This example is clearer if you consider method calls. Take for example the following, completely legal, method:

struct S
{
  public readonly int X;
  public int Y;
  private int Z;

  public void M()
  {
    this = new S();
  }
}

In your design would a readonly ref be able to call M without introducing a copy? In order to maintain the proposed semantics of readonly ref the answer must be no. This means then that readonly ref is only a useful distinction for accessible, mutable fields of a struct. I don't think that's enough of a benefit for the extra complexity.

@benaadams

This comment has been minimized.

Show comment
Hide comment
@benaadams

benaadams Oct 14, 2016

Contributor

@xoofx for params I could see the semi-muatable working as an in parameter (due to the loose keyword ordering of C# on ref and readonly)

// passed by val (or register)
void Process(MyStruct val)
 // fully mutable including assignment
void Process(ref MyStruct val)
// readonly struct; no assignment, no method calls (get props allowed?)
void Process(ref readonly MyStruct val)
// must be assigned in function
void Process(out MyStruct val) 
// semi-mutable struct, no assignment, but method calls & non readonly assignment fields allowed
void Process(in MyStruct val) 

However the in paramater wouldn't make sense for a return/local maintaning the same sematics

// value/register struct
MyStruct val0 = val;
// fully mutable including assignment
ref MyStruct val0 = val;
// readonly struct; no assignment, no method calls (get props allowed?)
ref readonly MyStruct val = val;
// semi-mutable struct, no assignment, but method calls & non readonly assignment fields allowed
// Not sure what would match for local
Contributor

benaadams commented Oct 14, 2016

@xoofx for params I could see the semi-muatable working as an in parameter (due to the loose keyword ordering of C# on ref and readonly)

// passed by val (or register)
void Process(MyStruct val)
 // fully mutable including assignment
void Process(ref MyStruct val)
// readonly struct; no assignment, no method calls (get props allowed?)
void Process(ref readonly MyStruct val)
// must be assigned in function
void Process(out MyStruct val) 
// semi-mutable struct, no assignment, but method calls & non readonly assignment fields allowed
void Process(in MyStruct val) 

However the in paramater wouldn't make sense for a return/local maintaning the same sematics

// value/register struct
MyStruct val0 = val;
// fully mutable including assignment
ref MyStruct val0 = val;
// readonly struct; no assignment, no method calls (get props allowed?)
ref readonly MyStruct val = val;
// semi-mutable struct, no assignment, but method calls & non readonly assignment fields allowed
// Not sure what would match for local
@xoofx

This comment has been minimized.

Show comment
Hide comment
@xoofx

xoofx Oct 14, 2016

Member

In your design would a readonly ref be able to call M without introducing a copy?

@jaredpar Yes. The struct itself know its state and is the owner of the implementation details. It disallows the callee to break anything that is not exposed by the public API on the struct, but the implementation in the struct can choose whatever is needed. Again, the readonly ref is just saying = The ref is not assignable by the callee, not that the struct behind is readonly. But I understand that the keyword could be misleading (though if we are introducing it for other locals/params, it feels more natural to me but well...)

As @benaadams is suggesting another keyword would be something like in ref or refin, basically a ref that cannot be out (assigned entirely by the callee)

Member

xoofx commented Oct 14, 2016

In your design would a readonly ref be able to call M without introducing a copy?

@jaredpar Yes. The struct itself know its state and is the owner of the implementation details. It disallows the callee to break anything that is not exposed by the public API on the struct, but the implementation in the struct can choose whatever is needed. Again, the readonly ref is just saying = The ref is not assignable by the callee, not that the struct behind is readonly. But I understand that the keyword could be misleading (though if we are introducing it for other locals/params, it feels more natural to me but well...)

As @benaadams is suggesting another keyword would be something like in ref or refin, basically a ref that cannot be out (assigned entirely by the callee)

@jaredpar

This comment has been minimized.

Show comment
Hide comment
@jaredpar

jaredpar Oct 14, 2016

Member

@xoofx

The ability to assign to a struct location and call methods without a copy are equivalent operations. Adding protection for one without protection for the other is just lulling developers into a false sense of confidence about their code.

This all has to do with how this is modeled. In a struct the type of this is ref T. Hence whenever you call a method on a struct the target must be convertible to ref T. That is why it's wrong from a language correctness standpoint to allow readony ref to call a method without a copy. It's implying there is a conversion between readonly ref T and ref T.

Member

jaredpar commented Oct 14, 2016

@xoofx

The ability to assign to a struct location and call methods without a copy are equivalent operations. Adding protection for one without protection for the other is just lulling developers into a false sense of confidence about their code.

This all has to do with how this is modeled. In a struct the type of this is ref T. Hence whenever you call a method on a struct the target must be convertible to ref T. That is why it's wrong from a language correctness standpoint to allow readony ref to call a method without a copy. It's implying there is a conversion between readonly ref T and ref T.

@xoofx

This comment has been minimized.

Show comment
Hide comment
@xoofx

xoofx Oct 14, 2016

Member

Well, If a library A provides a struct (that can be created in a valid state only by lib A using some internal constructors) and and interface with a readonly ref method, this can guarantee to an end user implementing it that It cannot modify the struct in unexpected ways that lib A hasn't covered. It provides confidence for user of lib A, but sure, It doesn't save the developer of lib A to make mistake internally. Can't really adhere to the idea of a strict equality in the behavior between assignment on a struct location and calling a method on it... and like the transient for stackalloc for class, it could be possible to detect struct method that are making such a "violation" and the compiler could report it...
But, seeing how much this idea is controversial, we can forget it, good sign that it is not a good idea after all... 😉

Member

xoofx commented Oct 14, 2016

Well, If a library A provides a struct (that can be created in a valid state only by lib A using some internal constructors) and and interface with a readonly ref method, this can guarantee to an end user implementing it that It cannot modify the struct in unexpected ways that lib A hasn't covered. It provides confidence for user of lib A, but sure, It doesn't save the developer of lib A to make mistake internally. Can't really adhere to the idea of a strict equality in the behavior between assignment on a struct location and calling a method on it... and like the transient for stackalloc for class, it could be possible to detect struct method that are making such a "violation" and the compiler could report it...
But, seeing how much this idea is controversial, we can forget it, good sign that it is not a good idea after all... 😉

@OndrejPetrzilka

This comment has been minimized.

Show comment
Hide comment
@OndrejPetrzilka

OndrejPetrzilka Oct 22, 2016

I'm testing ref locals and I've encountered following limitation. I declare variable ref int as reference to first element in array. How can I change where this variable points? Let's say I want to change it to point to last element, but it's not possible? (see my attempts below)

static int[] data = new int[] { 0, 1, 2, 3, 4 };
unsafe static void Main(string[] args)
{
    ref int slot = ref data[0];

    slot = data[4]; // This stores value "4" into data[0], I don't want that

    // This does not work
    //ref int slot = ref data[4]; // This would be it, except variable is already declared
    //slot = ref data[4];
    //ref slot = ref data[4];

    slot = 99; // When it works, this would overwrite last element

    foreach(var item in data)
    {
        Console.WriteLine(item);
    }
    Console.ReadKey();
}

I thought I'll try to compare performance of this approach in my tree-like collection implemented on array. Currently when looking for element to add/remove, I have locals like int currentIndex, int parentIndex. With this I thought I would use ref Node current, ref Node parent, but when it's not possible to modify current in while loop, it won't work.

OndrejPetrzilka commented Oct 22, 2016

I'm testing ref locals and I've encountered following limitation. I declare variable ref int as reference to first element in array. How can I change where this variable points? Let's say I want to change it to point to last element, but it's not possible? (see my attempts below)

static int[] data = new int[] { 0, 1, 2, 3, 4 };
unsafe static void Main(string[] args)
{
    ref int slot = ref data[0];

    slot = data[4]; // This stores value "4" into data[0], I don't want that

    // This does not work
    //ref int slot = ref data[4]; // This would be it, except variable is already declared
    //slot = ref data[4];
    //ref slot = ref data[4];

    slot = 99; // When it works, this would overwrite last element

    foreach(var item in data)
    {
        Console.WriteLine(item);
    }
    Console.ReadKey();
}

I thought I'll try to compare performance of this approach in my tree-like collection implemented on array. Currently when looking for element to add/remove, I have locals like int currentIndex, int parentIndex. With this I thought I would use ref Node current, ref Node parent, but when it's not possible to modify current in while loop, it won't work.

@axel-habermaier

This comment has been minimized.

Show comment
Hide comment
@axel-habermaier

axel-habermaier Oct 22, 2016

Contributor

@OndrejPetrzilka: AFAIK, that's unsupported. You can't reassign references in C++ either.

Contributor

axel-habermaier commented Oct 22, 2016

@OndrejPetrzilka: AFAIK, that's unsupported. You can't reassign references in C++ either.

@OndrejPetrzilka

This comment has been minimized.

Show comment
Hide comment
@OndrejPetrzilka

OndrejPetrzilka Nov 16, 2016

@axel-habermaier: That makes sense, otherwise it would be probably much harder if not impossible for compiler to detect invalid use. I'm not happy about it though. Is it possible to reassign reference in IL?

@axel-habermaier: That makes sense, otherwise it would be probably much harder if not impossible for compiler to detect invalid use. I'm not happy about it though. Is it possible to reassign reference in IL?

@VSadov

This comment has been minimized.

Show comment
Hide comment
@VSadov

VSadov Nov 16, 2016

Member

It is possible to assign managed pointer in IL, but it is not possible to reset ref local or parameter in C#. Not in C#7.

Safety of use is indeed an issue to solve here.

Member

VSadov commented Nov 16, 2016

It is possible to assign managed pointer in IL, but it is not possible to reset ref local or parameter in C#. Not in C#7.

Safety of use is indeed an issue to solve here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment