New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

C# Design Notes for Apr 15, 2015 #2133

Closed
MadsTorgersen opened this Issue Apr 20, 2015 · 27 comments

Comments

Projects
None yet
9 participants
@MadsTorgersen
Contributor

MadsTorgersen commented Apr 20, 2015

C# Design Meeting Notes for Apr 15

Agenda

In this meeting we looked at nullability and generics. So far we have more challenges than solutions, and while we visited some of them, we don't have an overall approach worked out yet.

  1. Unconstrained generics
  2. Overriding annotations
  3. FirstOrDefault
  4. TryGet

Unconstrained generics

Fitting generics and nullability together is a delicate business. Let's look at unconstrained type parameters as they are today. One the one hand they allow access to members that are on all objects - ToString() and so on - which is bad for nullable type arguments. On the other hand they allow default(T) which is bad for non-nullable type arguments.

Nevertheless it seems draconian to not allow instantiations of unconstrained type parameters with, say, string? and string!. We suspect that most generic types and methods probably behave pretty nicely in practice.

For instance, List<T>, Dictionary<T> etc. would certainly have internal data structures - arrays - that would have the default value in several places. However, their logic would be written so that no array element that hadn't explicitly been assigned a value from the user would ever expose its default value later. So if we look at a List<string!> we wouldn't actually ever see nulls come out as a result of the internal array having leftover nulls from its initial allocation. No nulls would come in, and therefore no nulls would come out. We want to allow this.

Overriding annotations

When type parameters are known to be reference types it may make sense to allow overriding of the nullability of the type parameter: T? would mean sting? regardless of whether the type argument was string!, string or string?.

This would probably help in some scenarios, but with most generics the type parameter isn't actually known to be a reference type.

FirstOrDefault

This pattern explicitly returns a default value if the operation fails to find an actual value to return. Obviously that is unfortunate if the element type is non-nullable - you'd still get a null out!

In practice, such methods tend to be so glaringly named (precisely because their behavior is a little funky) that most callers would probably already be wary of the danger. However, we might be able to do a little better.

What if there was some annotation you could put on T to get the nullable version if T should happen to be a reference type. Maybe the situation is rare enough for this to just be an attribute, say [NullableIfReference]:

public [return:NullableIfReference] T FirstOrDefault<T>(this IEnumerable<T> src)
{
    if ...
    else return default(T);
}

Applied to List<string!> (or List<string>) this would return a string?. But applied to List<int> it would return an int not an int?.

This seems perfectly doable, but may not be worth the added complexity.

TryGet

This pattern has a bool return value signaling whether a value was there, and an out parameter for the result, which is explicitly supposed to be default(T) when there was no value to get:

public bool TryGet(out T result) { ... }

Of course no-one is expected to ever look at the out parameter if the method returns false, but even so it might be nice to do something about it.

This is not a situation where we want to apply the [NullableIfReference] attribute from above. The consumer wants to be able to access the result without checking for null if they have already checked the returned bool!

We could imagine another attribute, [NullableIfFalse] that would tell the compiler at the consuming site to track nullability based on what was returned from the method, just as if there had been a null check directly in the code.

Again, this might not be worth the trouble but is probably doable.

@AdamSpeight2008

This comment has been minimized.

Show comment
Hide comment
@AdamSpeight2008

AdamSpeight2008 Apr 21, 2015

Contributor

@MadsTorgersen

Of course no-one is expected to ever look at the out parameter if the method returns false, but even so it might be nice to do something about it.

What about doing nothing with it? Maintain the already existing value, on false and not change it to default(T).

Contributor

AdamSpeight2008 commented Apr 21, 2015

@MadsTorgersen

Of course no-one is expected to ever look at the out parameter if the method returns false, but even so it might be nice to do something about it.

What about doing nothing with it? Maintain the already existing value, on false and not change it to default(T).

@svick

This comment has been minimized.

Show comment
Hide comment
@svick

svick Apr 21, 2015

Contributor

@AdamSpeight2008 What "already existing value"? C# allows you to use unassigned variables as out parameters, and I think the same should apply to non-nullable types.

I don't think that the facts that at the IL level, there is no out, and that variables tend to have default values should leak into C#.

Contributor

svick commented Apr 21, 2015

@AdamSpeight2008 What "already existing value"? C# allows you to use unassigned variables as out parameters, and I think the same should apply to non-nullable types.

I don't think that the facts that at the IL level, there is no out, and that variables tend to have default values should leak into C#.

@svick

This comment has been minimized.

Show comment
Hide comment
@svick

svick Apr 21, 2015

Contributor

I think that a method like FirstOrDefault() applied on a List<int> should return int?. I think there is no backwards-compatible way to do it for FirstOrDefault(), but it would be nice if I could write FirstOrNull(), which returns nullable type, no matter whether the element type was value type or reference type, nullable or not.

Contributor

svick commented Apr 21, 2015

I think that a method like FirstOrDefault() applied on a List<int> should return int?. I think there is no backwards-compatible way to do it for FirstOrDefault(), but it would be nice if I could write FirstOrNull(), which returns nullable type, no matter whether the element type was value type or reference type, nullable or not.

@AdamSpeight2008

This comment has been minimized.

Show comment
Hide comment
@AdamSpeight2008

AdamSpeight2008 Apr 21, 2015

Contributor

@svick I'm not talking about the unassigned case but the assigned case.

var value = 123;
if( .TryGet( text, value ) 
{   // }
else
{  // value should be 123 still.
}

Since null is sometimes considered a legit valid value.

Shame that we don't have union types.

type Nil
{
}

type Option<T>
{
  | Nil   : Nil
  | Value : T
}
Contributor

AdamSpeight2008 commented Apr 21, 2015

@svick I'm not talking about the unassigned case but the assigned case.

var value = 123;
if( .TryGet( text, value ) 
{   // }
else
{  // value should be 123 still.
}

Since null is sometimes considered a legit valid value.

Shame that we don't have union types.

type Nil
{
}

type Option<T>
{
  | Nil   : Nil
  | Value : T
}
@HaloFour

This comment has been minimized.

Show comment
Hide comment
@HaloFour

HaloFour Apr 21, 2015

@AdamSpeight2008 The compiler cannot enforce that distinction on the author of the method since it has no idea how it might be called. The parameter is out, it must be assigned to a new value before the method returns. Short of nixing out and going only with ref (or implementing all of those methods in another language) there's no real way around that.

HaloFour commented Apr 21, 2015

@AdamSpeight2008 The compiler cannot enforce that distinction on the author of the method since it has no idea how it might be called. The parameter is out, it must be assigned to a new value before the method returns. Short of nixing out and going only with ref (or implementing all of those methods in another language) there's no real way around that.

@Suchiman

This comment has been minimized.

Show comment
Hide comment
@Suchiman

Suchiman Apr 21, 2015

Contributor

@AdamSpeight2008 You are required to assign values to all out parameter before leaving the method and out parameter are considered unassigned inside the method. So there's no way of preserving a existing value. Lifting the requirement of assigning a value to out would lead to variables which can't be proven to be assigned. Your best chances would be using ref in this case but then you would have to always assign a default value first

Contributor

Suchiman commented Apr 21, 2015

@AdamSpeight2008 You are required to assign values to all out parameter before leaving the method and out parameter are considered unassigned inside the method. So there's no way of preserving a existing value. Lifting the requirement of assigning a value to out would lead to variables which can't be proven to be assigned. Your best chances would be using ref in this case but then you would have to always assign a default value first

@AdamSpeight2008

This comment has been minimized.

Show comment
Hide comment
@AdamSpeight2008

AdamSpeight2008 Apr 21, 2015

Contributor

@Suchiman The compiler can already track unassigned variables, so why not automagically insert a check if( outvalue.IsUnassigned ){ outvalue = default<T>() }

Contributor

AdamSpeight2008 commented Apr 21, 2015

@Suchiman The compiler can already track unassigned variables, so why not automagically insert a check if( outvalue.IsUnassigned ){ outvalue = default<T>() }

@HaloFour

This comment has been minimized.

Show comment
Hide comment
@HaloFour

HaloFour Apr 21, 2015

@AdamSpeight2008 Because Roslyn cannot track whether or not the variable is assigned by the caller from within the called method, particularly when these methods exist in a different already-compiled assembly.

HaloFour commented Apr 21, 2015

@AdamSpeight2008 Because Roslyn cannot track whether or not the variable is assigned by the caller from within the called method, particularly when these methods exist in a different already-compiled assembly.

@MrJul

This comment has been minimized.

Show comment
Hide comment
@MrJul

MrJul Apr 21, 2015

Contributor

I'm using ReSharper's nullability annotations as much as possible, which are very similar in concept to what is proposed here. Having direct compiler support instead would be awesome. Here are some scenarios R# handles that you might want to consider:

Known conditional nullability

string Append123IfNotNull(string input)
{
  return input != null ?  input + "123" : null;
}

Here, the nullability of the output depends on the nullability of the input.
ReSharper uses a special contract attribute to cover this case:
[ContractAnnotation("input:null => null; input:notnull => notnull")]

It can also handle the TryGet case shown above:
[ContractAnnotation("=> true, result:notnull; => false, result:null")]

Regarding Roslyn, if method contracts (#119) are implemented, they can and should be used to infer nullability rather than having NullableIfFalse, NullableIfTrue, NullableIfNull, NullableIfParam1And3AreNull, etc.

Nullability propagation for enumerables

string?[] values = new { "a", "b", null, "d" };
IEnumerable<string!> filtered = values.Where(v => v != null);

This is a pretty common use case. IMO, nullability analysis should understand Linq queries. Developers shouldn't be forced to write a plain foreach to benefit from this analysis.

Contributor

MrJul commented Apr 21, 2015

I'm using ReSharper's nullability annotations as much as possible, which are very similar in concept to what is proposed here. Having direct compiler support instead would be awesome. Here are some scenarios R# handles that you might want to consider:

Known conditional nullability

string Append123IfNotNull(string input)
{
  return input != null ?  input + "123" : null;
}

Here, the nullability of the output depends on the nullability of the input.
ReSharper uses a special contract attribute to cover this case:
[ContractAnnotation("input:null => null; input:notnull => notnull")]

It can also handle the TryGet case shown above:
[ContractAnnotation("=> true, result:notnull; => false, result:null")]

Regarding Roslyn, if method contracts (#119) are implemented, they can and should be used to infer nullability rather than having NullableIfFalse, NullableIfTrue, NullableIfNull, NullableIfParam1And3AreNull, etc.

Nullability propagation for enumerables

string?[] values = new { "a", "b", null, "d" };
IEnumerable<string!> filtered = values.Where(v => v != null);

This is a pretty common use case. IMO, nullability analysis should understand Linq queries. Developers shouldn't be forced to write a plain foreach to benefit from this analysis.

@paulomorgado

This comment has been minimized.

Show comment
Hide comment
@paulomorgado

paulomorgado Apr 28, 2015

@AdamSpeight2008, I don't think there's a VB equivalent to C#'s out parameters, and you seem to not be aware of what out parameters are in C#.

paulomorgado commented Apr 28, 2015

@AdamSpeight2008, I don't think there's a VB equivalent to C#'s out parameters, and you seem to not be aware of what out parameters are in C#.

@paulomorgado

This comment has been minimized.

Show comment
Hide comment
@paulomorgado

paulomorgado Apr 28, 2015

@svick, FirstOrDefault on int sequeneces returning an int? doesn't make any sense because the default value of int is 0 (which is an int).

I understand that, when the "no nullables" bunch is not around, you might feel the need for it, but it's just as simple as:

sequenceOfInt.Cast<int?>().FirstOrDefault()

paulomorgado commented Apr 28, 2015

@svick, FirstOrDefault on int sequeneces returning an int? doesn't make any sense because the default value of int is 0 (which is an int).

I understand that, when the "no nullables" bunch is not around, you might feel the need for it, but it's just as simple as:

sequenceOfInt.Cast<int?>().FirstOrDefault()
@AdamSpeight2008

This comment has been minimized.

Show comment
Hide comment
@AdamSpeight2008

AdamSpeight2008 Apr 28, 2015

Contributor

@paulomorgado out parameters requires the variable passed into be initialized and always a value passed back to that parameter.
@HaloFour Roslyn could possibly track it if there was an IL Language Model. That the .net languages (C#, VB.net) would / could be built on.

Contributor

AdamSpeight2008 commented Apr 28, 2015

@paulomorgado out parameters requires the variable passed into be initialized and always a value passed back to that parameter.
@HaloFour Roslyn could possibly track it if there was an IL Language Model. That the .net languages (C#, VB.net) would / could be built on.

@HaloFour

This comment has been minimized.

Show comment
Hide comment
@HaloFour

HaloFour Apr 28, 2015

@AdamSpeight2008 Not even sure how you imagine that to be feasible. Roslyn lacks the quantum computing mechanisms to support compiling based on how consuming code might invoke it sometime in the future. And frankly, we've already got a solution, it's called ref.

HaloFour commented Apr 28, 2015

@AdamSpeight2008 Not even sure how you imagine that to be feasible. Roslyn lacks the quantum computing mechanisms to support compiling based on how consuming code might invoke it sometime in the future. And frankly, we've already got a solution, it's called ref.

@Suchiman

This comment has been minimized.

Show comment
Hide comment
@Suchiman

Suchiman Apr 28, 2015

Contributor

@AdamSpeight2008

@paulomorgado out parameters requires the variable passed into be initialized and always a value passed back to that parameter.

Close but wrong:
out: Can be passed an uninitialized variable, thats why the method is required to assign a value to it and can't read from it prior to doing so.
ref: Variable must be initialized before passing to a method, therefore it's allowed to read from the variable before assigning something to it and assigning to it is optional.

Contributor

Suchiman commented Apr 28, 2015

@AdamSpeight2008

@paulomorgado out parameters requires the variable passed into be initialized and always a value passed back to that parameter.

Close but wrong:
out: Can be passed an uninitialized variable, thats why the method is required to assign a value to it and can't read from it prior to doing so.
ref: Variable must be initialized before passing to a method, therefore it's allowed to read from the variable before assigning something to it and assigning to it is optional.

@svick

This comment has been minimized.

Show comment
Hide comment
@svick

svick Apr 28, 2015

Contributor

@paulomorgado Yes, FirstOrDefault wouldn't be an accurate name for such method.

I didn't know you can use Cast this way. It does mean that there's probably no need to add FirstOrNull.

Contributor

svick commented Apr 28, 2015

@paulomorgado Yes, FirstOrDefault wouldn't be an accurate name for such method.

I didn't know you can use Cast this way. It does mean that there's probably no need to add FirstOrNull.

@paulomorgado

This comment has been minimized.

Show comment
Hide comment
@paulomorgado

paulomorgado Apr 29, 2015

@svick, since Cast() will be called, at most, once, I don't think the benefit justifies the cost.

But it would be nice if I could write something like this:

FirstOrNull<T> := Cast<T?>().FirstOrDefault();

and use it like this:

seq.FirstOrDefault();

paulomorgado commented Apr 29, 2015

@svick, since Cast() will be called, at most, once, I don't think the benefit justifies the cost.

But it would be nice if I could write something like this:

FirstOrNull<T> := Cast<T?>().FirstOrDefault();

and use it like this:

seq.FirstOrDefault();
@RonNewcomb

This comment has been minimized.

Show comment
Hide comment
@RonNewcomb

RonNewcomb Jun 3, 2015

Any sort of Find or Search or Parse op should always return a nullable, because returning null is how it signifies nothing could be found.

public T? FirstOrDefault(this IEnumerable src)

I think the ? or ! appellation should be able to be put on T to ensure/modify the inputted type to have/lack nullability. Attributes are verbose and clunky.

RonNewcomb commented Jun 3, 2015

Any sort of Find or Search or Parse op should always return a nullable, because returning null is how it signifies nothing could be found.

public T? FirstOrDefault(this IEnumerable src)

I think the ? or ! appellation should be able to be put on T to ensure/modify the inputted type to have/lack nullability. Attributes are verbose and clunky.

@gafter

This comment has been minimized.

Show comment
Hide comment
@gafter

gafter Jun 4, 2015

Member

@RonNewcomb That would change the type of the existing FirstOrDefault return type, for example when T is int it would change from int to int?.

Member

gafter commented Jun 4, 2015

@RonNewcomb That would change the type of the existing FirstOrDefault return type, for example when T is int it would change from int to int?.

@RonNewcomb

This comment has been minimized.

Show comment
Hide comment
@RonNewcomb

RonNewcomb Jun 4, 2015

@gafter Correct, which would fix the bug in FirstOrDefault() in which success and failure look identical:

        var i = new int[5] {13, 65, 0, 5, 99 };
        var foo = i.FirstOrDefault(x => x == 0);
        Console.WriteLine(foo);

RonNewcomb commented Jun 4, 2015

@gafter Correct, which would fix the bug in FirstOrDefault() in which success and failure look identical:

        var i = new int[5] {13, 65, 0, 5, 99 };
        var foo = i.FirstOrDefault(x => x == 0);
        Console.WriteLine(foo);
@HaloFour

This comment has been minimized.

Show comment
Hide comment
@HaloFour

HaloFour Jun 4, 2015

@RonNewcomb Is it a bug if it was intentionally designed that way? I don't think so. The "default" value of an int is 0, so to have the method return 0 seems entirely appropriate.

You can't have a single FirstOrDefault method that can return null for both struct or class due to conflicting constraints. You also cannot have overloaded FirstOrDefault methods that differ only by constraints. You're required to have a new method with a different name.

HaloFour commented Jun 4, 2015

@RonNewcomb Is it a bug if it was intentionally designed that way? I don't think so. The "default" value of an int is 0, so to have the method return 0 seems entirely appropriate.

You can't have a single FirstOrDefault method that can return null for both struct or class due to conflicting constraints. You also cannot have overloaded FirstOrDefault methods that differ only by constraints. You're required to have a new method with a different name.

@RonNewcomb

This comment has been minimized.

Show comment
Hide comment
@RonNewcomb

RonNewcomb Jun 4, 2015

@HaloFour I'm aware why it works that way, and that it is true for all value types, and that it is self-consistent in its own way. But within the larger context of this discussion about non-nullables + generics and the possibility of taking T or T! but returning T? to avoid issues with the non-existence of default(T!), I maintain that allowing attaching the ? or ! to T is not only a good idea, but would also solve some existing gotchas like this as well.

(Similarly, it'd be nice if int.TryParse() returned int? so the clunky out parameter wouldn't be needed.)

RonNewcomb commented Jun 4, 2015

@HaloFour I'm aware why it works that way, and that it is true for all value types, and that it is self-consistent in its own way. But within the larger context of this discussion about non-nullables + generics and the possibility of taking T or T! but returning T? to avoid issues with the non-existence of default(T!), I maintain that allowing attaching the ? or ! to T is not only a good idea, but would also solve some existing gotchas like this as well.

(Similarly, it'd be nice if int.TryParse() returned int? so the clunky out parameter wouldn't be needed.)

@HaloFour

This comment has been minimized.

Show comment
Hide comment
@HaloFour

HaloFour Jun 4, 2015

@RonNewcomb As non-nullable reference types are effectively going to boil down to little more than attribute-driven analytics we're not going to gain the ability to treat them differently at the CLR level. A method won't be able to generically declare that it returns both a nullable reference type as a nullable value type, only that it returns a generic type.

The other issues are really corefx concerns. Overloads to Int32.TryParse that returns a Nullable<Int32> could be easily added.

HaloFour commented Jun 4, 2015

@RonNewcomb As non-nullable reference types are effectively going to boil down to little more than attribute-driven analytics we're not going to gain the ability to treat them differently at the CLR level. A method won't be able to generically declare that it returns both a nullable reference type as a nullable value type, only that it returns a generic type.

The other issues are really corefx concerns. Overloads to Int32.TryParse that returns a Nullable<Int32> could be easily added.

@RonNewcomb

This comment has been minimized.

Show comment
Hide comment
@RonNewcomb

RonNewcomb Jun 5, 2015

@HaloFour I don't know what to do with your reply. The article specifically floats the idea of allowing attaching ? to T. I brought examples how that's useful outside of non-nullables as well.

RonNewcomb commented Jun 5, 2015

@HaloFour I don't know what to do with your reply. The article specifically floats the idea of allowing attaching ? to T. I brought examples how that's useful outside of non-nullables as well.

@HaloFour

This comment has been minimized.

Show comment
Hide comment
@HaloFour

HaloFour Jun 5, 2015

@RonNewcomb From the meeting notes it looks like it would affect reference types only. When calling FirstOrDefault with IEnumerable<string!> the return type would be interpreted as a string?. But that would be strictly compiler candy, and it would only apply to reference types. FirstOrDefault with IEnumerable<int> can't return an int? or change the semantics of how that existing method works.

It is possible for the C# compiler to rewrite the calling logic, but in my opinion it involves too much voodoo:

IEnumerable<int> numbers = ...;
int? result = numbers.FirstOrDefault(i => i % 2);

// converted into:
IEnumerable<int> numbers = ...;
int? result = numbers.Select(i => (int?)i)
    .FirstOrDefault(i => i.Value % 2);

Even if the C# language adopted ? to denote an explicitly nullable reference type it still wouldn't be compatible with the syntax for nullable value types as Nullable<T> defines a constraint where T is struct, enforced by the CLR. A string? or string! is still just a normal run of the mill System.String, but a int? is a Nullable<Int32>.

HaloFour commented Jun 5, 2015

@RonNewcomb From the meeting notes it looks like it would affect reference types only. When calling FirstOrDefault with IEnumerable<string!> the return type would be interpreted as a string?. But that would be strictly compiler candy, and it would only apply to reference types. FirstOrDefault with IEnumerable<int> can't return an int? or change the semantics of how that existing method works.

It is possible for the C# compiler to rewrite the calling logic, but in my opinion it involves too much voodoo:

IEnumerable<int> numbers = ...;
int? result = numbers.FirstOrDefault(i => i % 2);

// converted into:
IEnumerable<int> numbers = ...;
int? result = numbers.Select(i => (int?)i)
    .FirstOrDefault(i => i.Value % 2);

Even if the C# language adopted ? to denote an explicitly nullable reference type it still wouldn't be compatible with the syntax for nullable value types as Nullable<T> defines a constraint where T is struct, enforced by the CLR. A string? or string! is still just a normal run of the mill System.String, but a int? is a Nullable<Int32>.

@gafter

This comment has been minimized.

Show comment
Hide comment
@gafter

gafter Jun 5, 2015

Member

@RonNewcomb We're not going to propose making incompatible breaking changes to existing APIs as part of this feature. The method FirstOrDefault has well-defined and well-documented semantics that, while it might not match your use cases, are perfectly well suited to many existing use cases and code bases. We want to ensure that, as we extend the type system, we can annotate existing APIs in such a way that they interact well with the extensions.

Member

gafter commented Jun 5, 2015

@RonNewcomb We're not going to propose making incompatible breaking changes to existing APIs as part of this feature. The method FirstOrDefault has well-defined and well-documented semantics that, while it might not match your use cases, are perfectly well suited to many existing use cases and code bases. We want to ensure that, as we extend the type system, we can annotate existing APIs in such a way that they interact well with the extensions.

@paulomorgado

This comment has been minimized.

Show comment
Hide comment
@paulomorgado

paulomorgado Jun 10, 2015

There's always .Cast<int?>(), @RonNewcomb.

paulomorgado commented Jun 10, 2015

There's always .Cast<int?>(), @RonNewcomb.

@gafter gafter modified the milestone: C# 7 and VB 15 Nov 21, 2015

@gafter

This comment has been minimized.

Show comment
Hide comment
@gafter

gafter Apr 25, 2016

Member

Design notes have been archived at https://github.com/dotnet/roslyn/blob/future/docs/designNotes/2015-04-15%20C%23%20Design%20Meeting.md but discussion can continue here.

Member

gafter commented Apr 25, 2016

Design notes have been archived at https://github.com/dotnet/roslyn/blob/future/docs/designNotes/2015-04-15%20C%23%20Design%20Meeting.md but discussion can continue here.

@gafter gafter closed this Apr 25, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment