Proposal: Nullable reference types and nullability checking #5032

Open
MadsTorgersen opened this Issue Sep 5, 2015 · 156 comments
@MadsTorgersen
Contributor

Null reference exceptions are rampant in languages like C#, where any reference type can reference a null value. Some type systems separate types into a T that cannot be null and an Option<T> (or similar) that can be null but cannot be dereferenced without an explicit null check that unpacks the non-null value.

This approach is attractive, but is difficult to add to a language where every type has a default value. For instance, a newly created array will contain all nulls. Also, such systems are notorious for problems around initialization and cyclic data structures, unless non-null types are permitted to at least temporarily contain null.

On top of these issues come challenges stemming from the fact that C# already has null-unsafe types, that allow both null values and dereferencing. How can a safer approach be added to the language without breaking changes, without leaving the language more complex than necessary and with a natural experience for hardening existing code against null errors in your own time?

Approach: Nullable reference types plus nullability warnings

The approach suggested here consists of a couple of elements:

  • Add a notion of safe nullable reference types T? to the language, in addition to the current ones T which are from now on non-nullable.
  • Detect and warn on cases where nullable types are dereferenced, or when null is values are assigned to (or left in) non-nullable variables.
  • Offer opt-in means to deal with breaking behavior and compat across versions and assemblies.

The feature will not provide airtight guarantees, but should help find most nullability errors in code. It can be assumed that most values are intended not to be null, so this scheme provides the minimal annotation overhead, leaving current code to be assumed non-nullable everywhere.

Nullable reference types are helpful, in that they help find code that may dereference null and help guard it with null checks. Making current reference types Non-nullable is helpful in that it helps prevent variables from inadvertently containing an null value.

Nullable and non-nullable reference types

For every existing reference type T is now "non-nullable", and there is now a corresponding nullable reference type T?.

Syntactically speaking, nullable reference types don't add much, since nullable value types have the same syntax. However, a few syntactic corner cases are new, like T[]?.

From a semantic viewpoint, T and T? are mostly equivalent: they are the same type in that they have the same domain of values. Specifically, because the system is far from watertight, non-nullable reference types can contain null. The only way in which they differ is in the warnings caused by their use.

For type inference purposes T and T? are considered the same type, except that nullability is propagated to the outcome. If a reference type T is the result of type inference, and at least one of the candidate expressions is nullable, then the result is nullable too.

If an expression is by declaration of type T?, it will still be considered to be of type T if it occurs in a context where by flow analysis we consider it known that it is not null. Thus we don't need new language features to "unpack" a nullable value; the existing idioms in the language should suffice.

string s;
string? ns = "hello";

s = ns; // warning
if (ns != null) { s = ns; } // ok

WriteLine(ns.Length); // warning
WriteLine(ns?.Length); // ok

Warnings for nullable reference types

Values of nullable reference types should not be used in a connection where a) they are not known to (probably) contain a non-null value and b) the use would require them to be non-null. Such uses will be flagged with a warning.

a) means that a flow analysis has determined that they are very likely to not be null. There will be specific rules for this flow analysis, similar to those for definite assignment. It is an open question which variables are tracked by this analysis. Just locals and parameters? All dotted names?

b) means dereferencing (e.g. with dot or invocation) or implicitly converting to a non-nullable reference type.

Warnings for non-nullable reference types

Variables of non-nullable reference types should not be assigned the literal null or default(T); nor should nullable value types be boxed to them. Such uses will result in a warning.

Additionally, fields with a non-nullable reference type must be protected by their constructor so that they are a) not used before they are assigned, and b) assigned before the constructor returns. Otherwise a warning is issued. (As an alternative to (a) we can consider allowing use before assignment, but in that case treating the variable as nullable.)

Note that there is no warning to prevent new arrays of non-nullable reference type from keeping the null elements they are initially created with. There is no good static way to ensure this. We could consider a requirement that something must be done to such an array before it can be read from, assigned or returned; e.g. there must be at least one element assignment to it, or it must be passed as an argument to something that could potentially initialize it. That would at least catch the situation where you simply forgot to initialize it. But it is doubtful that this has much value.

Generics

Constraints can be both nullable and non-nullable reference types. The default constraint for an unconstrained type parameter is object?.

A warning is issued if a type parameter with at least one non-nullable reference constraint is instantiated with a nullable reference type.

A type parameter with at least one non-nullable constraint is treated as a non-nullable type in terms of warnings given.

A type parameter with no non-nullable reference constraints is treated as both a nullable and a non-nullable reference type in terms of warnings given (since it could without warning have been instantiated with either). This means that both sets of warnings apply.

? is allowed to be applied to any type parameter T. For type parameters with the struct constraint it has the usual meaning. For all other type parameters it has this meaning, where S is the type with which T is instantiated:

  • If S is a non-nullable reference type then T? refers to S?
  • Otherwise, S? refers to S

Note: This rule is not elegant - in particular it is bad that the introduction of a struct constraint changes the meaning of ?. But we believe we need it to faithfully express the type of common APIs such as FirstOrDefault().

Opting in and opting out

Some of the nullability warnings warn on code that exists without warnings today. There should be a way of opting out of those nullability warnings for compatibility purposes.

When opting in, assemblies generated should contain a module-level attribute with the purpose of signaling that nullable and non-nullable types in signatures should generate appropriate warnings in consuming code.

When consuming code references an assembly that does not have such a top-level attribute, the types in that assembly should be treated as neither nullable nor non-nullable. That is, neither set of warnings should apply to those types.

This mechanism exists such that code that was not written to work with nullability warnings, e.g. code from a previous version of C#, does indeed not trigger such warnings. Only assemblies that opt in by having the compiler-produced attribute, will cause the nullability warnings to happen in consuming code accessing their signatures.

When warnings haven't been opted in to, the compiler should give some indication that there are likely bugs one would find by opting in. For instance, it could give (as an informational message, not a warning) a count of how many nullability warnings it would have given.

Even when a library has opted in, consuming code may be written with an earlier version of C#, and may not recognize the nullability annotations. Such code will work without warning. To facilitate smooth upgrade of the consuming code, it should probably be possible to opt out of the warnings from a given library that will now start to occur. Again, such per-assembly opt-out could be accompanied by an informational message reminding that nullability bugs may be going unnoticed.

Libraries and compatibility

An example: In my C# client code, I use libraries A and B:

// library A
public class A
{
  public static string M(string s1, string s2);
}

// library B
public class B
{
  public static object N(object o1, object o2);
}

// client C, referencing A and B
Console.WriteLine(A.M("a", null).Length);
Console.WriteLine(B.N("b", null).ToString());

Now library B upgrades to C# 7, and starts using nullability annotations:

// upgraded library B
public class B
{
  public static object? N(object o1, object o2); // o1 and o2 not supposed to be null
}

It is clear that my client code probably has a bug: apparently it was not supposed to pass null to B.N. However, the C# 6 compiler knows nothing of all this, and ignores the module-level attribute opting in to it.

Now I upgrade to C# 7 and start getting warnings on my call to B.N: the second argument shouldn't be null, and I shouldn't dot into the return value without checking it for null. It may not be convenient for me to look at those potential bugs right now; I just want a painless upgrade. So I can opt out of getting nullability warnings at all, or for that specific assembly. On compile, I am informed that I may have nullability bugs, so I don't forget to turn it on later.

Eventually I do, I get my warnings and I fix my bugs:

Console.WriteLine(B.N("b", "")?.ToString());

Passing the empty string instead of null, and using the null-conditional operator to test the result for null.

Now the owner of library A decides to add nullability annotations:

// library A
public class A
{
  public static string? M(string s1, string s2); // s1 and s2 shouldn't be null
}

As I compile against this new version, I get new nullability warnings in my code. Again, I may not be ready for this - I may have upgraded to the new version of the library just for bug fixes - and I may temporarily opt out for that assembly.

In my own time, I opt it in, get my warnings and fix my code. I am now completely in the new world. During the whole process I never got "broken" unless I asked for it with some explicit gesture (upgrade compiler or libraries), and was able to opt out if I wasn't ready. When I did opt in, the warnings told me that I used a library against its intentions, so fixing those places probably addressed a bug.

@MadsTorgersen MadsTorgersen self-assigned this Sep 5, 2015
@MgSam
MgSam commented Sep 5, 2015

So are the thoughts on the ! operator from the Aug 18 notes not part of the official proposal yet?

Additionally, fields with a non-nullable reference type must be protected by their constructor so that they are a) not used before they are assigned, and b) assigned before the constructor returns. Otherwise a warning is issued. (As an alternative to (a) we can consider allowing use before assignment, but in that case treating the variable as nullable.)

This rule seems like a non-starter to me, as it is impossible for the compiler to verify this if you use a helper method called from the constructor to assign fields. That is unless you provide an in-line way of disabling warnings (essentially telling the compiler that you know for certain the fields are being assigned). ReSharper has such a feature to disable warnings.

@HaloFour
HaloFour commented Sep 5, 2015

@MgSam Isn't that currently a pain-point with non-nullable fields in Apple Swift? What's worse is that the base constructor could call a virtual method overridden in the current class that could access those fields before they've been assigned so not even the constructor is good enough.

@tpetricek

This looks nice!

Can the proposal be clarified on how are the nullability annotations going to be stored at the CLR level? (I suppose these have no runtime meaning and they would just produce a .NET attribute or some other sort of metadata?)

It would be nice if some future version of the F# compiler could read & understand these and offer some sort of "strict" mode where nullable types are automatically exposed as option<'T> (or something along those lines).

One other possible related idea would be to have an annotation not just when referring to a type, but also when defining a type - for example, types defined in F# do not allow null value (when used safely from F#) and so when the C# compiler sees them, it should (ideally) always treat them as non-nullable, but when working with a type that came from some old .NET library that has nulls everywhere, it makes sense to treat the type as nullable by default. (This would perhaps make things too complicated - but some declaration-site/use-site option might be worth considering...)

@rmschroeder

Seconding Tomas' points above, this is a great as it stands, but it would be nice if there could be some way to enable a 'strict' mode that guaranteed no nulls and if that could be trusted by consumers of libraries, or is there some way to determine if a dependency did not ignore the warnings?
I would also like to hear about how this would be exposed via metadata, reflection, etc.

@mariusschulz

What I love about this approach is that non-nullability would be the default for reference types, which I think is a lot better than having to opt in to it explicitly every single time (like string! or object!).

@HaloFour In Swift, you can't call a base class' constructor until all (non-nullable) fields have been initialized. That prevents exactly the problem you mentioned.

@deiruch
deiruch commented Sep 5, 2015

I like this idea and implementation very much. +1 vote to enable (possible) warnings for arrays of non-nullable references.

@YuvalItzchakov

First of all, kudos to the language team for this proposal. It definitely looks like effort and time has been put to thinking about this.

Regarding the proposal, It seems to me that non-nullability in C#, given its age and maturity would be confusing to many developers, senior and junior as one. Given that the proposal is mainly around compiler time warnings and not errors, this would lead to even more confusion, as it would feel like something developers can easily ignore. Also, we are so used to nullability being around structs, that suddenly allowing them on reference types can add to that same confusion and peculiar feeling.

Think feature doesn't "feel" like it can be done properly without baking it into the CLR. I think attempts to work around not creating a breaking change wouldn't be as good as it can be.

@FlorianRappl

One question that directly appeared was how library authors should handle parameters, which are not supposed to be null, but are given as null.

The example reads:

public static string? M(string s1, string s2); // s1 and s2 shouldn't be null

Now prior to C# 7 we could introduce a runtime check such as:

public static string M(string s1, string s2) // s1 and s2 can be null
{
  if (Object.ReferenceEquals(s1, null))
    throw new ArgumentNullException(nameOf(s1));
  if (Object.ReferenceEquals(s2, null))
    throw new ArgumentNullException(nameOf(s2));
  /* ... */ // won't crash due to s1 or s2 being null
}

This result, of course, in a runtime error if null is passed-in. In C# 7, knowing that the input should be non-null, a warning in generated. Now we could omit these checks and maybe have a crash, which is a consequence of passing in null references, however, since the user has been warned the origin should be obvious.

public static string? M(string s1, string s2) // s1 and s2 shouldn't be null
{
  /* ... */ // could crash due to s1 or s2 being null
}

Is it supposed to be used like that or would the recommendation be to have both kind of checks (a runtime guard and the compiler mechanism to have an initial warning)?

Needless to say that the runtime guard may have other benefits and provides backwards compatibility, but also adds noise that could be reduced with the new approach.

Additionally I agree to @YuvalItzchakov (especially regarding baking it into the CLR).

@sandersaares

@FlorianRappl I imagine the null-checking operator from #5033 would be an ideal match for that scenario if it could also apply to method arguments.

public static string? M(string s1!, string s2!)
{
}

Or maybe with an alternate syntax (inconsistent with null-checking operator elsewhere but potentially more suitable for method arguments):

public static string? M(string! s1, string! s2)
{
}

Such an approach would check for null (at runtime) in addition to emitting non-nullable reference warnings at compile time.

The visuals look a bit odd at first glance but having an easy way to specify "throw if null" is very valuable, especially if we also consider that nullability analysis by the compiler would be one basis for the non-nullable reference type warnings, meaning that you would normally run into the following issue:

public static void Bar(string s) { ... }

public static void Foo(string? s)
{
    Argument.VerifyIsNotNull(s, nameof(s)); // Think green, save energy by saving 1 line of code!

    Bar(s1); // Warning: potentially null value used.
    // The compiler can't tell (without extra annotations) that a null check was done out of band.
}

With the null-checking operator applying to a method argument, this is no problem:

public static void Bar(string s) { ... }

public static void Foo(string s!)
{
    Bar(s1); // The compiler knows that a null check was done, so no problem here.
}
@HaloFour
HaloFour commented Sep 5, 2015

@mariusschulz It solves one problem by creating another. Since the ctor must come first you're stuck having to initialize those fields via initializers to some arbitrary non-null but still invalid values.

@tpetrina
tpetrina commented Sep 5, 2015

What if we have two functions defined as

public void foo(string s) {}
public void foo(string? s) {}

Are these two different functions or will this trigger an error? Can we restrict generics to accept only non-nullable references?

@HaloFour
HaloFour commented Sep 5, 2015

@tpetricek

Considering that the ? is just shorthand for attribute-based metadata those two functions would have the same signature and such an overload would not be permitted.

@ruuda
ruuda commented Sep 5, 2015
string s;
string? ns = "hello";

s = ns; // warning
if (ns != null) { s = ns; } // ok

I am a big fan of non-nullable types and I applaud this proposal, but in my opinion this is a bad example. If string? ns = SomeFunctionThatReturnsAString() this would make sense, but in the current case perhaps it would be more appropriate to warn about the unnecessary nullability of ns, as it is assigned a non-null value.

@govert
govert commented Sep 5, 2015

+1 for making non-nullable the default, and not introducing string!. This proposal points us in the right direction, even if the road is long.

Although the scope here is limited to a language feature, one might anticipate a future where the runtime type system is extended to include non-nullable reference types, and where the runtime can enforce type safety for non-nullable types. In other words, an 'airtight' system should still be the end goal.

With such a future in mind, +1 also for the suggestion of @rmschroeder to have an opt-in 'strict' compilation mode which ensures the resulting assembly would (potentially) be type-safe in such a non-nullable-aware runtime. When compiling in 'strict' mode:

  • The warnings should be errors.
  • Arrays should be definitely initialized, perhaps in one of a few compiler-recognized constructs like an appropriate for(;;) loop, or a ToArray() call on an Enumerable of non-nullable values, or a designated array initializer method taking some int => T delegate. So there would be a limited set of statically-checked ways to build a 'strict' non-nullable array.
  • other edge cases be dealt with in a conservative way, that always ensures non-nullable type safety.

References between 'strict' and non-'strict' assemblies are still a problem. At least the compiler should be able to generate a null-checking wrapper that allows a 'strict' assembly to be safely used by a non-'strict' assembly, and vice versa. The wrapper would generate the null check to ensure that void MyMethod(string s) never receives a null when called from another assembly through the wrapper. Maybe the checking code is built into the 'strict' assembly, but not invoked when called from another 'strict' assembly.

While it makes sense to decouple the language feature from having a runtime that is non-nullable-aware, let's also keep in mind where we really want to go. That will allow C# 7.0 to build our trusted assemblies of the future.

@JamesNK
JamesNK commented Sep 5, 2015

There is already a "strict" mode, it's called treat warnings as errors.

@govert
govert commented Sep 5, 2015

@JamesNK That could work, if we have:

  • an 'airtight' conservative warning mode that will generate warning when the hard-to check cases fail, like arrays or fields that are not definitely initialized, and
  • something (an attribute) baked into the assembly that indicates it was built in the 'non-nullable-warnings-as-errors-mode' so that the tooling and future infrastructure can recognize these.

I kind of see this like "unsafe" code, which opts out of type safety, and is not implemented merely as warnings.

@naasking
naasking commented Sep 5, 2015

The discussion of generics is incomplete. For one, it doesn't actually explain how to declare a non-null type parameter. Unconstrained type parameters are T:object? by default, so then if you explicitly constraint it as T:object, it's non-nullable?

Except T:object is currently forbidden as a constraint by [1]: constraints can't be special type "object". I'm all for eliminating that constraint entirely since it makes no sense (along with the sealed class/struct constraint error!).

Finally, creating a guaranteed non-null array seems achievable by using a new array constructor:

public static T[] Array.Create<T>(int length, Action<T[], int> initialize)
    where T : object

Then a simple flow analysis ensures that the unmodified index parameter is used to initialize the array parameter.

[1] https://github.com/dotnet/roslyn/blob/051ad4381100859442af93e16d8d8edbb681b05f/src/Compilers/CSharp/Portable/Binder/Binder_Constraints.cs#L164

@xdaDaveShaw

Just a thought as someone who uses "Treat Warnings as Errors" on everywhere, there's no mention of how likely false warnings are to appear and how are people going to deal with them. Will it be case of people just sticking ?. instead of . to try and avoid dereferencing null, which could lead people to introduce other subtle bugs if they are not thinking things through.
The last think I'd want to see is a code base full of #pragma disable CSXXXX around blocks of code - it's bad enough when Code Analysis flags false warnings and they need to be suppressed, but at least those have support for a Justification - which can be enforced with another CA rule if needed :).

@robertknight robertknight referenced this issue in Microsoft/TypeScript Sep 5, 2015
Closed

Suggestion: non-nullable type #185

@danieljohnson2

I think you really should separate the checked-nullable-references (that is: T? for reference type T) from this proposal; there is some value in having a clear indication that this or that field or parameter is expected to be null sometimes, and having the compiler enforce that null is checked for.

But it seems to me that he rest of this is just not viable as given. The proposal is to declare the existing references types non-nullable, without enforcing this at all. All existing C# code is to be declared broken, and to be punished with an avalanche of useless warnings.

I do feel some confidence about that last point: I don't believe you can generate the warnings accurately enough. The false positives will be overwhelming. We'll all wind up silencing that particular warning.

This is a shame. The advantage of non-nullable references, if they are enforced somehow, is not that we are forced to check for null everywhere. It is that we would not need to anymore.

@MiddleTommy

I think getting rid of null (mostly) all together is a bad idea. We are so entrenched with it. If your reference type can not be null we will be doing default value checking and error catching more.
We need better automatic ways to handle null.
I think we need something like default values to ease our problem.

public int Count(this IEnumerable items != null) { return ....;}

In this method items is guaranteed to not be null by the compiler. There can be compiler checks to help this but also automatic ArgumentNullException throwing.

I use null a lot instead of throwing exceptions. I always felt exceptions mean something went wrong. Null means something doesn't exist not that that is wrong.

So there will be a host of people who would like the != null to just pass the method by returning a default
public int Count(this IEnumerable items != null ?? 0)
{ return ....;}

This is stating that if items is null don't throw an exception, return the default value instead.

@Eirenarch

I dislike the proposal. Reliance on warnings, not proving correctness and punishing existing code with a bunch of warnings all feel like we are discussing JavaScript rather than C#. I'd prefer a proposal that works in fewer cases but is proven to be correct and results in errors rather than warnings. The current one seems very confusing to existing programmers and will be very frustrating when the compiler declares their existing project broken.

@ScottArbeit

This is brilliant. Can anything be easy and perfect 16 years into the lifespan of C#? No. Is this a robust and minimalist way to accomplish this? Yes. And two years after it's available in C# 7? We'll go through the phase of having warnings, we'll clean up our code, and we'll finally get rid of (almost) all of the null-reference exceptions we're all plagued with. Thank you, Mads, as a .NET developer since Beta 2, I really appreciate it.

@jonathanmarston

I like the idea of non-nullable reference types in C#, but I'm not sold on them only being implemented as compiler warnings. Consider the following code:

string GetAsString(object o)
{
    return o.ToString();
}

Under the current proposal the method above would not produce any warnings or errors, but neither the compiler nor the runtime actually provide any guarantees that "o" isn't null.

Yes, the caller to this method would receive a warning if the compiler determines that it is not "very likely" that the value being passed is not null, but what if this method is part of a shared class library? And what if the consumer of this library isn't using C# 7, or ignored the warning? This code would generate an NRE just as it would today.

With the current proposal the writer of the library would still need to add a runtime null check to be safe, even though he was told that a reference type without a question mark is non-nullable. The compiler is just giving the illusion of null-safety without actually providing it. Developers will get used to receiving warnings when they write unsafe code (most of the time), assume that if they don't get them that they wrote null-safe code, and start neglecting to write proper runtime null checks when necessary, in some ways making matters worse than they are today!

@diryboy
Contributor
diryboy commented Sep 6, 2015

+1 @YuvalItzchakov and @FlorianRappl for the concern about the potentially broad confusion.

Traditionally, reference types are already expected to be null-able, adding new notation to specify the same thing and changing the meaning of the existing type identifiers can cause a lot of troubles, for example, when you read existing open source code, you have to check document for the language version, when you use some library, you have to check that too, but the library wouldn't be necessarily written in C#. Today we just check the runtime version.

A more reasonable approach would be to add new notation for types that are expected to be non-nullable, like the ! mentioned by @sandersaares

When old code calls into new code where new code expect objects to be non-nullable, runtime failures are expected under both proposals (implicit non-nullable T or explicit non-nullable T!). I would imagine the binary of the new code would be the same, but only diff at human readable code, in that case, the explicit ! seems to be easier for human to spot and identify the problem.

@naasking
naasking commented Sep 6, 2015

Treating this broad proposal as compile-time errors instead of warnings is simply infeasible for obvious reasons. Those who wish to assure null correctness right off the bat can just use the csc compiler flag /warnaserror [1].

@diryboy, the point of the proposal is to stop thinking about reference types as inherently nullable, because they're mostly not. There are few scenarios where nullable type is justified, just as there are few scenarios where a nullable struct is justified, and making this explicit via ? is the sane default. We program with values, and whether null is a valid value is contextual, not universal to a whole class of types. The "null everywhere" property is baggage from other languages, and thank god it's finally dying off.

When old code calls into new code where new code expect objects to be non-nullable, runtime failures are expected under both proposals (implicit non-nullable T or explicit non-nullable T!).

Except old code already expected such runtime failures. The point is that linking new code with new code can prevent any such failures by addressing all of the warnings. The old code can also address these warnings the same way via recompilation.

Doing this via a new annotation won't fix existing code either. Warnings provide an easy migration path that doesn't break your existing compiles, to a point where you can eventually use /warnaserror everywhere and be assured your program will never throw a null reference exception.

[1] https://msdn.microsoft.com/en-us/library/406xhdz3.aspx

@diryboy
Contributor
diryboy commented Sep 6, 2015

Are there any good way to express the context where T? is actually T?

Consider this simple class, when execution is success, Result is not null, Error is null, when not, opposite

class ExecutionResult
{
  public object? Result;
  public object? Error;
}

Under this proposal seems one have to perform 2 null checks in either success or failure case.

When the above class was declared as

class ExecutionResult<T>
{
  public T? Result;
  public object? Error;
}

This even bring more questions. What is ExecutionResult<int?>? What about ExecutionResult<string?>?

Let's consider another case.

class Exception
{
  public Exception? InnerException { get; }
}

For Tasks that throws exceptions, it's wrapped into InnerException and thrown as AggregatedException, how to specify that InnerException is not null? By overriding the property with Exception?

@devuxer
devuxer commented Sep 6, 2015

I think this is an exciting proposal. Besides the promise of cutting way down on null reference exceptions, it would make expectations clear. If there is a ? after the type, null is acceptable, otherwise, it's not. I've always thought value types had an unfair advantage over reference types because they could be explicitly marked as nullable or not. With this proposal, reference types would no longer be second-class citizens in this respect.

That said, I'm a little confused by the opt-in process. I may be misunderstanding, but it seems like both the author and consumer of a C# 7 assembly must decide whether to opt-in to default non-nullability. If this is correct, this is how I think it should work:

  • Author opts-out, consumer opts-out --> It's as if we never left C# 6. Nothing to see here.
  • Author opts-in, consumer opts-in --> Non-nullable by default is the law of the land. Give us compiler errors (not warnings).
  • Author opts-in, consumer opts-out --> Sure, go ahead and show us a compiler message telling us we're missing out on some nice compiler checking.
  • Author opts-out, consumer opts in --> Everything (except value types) should look to us as if its nullability is a mystery. I'm not wedded to this, but perhaps a double question mark would do the trick (e.g., string?? Name). Here, I think compiler warnings would make more sense (e.g., "possible null reference exception").
@jfrijters

I really like this proposal. One thing I'm missing is field initialization before base constructor invocation. This would solve many common cases for non-nullable fields (i.e. those initialized with values passed in to the constructor).

Something like this:

class NamedFoo : Foo {
  readonly string name;
  public NamedFoo(string name)
    : name(name), base() {
  }
}

Something else: I think its interesting to consider encoding non-nullable T as modopt(NonNullable) in signatures and allow method overloading on nullableness. This would make it easier to update APIs in a backward compatible way (as the previous C# compiler would prefer the overload without the modopt (i.e. the nullable one)). The obvious downside is that this doesn't scale very well to multiple arguments, but there could be some guidance around that (it's probably possible to explain to people that overloads with to least number of nullable references types will be preferred by downlevel compilers).

@AdamsLair

Would this proposal work completely on a compiler level, or would it actually introduce changes to the underlying Types? And if so, what would be the effect on Reflection, should this proposal be implemented?

Type first = typeof(string);
Type second = typeof(string?);
  1. Would first and second be different / non-equal Types?
  2. Would second still be a string Type, or some kind of wrapping generic like NullableReference<string>?
  3. Would the answer to either of this change, depending on whether the (calling or object's defining) code was compiled with or without nullable references active?

Especially with regard to automated serialization and copying / cloning systems, this could make a difference. Since they can easily encounter objects / Types that were just passed on to them in some way, I'm not sure how well opt-in strategies would apply here.

I didn't read the entire thread and merely skimmed certain parts, so I might have overlooked the answer to this - in that case, feel free to disregard my comment.

@JesperTreetop

@AdamsLair According to this part in the first post:

From a semantic viewpoint, T and T? are mostly equivalent: they are the same type in that they have the same domain of values. Specifically, because the system is far from watertight, non-nullable reference types can contain null. The only way in which they differ is in the warnings caused by their use.

It looks like they both have the same System.Type, as opposed to e.g. *T. It's an extra value that follows the type around during analysis (not known to be anything, known to be nullable, known to be non-nullable), maybe in a way that's comparable to dynamic vs object, and probably similar to dynamic being distinguished by DynamicAttribute in the IL since this will be something you'll be able to see in libraries and thus can't evaporate in compilation.

I support this proposal. There are tons of corner cases, I'm sure (like is T?[]? going to be a thing) and lots of things to figure out, but it is the best that can be done to move forward and allow for progress to be made without a full reboot. (I'm in favor of a reboot too, I just don't like the looks I get from the people who just spent six years rewriting the compiler.)

@naasking
naasking commented Sep 6, 2015

@diryboy > What is ExecutionResult<int?>? What about ExecutionResult<string?>

It's the same result as you'd expect with ExecutionResult<Nullable>, if ExecutionResult where T:struct. If a ? appears next to a type, we expect it to be nullable. A Nullable<Nullable> is possible. While semantically dubious, why should we care what the developer writes as long as it's not unsound?

Under this proposal seems one have to perform 2 null checks in either success or failure case.

Why? Presumably your constructors will enforce that one of either Result or Error cannot be null, so just check whether one is null, and if it is, you know the other isn't.

For Tasks that throws exceptions, it's wrapped into InnerException and thrown as AggregatedException, how to specify that InnerException is not null? By overriding the property with Exception?

That's a good example. I think the only answer is that it requires a new property that shadows the base property, ie. "public new Exception InnerException { get { (Exception)return base.InnerException; } }", because you can't change the type while overriding (if ? is treated as a type). For clients that work on AggregatedException, they see the new property and know it's not null, and for clients accepting Exception, they see that it may be null and check for that as usual.

@paul1956
paul1956 commented Sep 6, 2015

Then should I be able to do the following without getting a warning/error or being told I need a cast on "x?.getvalue" and A is unchanged if x is Nothing, correct?

Dim A as MyTypeValue = 4 ' none nullable
A = x?.getvalue

What is the propose behavior of a CType (cast) from a nullable to a non-nullable when the value being cast is Nothing as the result of a null propagation?

Sent from my iPhone
I apologize for any typos Siri might have made.
(503) 803-6077

On Sep 6, 2015, at 5:52 AM, JesperTreetop notifications@github.com wrote:

@AdamsLair According to this part in the first post:

From a semantic viewpoint, T and T? are mostly equivalent: they are the same type in that they have the same domain of values. Specifically, because the system is far from watertight, non-nullable reference types can contain null. The only way in which they differ is in the warnings caused by their use.

It looks like they both have the same System.Type, as opposed to e.g. *T. It's an extra value that follows the type around during analysis (not known to be anything, known to be nullable, known to be non-nullable), maybe in a way that's comparable to dynamic vs object, and probably similar to dynamic being distinguished by DynamicAttribute in the IL since this will be something you'll be able to see in libraries and thus can't evaporate in compilation.

I support this proposal. There are tons of corner cases, I'm sure (like is T?[]? going to be a thing) and lots of things to figure out, but it is the best that can be done to move forward and allow for progress to be made without a full reboot. (I'm in favor of a reboot too, I just don't like the looks I get from the people who just spent six years rewriting the compiler.)


Reply to this email directly or view it on GitHub.

@diryboy
Contributor
diryboy commented Sep 7, 2015

@naasking It's a trap. Nullable<Nullable<int>> won't compile. T?? causes parser error like old C++ compilers parse A<B<C>> where >> or ?? happens to be another operator. So actually it's more reasonable that they collapse to single ? for both generic examples of ExecutionResult.

Making the compiler considering the constructor looks very smart. I'm not sure it's possible. I meant when you consume ExecutionResult from some binary, I thought the sematic analyzer only sees the "type" of Result and Error as nullable of something. ExecutionResult could be written in other languages.

Not sure one should override or shadow in the AggregatedException case, I felt like overriding, as ? might just be some kind of extra metadata annotated via some attribute. Like dynamic, it's fine to do things like

class A
{
  public virtual dynamic M(){ return null; }
}
class B : A
{
  public override object M() { return 1; }
}
@naasking
naasking commented Sep 7, 2015

Nullable<Nullable> won't compile.

But it won't compile due to extra rules added specifically for this case, not because it's semantic nonsense according to C#'s standard rules. System.Nullable is a struct, so it satisfies the requirements of it's own type parameter.

I agree collapsing is probably the better default just for consistency with these extra rules, but there have been plenty of discussions in the functional programming community whether "Maybe Maybe T" is sensible, and it really is necessary in some cases. Consider a Dictionary<TKey, TValue> where Dictionary.Add returns TValue? that's null if the key isn't present. Dictionary<string, object?> should be a valid instantiation, which means null should be a valid value, which means Add should return object??, and this is all perfectly sensible, but would be unrepresentable in C#.

I thought the sematic analyzer only sees the "type" of Result and Error as nullable of something. ExecutionResult could be written in other languages.

Regardless, the developer clearly understands the intended semantics of ExecutionResult, when either one or the other is not null. It's up to the ExecutionResult's developer to ensure those semantics, and the client relies on those semantics. I still don't see any motivation for checking both members for null.

Not sure one should override or shadow in the AggregatedException case, I felt like overriding, as ? might just be some kind of extra metadata annotated via some attribute.

Shadowing is compatible with both scenarios, where non-null is treated as a type or it's just metadata, ie. it's future-compatible. Overriding is compatible only with one scenario. I don't see any motivation for overriding in this case, unless you can provide one?

@diryboy
Contributor
diryboy commented Sep 7, 2015

@naasking nullable nulls or Just Nothing.. Given int?? i = new Nullable<Nullable<int>>(null), how to construct something of the similar type object??? What would be the result of i == null? What about int?? j = new Nullable<Nullable<int>>(0) and the result of j == 0?

It seems to me that if this proposal implements Maybe<TRef> to pair Nullable<TVal> makes more sense for nullable nulls?

When you are given some type with some property defined as nullable and the compiler give you warning(or even treated as error) if you don't check null before use, would you bother check how that type constructs? In my ExecutionResult example:

// suppose we have a function expects non-nullable object
static void use( object o ) {...}

// check error first
if ( r.Error == null )
{
  use(r.Result);  // warning or even error
  use(r?.Result); // happy?
}
else ...

// check result first
if ( r.Result == null )
{
  use(r.Error);  // warning or even error
  use(r?.Error); // happy?
}
else ...

Unless we have subclass SucessfulResult and FailureResult which implement the similar thing we discussed in the AggregatedException that override/shadow Result/Error, turning the above code into something like

var sr = r as SucessfulResult;
if ( sr != null )
{
  use(sr.Result); // happy
}
else
{
  var fr = (FailureResult)r;
  use(r.Error);  // happy
}

Shadowing makes a new member, diff behavior happen when cast the object to base type. No diff in this simple property getter case, but could be a problem when subtype do more things than that.

@qrli
qrli commented Sep 8, 2015

+1000 to non-nullable by default, thus a consistent syntax with value types.

PS: it seems better to have a compiler option to generate run-time non-nullable argument check code, at least for public functions.

@jonathanmarston

@qrli

I like the idea of generated run-time null checks of non -nullable arguments. This would totally remove the need for the oft-repeated "if null throw ArgumentNullException" code. The only issue would be that you'd now be doubling the checks with legacy code that already has these checks in place. Maybe one of the new warnings could be for comparing a non-nullable variable to null. This way the developer would get a warning as a signal to remove the redundant, manual checks.

@danieljohnson2

@jonathanmarston Adding runtime checks would make this feature rather less useless, but we'd really need a distinct 'mandatory T' type to trigger that, or all existing C# code would be broken right out of the gate. In that case, you might as well have errors rather than warnings- better to break everything at compile time rather than runtime.

Then again, for anyone who treats errors-as-warnings, this proposal already breaks everything anyway. In for a penny, in for a pound, perhaps?

@jonathanmarston

@danieljohnson2

In my opinion, being overly concerned with backward compatibility is a bit counter-productive for this feature. We are talking about a fundamental change to the way C# thinks about reference types here. The current proposal already acknowledges this by making even just the warnings opt-in, but I don't feel like warnings are enough. If I am opting in, let me go all the way and opt in to enforced non-nullability.

In my opinion, if this feature is implemented, it should come with a compiler option with three choices for how to handle non-nullables:

None: works the same as C# 6. You're opting out of the new functionality. Any upgraded projects would have this set by default and the developer would have to go to the project properties to change it to get the new functionality. This means all existing projects still compile fine in C# 7 by default.

Warn: works as described in the current proposal. This would give developers a migration path to start coding for non-nullables without diving in completely.

Enforce: the default for new C# 7 projects. Most of the warnings from the "warn" option are now errors, and runtime null argument checks are generated for non-nullable parameters in public methods. This protects class libraries that are fully using non-nullables from consumers that have not been upgraded to use non-nullables yet.

There a few reasons why I prefer this over depending on errors-on-warning for those of us that want to go all-in on non-nullables. The first is that I may have other warnings that I don't have time to fix right now (calling deprecated APIs, for instance) because it would require a large refactoring, but I would like to start using non-nullables right away. Another is the issue of class libraries that have opted in to using non-nullables, but consumers that have not. In this situation the developer of the class library would still have to write null argument checks, even though she has opted into and is using non-nullable parameters. It's totally counter-intuitive.

Even what I suggest above still leaves the rather ambiguous case of opted-in code calling old code that has not opted-in. I think the compiler giving the benefit of the doubt and not raising errors or warnings in this case is the best route (as opposed to flooding the developer with hundreds of warnings), but the type should somehow be displayed differently in the IDE so a developer knows that they are dealing with unknown nullability.

As a side note, what is the plan for string v = default(string)? Would this produce a warning? It would be really nice if somehow default(string) was string.Empty, and default(string?) was null...

@qrli
qrli commented Sep 9, 2015

@jonathanmarston

Enforce: the default for new C# 7 projects. Most of the warnings from the "warn" option are now errors, and runtime null argument checks are generated for non-nullable parameters in public methods.

I prefer runtime-check for all only for debug build. For release build, I'd want runtime-check only for public members, if I am building a library; or none, if I am building an application.

As a side note, what is the plan for string v = default(string)? Would this produce a warning? It would be really nice if somehow default(string) was string.Empty, and default(string?) was null...

That may be too much breaking change, and it is incompatible with generic case. And what about custom string-like classes (e.g. if I implement a Matrix class) which has its own Empty value? I think the simple rule is default(string) == null and it is not allowed for non-nullable variable.

@tpetrina
tpetrina commented Sep 9, 2015

Can we define default members for each class/struct? Something like

public class Matrix
{
    public static default Matrix Empty { get; } = new Matrix();
}

// works with
var emptyMatrix = default(Matrix);
// also for arrays
var matrices = new Matrix[5];
@danieljohnson2

@jonathanmarston, backwards compatibility would certainly hinder this feature- but it is still a very important thing. This is a "fundamental change to the way C# thinks about reference types", and having it turned on by a compiler switch means you have two C# dialects that you can't mix in the same project.

Turning that switch on isn't going to be easy for existing code: it's not enough to sprinkle your code with '?', you have to find the idioms needed to convince the compiler that you are checking for null and apply them throughout.

This is going to be ugly.

@jonathanmarston

@tpetrina

That's sort of like what I had in mind. If default became an overridable operator a class implementation could make default whatever makes sense for that type. But what would default be for a class that doesn't override it?

@danieljohnson2

I totally agree. This could get messy. Maybe rather than fundamentally change C# it's time for C## instead?

@ljr1981
ljr1981 commented Sep 9, 2015

The Eiffel language has completely solved and resolved pointer dereferencing once and for all. We refer to it as "Void Safety". There is a simple and elegant solution, which requires knowledge by the compiler and language constructs that do not obfuscate. The solution also involves a couple of keywords with new code constructs: detachable and attached.

The language constructs are call CAPs or Certifiable Attachment Patterns. They are code patterns, which the compiler can parse and logically deduce that a reference will not become Void at every time it is used.

This technology works for both synchronous and asynchronous software.

@HaloFour
HaloFour commented Sep 9, 2015

@ljr1981

"attached" and "detached" read as non-nullable and nullable respectively. Most of CAP seems to just be flow-analysis. I'm not seeing much in the paper on the subject that seems really new given the type-checking and null coalescing operators that C# has today. I'd think that the C# compiler could apply such techniques today on existing reference types without requiring any syntax changes.

The paper does highlight a lot of the same problems facing C#, particularly where generics and arrays are concerned. The pain of the migration is also mentioned.

@divega divega referenced this issue in aspnet/Identity Sep 10, 2015
Merged

Using [NotNull] and 'nameof' operator #485

@paulomorgado

This was an hard to understand proposal.

If I understand it correctly, this is nothing more that a method contracts on steroids. There are no guarantees made by the compiler and less by the runtime. The compiler just does dataflow analysis on variables declared to be nullable.

So, those on the non-nullable reference types parade may store away their drums and pipes. This isn't nor will not be in the foreseeable future about non nullable reference types.

Although I understand this the best that can be done with the current runtime, there are a few things that bother me:

  1. Like value types are inherently not nullable, reference types are inherently nullable. Likewise, if a modifier is needed to indicate that a value type is nullable, a modifier should be needed to indicate that a reference type is not nullable.
  2. Reference types are inherently nullable. This proposal artificially tries to pass them as inherently not nullable.

@MadsTorgersen, can you elaborate on why it was chosen a modifier for nullable reference type variables and not a modifier for non nullable reference type variables?

@psmacchia

I agree with @paulomorgado "this sounds like artificial" + "I understand this the best that can be done with the current runtime".

Also, unless I miss this piece of info (but I searched for it in the various responses), we'd really need to understand how this would work at metadata level to provide our definitive thoughts on this. (see first questions from @tpetricek @rmschroeder).

I am a heavy consumer of the pattern: bool TryGet(object obj, out T t); to avoid things like T Get(object obj) where the result can be null in case of failure to get. With this syntax the caller code would have to be:

T? t;
if(!TryGet(obj, out t) { return; } // t is null, if and only if, TryGet() returns false
if(t == null) { return; }

which is awkward.

Also some tools should be developed upon Roslyn, to automatically modify the legacy code to avoid warnings. This could be done for at least simple cases.

@dsaf
dsaf commented Sep 14, 2015

@MadsTorgersen do you have any comment about @FlorianRappl 's question?

...would the recommendation be to have both kind of checks (a runtime guard and the compiler mechanism to have an initial warning)?
Needless to say that the runtime guard may have other benefits and provides backwards compatibility, but also adds noise that could be reduced with the new approach.

It would be nice to know, because it affects the code written by us today. Thank you.

@tpetrina

After littering my code with [NotNull] and fixing obvious nullref bugs, I am ready for shorthand syntax. Actually, no. I would prefer that by default references are non-nullable in my projects.

Adding ! suffix for non-nullability is a readability nightmare and using ? for nullable references is a much better choice.

@NightElfik

I am really excited that the The Billion Dollar Mistake is being addressed!! As people already mentioned (@rmschroeder, @govert, @jonathanmarston), I am strongly in favor of following behaviors:

  1. By default, reference types should be non-null.
  2. The non-null checking should be strictly enforced by compiler (leak-proof) - this might be compilation option.

With current proposal, do I understand it correctly that even if I have string argument somebody can in fact pass me null, However, if I decide to check it for null I will get a warning?

I am personally trying to not use null at all in my recent C# programs. I use struct Option<T> where T : class for nullable types and I am really happy with it - I haven't seen NullReferenceException for very long time :) Language support would be amazing!

I understand that there is big problem to make it backwards compatible. On the other hand, this is very very important feature and huge step forwards for C# - I could imagine having a compiler switch that would make the null checking leak-proof with cost of breaking backwards compatibility. Otherwise just generate warnings.

Thanks for the proposal!

@paulomorgado

So, @NightElfik, instead of NullReferenceExceptions, what have you been seeing?

This proposal only does dataflow analysis on varriables declared to be nullable. Nothing else.

@jacekbe
jacekbe commented Sep 21, 2015

This proposal looks very nice and I would really like it implemented. Imho it's much better than introduction of explicit non-nullable types for several reasons:

  • better consistency with value types with regard to nullability
  • non nullable references are usually what the user wants therefore should be the default.
  • introduction of explicit nullable and non-nullable reference types would probably lead to deprecation of regular references as there would be little use for them. 3 types of references that differ only by their nullability is an overkill.

I can imagine that adoption of new semantics would take long time so users should be able to control whether incorrect usage of references leads to warning, error or is not signalled at all.
In the worst case scenario people would just disable nullability tracking, but I doubt it as this feature is going to be very helpful.

@or150
or150 commented Sep 23, 2015

I wonder what will happen with the default clause.
ValueTypes have a default ctor and therefore a default value. Reference types on the other hand dont. It will break lots of generic types. Also what will happen to all the TryGetX methods. Will there be a wrapper type for mandatory references (Mandatory<>)?
I think that the mandatory types (T!) proposal will fit better with existing constucts.

@HaloFour

@or150 Actually the ctor of a value type is entirely optional. C# 6.0 allows defining a default parameterless ctor for a value type and if you then use default(type) that ctor is never called:

public struct Foo {
    public int X { get; }

    public Foo() { X = 1; }
}

static void Main() {
    Foo foo1 = new Foo();
    Foo foo2 = default(Foo);
    Debug.Assert(foo1.X == 1);
    Debug.Assert(foo2.X == 0);
}

It doesn't appear that the behavior of default() is mentioned in the proposal. The question I think would be whether or not it could be applied to a non-null reference type. Would default(string) be legal or would you be expected to use default(string?)? If default(string) is legal would it also result in a string? of null? Could type x = default(type) resolve to type x = new type()? Probably not.

This gets much more complicated when generics come into play. I'd love to hear how the compiler might be able to handle default(T) in a method where T has no generic type constraints but the consumer wishes to use a non-nullable reference type for T.

@NightElfik

@or150 In ideal world without backwards compatibility and with strict nullability rules, I can imagine that default(T) is not valid for non-nullable T. In this world you would not need TryGet methods at all because Get (or indexer) would return T? - nullable reference/struct. When you need detault(T) it is a sign that you actually need nullable type anyways and default(T?) should give you what you need.

In real world, this is a problem and having T be strictly non-nullable type breaks backwards compatibility. If the rules are not strict then it is not as useful as it could be. I am still hoping for some kind of strict-nullability-checking solution though. The T! might work better?

@danieljohnson2

Java has already done something similar to this warningtastic proposal. Java's erased generics are enforced through warnings; if you can avoid all such warnings, your program is statically type-safe and runtime checks are not needed.

But you can disable the warnings, and then Java's generics become something else. They become more like Objective-C's 'id' type: they imply invisible, automatic, and unchecked conversions. This is actually very convenient, if not very safe.

It's a good example of the 'dialect' thing: you can write useful programs that work but do not type-check in Java generics. You can just disable the warnings and go on with your day. But you've written a program that is not legal in the other, strictly typed, dialect of Java.

The ugly part is what happens when these worlds collide. The safe, statically typed java world comes crashing down, because the static guarantees only work when everyone plays along. I believe they call this "heap pollution"- you can get data structures whose dynamic content does not agree with the static types originally written for them, so that code that is strictly correct will fail in surprising ways.

I think we should avoid introducing this "heap pollution" into .NET.

BTW, @HaloFour I think that the parameterless constructors in structs got dropped. My VS 2015 won't do them.

@HaloFour

@or150 Ah, you're right. That must've been cut prior to release. But default(type) doesn't invoke any ctors (default or otherwise), it only emits the IL opcode ininobj which just assures that the memory location is zeroed out.

As for the whole dialect thing, there's a lot about it that leaves room to be desired. For example, the entire "opt-in" for your project thing. That seems way too clumsy and coarse. Turn it on and not only are you forced to fix potentially huge numbers of warnings in otherwise perfectly legit code but you're also immediately cut off from the 15 years of existing sample code on the Internet. I think most projects won't adopt it simply because it's too daunting to do so. Yes, it makes sense, pull the bandaid and get it over with. But there are also countless examples of projects remaining on older versions of languages/frameworks specifically because of breaking/dialect changes like this. My company still has projects compiling with VC++ 6.0 because nobody wants to expend the time and energy updating it for the sake of updating it.

I'm not saying that there is a better answer; frankly there isn't. We're 15 years and 6 versions too late to make such a fundamental paradigm shift. I'd honestly rather see it play out in a forked experimental language in the C# 7.0 timeframe, discover the problems and get real feedback. Perhaps it's not worth it without CLR enhancements.

@qrli
qrli commented Sep 24, 2015

@NightElfik @HaloFour
I think the generic dilemma is already mentioned in OP. There is one paragraph talking about how to solve the generic type issue. In my understanding, it means generic T is treated the same as before, it means T? for reference types and T! for value types. And it allows you to use T? and T! to enforce it.
So default(T) and default(T?) are null for reference types, while default(T!) is illegal.

To me, it is an acceptable solution.

Another solution I can imagine is to add generic constraints:

void F<T>() {} // same behavior as now
void F<T>() where T : nullable {} // T is nullable
void F<T>() where T : nonnullable {} // T is non-nullable, so default(T) is illegal.
@scottsteesy

The concept of having a non-nullable reference type is very interesting, but it seems like most of this proposal is guaranteed to cause all kinds of confusion and problems when mixing old and new code.

Just a thought, but wouldn't it be easier to just add an attribute to the class that says it is supposed to be non-nullable and define a special (static?) constructor ( MyClass!() ) to define the "default" value for the class. Then have the compiler see the attribute and always use the result of the special constructor as the default value for the class. It could also raise errors if a null is assigned to the class type variable.
Wouldn't this make it so that new code can have non-nullable reference types?
Granted I suppose that if the new code (assembly) is consumed by old code with an older compiler, it wouldn't be enforced, but if they were using a new compiler even with old code and framework shouldn't it work?

@HaloFour

@qrli

New generic constraints would require CLR changes, although perhaps they could be expressed through the same attribute/erasure method.

Let's take the following method:

public T Foo<T>() => default(T);

It would seem that the compiler would be required to treat consumption of this method as follows:

int i = Foo<int>(); // legal
int? ni = Foo<int?>(); // legal
string? ns1 = Foo<string?>(); // legal
string? ns2 = Foo<string>(); // ?

string s = Foo<string>(); // can't imagine this could be legal, caller cannot enforce this
@qrli
qrli commented Sep 25, 2015

@HaloFour
Your example is interesting. Generalize the question a bit: Since X is a subset of X?, should X be allowed to be used as T??

string? ns2 = Foo<string>();

I would like to see it is allowed, because it would provide better backward-compatibility with old code. But the default(T) inside Foo() is indeed a trouble. More over, Foo() may simply assign null to variable of T if where T : class. So it seems this cannot be allowed.

However, it means T without any constraint can only represent value types, nullable value types, reference types, but not non-nullable reference types. This is not good...

One idea is: in the generic case, only enforce non-nullability on parameters and return value, so default(T) is still allowed as long as its value is not passed outside.

@fschmied
fschmied commented Oct 2, 2015

See also #227.

@fschmied
fschmied commented Oct 2, 2015

There is a ReSharper plugin that causes ReSharper's nullability inspections to treat unannotated type references in parameters, properties, etc. as non-nullable by default: https://resharper-plugins.jetbrains.com/packages/ReSharper.ImplicitNullability/. For example, this generates warnings if you try to pass nullable values to non-nullable parameters. Our team has been using the plugin during the past few months, the experience of having reference types be non-nullable by default is great.

Of course, adding this to the language (and compiler) would be very much superior to the plugin's approach.

@dsaf
dsaf commented Oct 2, 2015

@fschmied

Of course, adding this to the language (and compiler) would be very much superior to the plugin's approach.

but

The feature will not provide airtight guarantees...

Is it really that better than a set of analyzers? I guess at least it will be "standard" and come for free.

@jonathanmarston

The more I think about this proposal the more I think that it's a bad idea.

What bothers me is that it suggests changing the language spec to make non-nullable the default, but then only enforcing the change through warnings.

C# has is a type-safe language and this change implies that string and string? are different types. If I have a method parameter typed as a string (not string?) I should be able to know, without a doubt, that I will always get a string, not a null. No question. No caveats.

If all it is going to do is produce warnings then it would be better implemented as an attribute and an analyzer.

@AdamSpeight2008
Contributor

I think this should be combined with the ability to specify a specialised non-null <T>.Default
(note default(T) remains unaffected)

@AdamSpeight2008
Contributor

Can constructors and initialisation of a type, be made to be "out-of-phase" until fully constructed and initialised. After which it "instantly" becomes usable. The only way I think it could be done is globally locking when doing so.

@danieljohnson2

@AdamSpeight2008, it's really not generally possibly to prevent a class from 'leaking' and being observed before it has finished initialization. You can come close by doing your initialization stuff before you chain to your base class (which can then leak a reference safely) but if your class is not sealed, a subclass can do as it likes before you get to run.

The most obvious thing a subclass could do to ruin your non-nullable day is to have a finalizer. If you take an exception during construction, the finalizer will still run. And perhaps crash on an 'impossible' null reference.

But if we can't change the CLR itself, we can't get this to be perfectly airtight no matter what we do. Finalizers are a bad idea in the first place; maybe problems with them are not so terrible.

Nevertheless, while some flaws are tolerable, I think the warning-only approach is too leaky and makes the 'mandatory' references too unreliable.

@vladd
vladd commented Oct 3, 2015

@danieljohnson2 One thing I don't understand is why it's not possible to change the CLR to add support for "perfectly non-nullable" reference types. I don't think it's actually impossible, it's just believed to be quite complicated. A bigger problem which I see is the BCL: some of the code will be silently not valid any more (e.g.: default(T) for non-nullable T is we allow an arbitrary T for generic parameters), so this implies a big redesign/rewrite of BCL. (Example: for the List<T>, the actual storage be done not with T[] but rather with T?[]).

@danieljohnson2

@vladd, the MS compiler guys here seem very reluctant to consider CLR changes, especially big ones. I think it's partly an organizational thing- they aren't in the same department or something. But CLR changes are also scarier from a compatibility point of view.

In particular, if they change the CLR so that unmarked reference types are really not nullable, that will break compatibility with all existing .NET binaries. That's not a goer.

I think the alternative to CLR changes (and a much more complex and ambitious proposal) is really a much less ambitious one- restrict the feature to places where the compiler can do this reliably. But this means restricting it quite a lot.

The middle ground here seems to me to be a proposal that doesn't work, a C# divided into dialects, and compatibility problems anyway.

@olmobrutall

As someone who has spent some time thinking about non-nullable reference types (https://gist.github.com/olmobrutall/31d2abafe0b21b017d56) I've to say I really like this solution.

I see many developers having problems spotting value and reference types and having a symmetric solution is much better, even if is based on warnings. We can always opt-in to turn them into errors, or even change the CLR in a far future when everybody has made his homework :).

I've two issue however:

Comparing non-nullable reference types with null literal

Today if you write this code you get a warning:

int a = 2;
if(a == null) //warning CS0472: The result of the expression is always 'false' since a value of type 'int' is never equal to 'null' of type 'int?'
{

}

Should the same code generate a warning for non-nullable reference types?

string s = "";
if(s == null) //<--- warning CS0472 as well?
{    
}

The benefit of adding this warning is that will be easier for a developer migrating code to C# 7 to spot potential variable/fields/parameters/return types that should be made explicitly null-able based on how is used, or remove redundant code.

On the other side, since the whole proposal is not watertight, we need a way to check for null on non-nullable reference types without triggering a warning. Maybe casting before?

string s = "";
if(((string?)s) == null) //<--- no warning
{    
}

One typical scenario is argument checking:

public static string Reverse(string s)
{
    if(s == null) //warning CS0472?
        throw new ArgumentNullException(nameof(s1));
} 

Should we warn code that does compile-time AND run-time checking? Or complicate the code with a casting?

Being to string with definitely-assignment on fields

Also typical are object witch initialization is not finished after construction, like objects with cycles.

class Head 
{ 
      Body body; //warning CS0472?
}

class Body 
{ 
     Head head;
}

public Body CreateBody()
{
    Head head = new Head(); 
    Body body = new Body{ Head = head };
     //head.Body is null!!!
     head.Body = body;
     //world-order is restored
     return body;
}

One particular case are ORMs entities, that are not really 'created' till they are saved in the database:

class Invoice 
{
     Customer Customer { get; set;} 
     DateTime CreationDate { get; set; } = DateTime.Now;

     DateTime? CancellationDate { get; set; }
     User? CancelledBy { get; set; }
}

In this example, is convenient to write just Customer to mean that CustomerID column in the Invoice table is non-nullable, even if when the Invoice is created is temporally null, until the user picks one customer and saves the entity. Having to create a constructor with one parameter for each non-nullable referenced entity will be a pain.

@NightElfik

@olmobrutall While I really like your write-up, great work! However, I have to disagree with what you suggested, mostly with the second part.

The "exceptions" from the rule to achieve convenience are only making the whole rule less useful.

When there are two classes with cyclic dependencies, at leas one of them has to have a nullable reference. There should be no other way around. If you want to have nice interface, just make it using non-nullable properties.

class Head { 
    private Body? m_body;
    public Body {
        get {
            if (m_body == null) { throw new InvalidOperationException(); }
            return (Body)m_body;
        }
        set { m_body = value; } 
    }
}

class Body { 
    private Head? m_head;
    public Head {
        get {
            if (m_head == null) { throw new InvalidOperationException(); }
            return (Head)m_head;
        }
        set { m_head = value; } 
    }
}

public Body CreateBody() {
    Head head = new Head(); 
    Body body = new Body();
    body.Head = head;
    head.Body = body;
    return body;
}

In fact, this could be the default implementation of properties with non-nullable types :).

And in case you want to make the classes "immutable", it's simple extension:

class ImmutableHead { 
    private Body? m_body;
    public Body {
        get {
            if (m_body == null) { throw new InvalidOperationException(); }
            return (Body)m_body;
        }
        set {
            if (m_body != null) { throw new InvalidOperationException(); }
            m_body = value;
        }
    }
}

The second example is "broken" is a sense that you can freely have "invalid" instance and pass it around. And if you want to check at some point that your instance is in fact "invalid", you will be bombarded by warnings CS0472? That is not right...

I do understand that without CLR support it will be never watertight, but making exceptions "for convenience" will never bring it close enough to make the eventual switch.

@danieljohnson2

@olmobrutall, I think your first section highlights this proposals big problem- its warnings are just wrong, because the "non nullable references" are really entirely nullable.

But your second issue, about initialization, illustrates one path out of the mess.

We could say that it is not an error for an ordinary field to be null, but it is an error to observe the null. Null then means 'uninitialized', but that's okay as long as you do initialize before use.

We could enforce this with runtime checks. Since the 'T!' syntax seems to have been rejected, let us consider a [NullSafe] attribute that can an apply at method, class, or assembly level. The marked body of code would get additional null checks compiled in- not just when dereferencing a null, but also when reading or writing one from any field not marked nullable.

You could then have this:

[NullSafe] Object example()
{
  Object[] notNullable = null; // warning here, of course
  notNullable = new Object[47]; // but this fixes it at runtime
  for(int i=1;i<47;++i) notNullable[i]=new Object(); // oops, missed one!
  return notNullable[0]; // throws exception *reading* a null
}

The warnings here are not sufficient, but the runtime check means that no caller will ever see a null come back from this method. It's runtime checking, but it does check at an earlier point that what you get today.

The thing about this approach is that Object and Object? are no longer actually different types; they are just notations for how to deal with nulls. In practice it becomes like C++ const; you can cast it away and cheat, thus:

Object[] notNullable = { 1, 2, 3 }; // safe, no nulls here!
Object?[] nowItIsNullable = (Object?[])notNullable; // allowed with cast!
nowItIsNullable[0]=null; // allowed!
return nowItIsNullable[0]; // no runtime check, returns null

but

return notNullable[0]; // runtime check, throws instead

This is all weaksauce compared to CLR-enforced mandatory reference types, but as long as we won't change the CLR, weaksauce is the sauce we get. In the CLR these things are all nullable as heck, and we should mitigate this without pretending it isn't so.

@olmobrutall

Hi @NightElfik

Thanks for your feedback. Actually my example can also be implemented with all non-nullable referenes:

class Head
{
    public Body Body { get; set; }

    public Head (Body body)
    {
        this.Body = body;  //body.Head is null at this point!!
    }
}

class Body
{
    public Head Head { get; set; }

    public Body()
    {
        this.Head = new Head(this);
    }
}

Still, this solution has two problems:

  1. Even if it looks watertight, in the Head constructor you could observe a head-less body.
  2. In general, you have to add a constructor parameter for each non-nullable member, making POCOs so much more complicated.

You can choose to complicate the code using custom property setters (your example), or custom constructors (my example), but at the end I'm not sure if it's worth.

Watertight languages are usually functional languages with algebraic data types (F#, Haskell), or a very complicated constructor rules are necessary (Swift).

In C# your are usually using POCOs, DataBinding and ORMs, and making the not nullability too strict could create more problems than it solves, backwards-compatibility apart.

I think our experience with generics (where JAVA uses type erasure and C# gives us strong guarantees backed in the CLR with better results) is misleading us: There is no good reason to store an int in a List<string>, and it's easy to enforce, but there are lots of reasons to have an initial null on a variable/array/property that, on the long run, should be expected to be not null.

Take Typescript as an example. The type system is a fantasy and can be easily cheated but the type annotations help programmers express intent and that's enough to remove most of the errors. Maybe this time being strict is not what C# needs.

@fschmied

As I menationed earlier, I'm on a team that has been using "implicitly non-nullable reference types" for method parameters via a ReSharper plugin for some time. We're currently considering enabling that feature also for return values, and I'd like to share our experience with this matter.

ReSharper has nullability control flow analysis and nullability annotations: it allows parameters, methods (i.e., return values), and fields to be annotated as "can be null" and "not null" via custom attributes. This information is then used to warn when a (provable) nullable value is used in a place where "not null" is expected. And it is used to suggest removing null checks when a value is provable not null based on these annotations.

There are two drawbacks with how this is implemented by ReSharper:

  • First, the feature requires explicit opt-in. In our code-base, 99% of all parameters (and method return values) are non-nullable - annotating all of them is a lot of effort. That's why we're using the aforementioned plugin to change the default for all parameters in our code base to be "not null". This makes our situation very similar to what is proposed in this issue.
  • Second, the feature is incomplete. Values for which ReSharper can neither prove "can be null" nor "not null" cause no warnings to be issued. (It is possible to make the control flow analysis stricter, but due to the large number of unannotated code in libraries, this produces so many false positive warnings that it is virtually unusable.) With fields, it's even worse - since uninitialized fields are null by default and ReSharper doesn't and cannot enforce their initialization, fields marked "not null" are not reliably not null.

In my opinion, this second point - that you cannot rely on the non-nullability, i.e., it's not truly contract, is a real problem. When non-nullability is a 95% thing only, you still need to check every non-nullable value for being null before you do anything with it if you want to avoid errors caused by null values. And it's worse than before: ReSharper will suggest you remove those null checks! If you do remove them, you can get null-related errors - maybe near the caller who supplied null, maybe not. Maybe down the callstack, maybe up the callstack. Maybe two months later because the invalid null value was written into a database and caused an error only when it was read again.

That's why, in our case, we haven't considered implicit non-nullability for fields yet, and have combined non-nullability of parameters and method return values with automatically generated runtime checks via https://github.com/Fody/NullGuard. This makes the otherwise heuristical feature safe because if some code breaks the contracts, e.g., due to unannotated code or a null value escaping the incomplete control flow analysis, it is stopped at a defined point in the code. And in contrast to performing runtime checks whenever a non-nullable item is read, it's much less expensive and often nearer to the place that caused the null value to leak into the non-nullable context.

So, in essence, I'd highly suggest combining the feature in this proposal with automatically generated runtime checks in defined places. For example, when a method is called (for arguments), when a method returns (for return values and non-nullable fields assigned in the method), and when a constructor returns (for field assignments).

Of course, I know that this is neither easy, nor perfect, especially with fields of non-nullable types:

  • Due to the initobj semantics of value types, there is no suitable place to insert runtime checks for non-nullable fields of value types - maybe reference type fields within value types should be required to be declared nullable?
  • It's also a problem with arrays of non-nullable types because they're expensive to check at runtime. I'm not sure what's the best way of dealing with this. Maybe arrays of non-nullable reference types should also be forbidden? (Higher-level collection types could be implemented using arrays of nullable types; runtime checks in their APIs would ensure null values wouldn't be able to creep in.)
  • For constructors and members assigning fields, it's complicated. When constructors leak a this reference (e.g., by storing it in some global variable or by calling other instance methods on the same object) before initializing all non-nullable fields, a warning (or even compiler error) could be produced. When this occurs after all non-nullable fields have been initialized - or when an ordinary method reassigns a non-nullable field -, the corresponding runtime check could be moved up to that point.
  • There's no good way to prevent finalizers and virtual methods called from base ctors from seeing null values in non-nullable fields, even with runtime checks.
    -- For finalizers and virtual methods, I think one could live with that. Finalizers already have so many special rules, and one hardly ever needs to write one anyway. And virtual method calls from base ctors already need to deal with unfulfilled invariants defined by subclass ctors, so allowing nulls to creep in might be okay.

Despite runtime checks having their own set of problems, I think that reliability is one of the most important aspects of non-nullable reference types. Without reliability, the feature could actually cause more bugs than before.

@dsaf:

Is it really that better than a set of analyzers? I guess at least it will be "standard" and come for free.

I think it's better if (and only if) implemented in a reliable way :)
In that case, it's better because it has a standard syntax, it's used by many library authors (including the BCL), and it can express (non-)nullability in places such as variable declarations or cast expressions, where it can't be expressed without language support.

@olmobrutall

@danieljohnson2 Thanks for your feedback!

We could say that it is not an error for an ordinary field to be null, but it is an error to observe the null. Null then means 'uninitialized', but that's okay as long as you do initialize before use.

I'm more and more thinking in relaxing the warnings for the non-nullable reference types. Let's face it, arrays, databinding, POCOs and object initializers, ORMs... all are fundamental use cases of C#, and will be benefited from a soft solution, but if you want a strong solution you have to write code the Haskell way.

Bare Minimum 👍 👍

Just being able to annotate types with ? to detonate intended nullability, and store this information in the metadata using a NullableAttribute, just like DynamicAttribute does it, will already provide some real value without creating any new problem:

  • Developers will be aware of the intent of the APIs and data structures.
  • ORMs and UIs tools could be smarter and require less configuration.

Warnings

Let's see what warnings we can think of that will create more solutions than problems.

To not-nullable

string a = null; //implicit conversion from null to string (not-nullable)
string a = (string)null; //OK

public string Bla()
{
     return null; //implicit conversion from null to string (not-nullable)
}

string? b = "hi"; 
string a = b;  //implicit conversion from string? to string (not-nullable)
string a = (string)b; //OK

void Print(string s)
string? b = "hi";
Print(b); //implicit conversion from string? to string (not-nullable)

string? b = "hi";
b.Length //implicit conversion from string? to string (not-nullable)

While the first ones look more appealing, as you move down they become more annoying to use. I'm worried than using explicit nullable types will be so tiresome that people will just use not-nullables, since they are more friendly, familiar and can contains nulls anyway. A good casting operator makes a big difference here.

To-nullable

string a = "hi";
if(a != null) //warning CS0472: The result of the expression is always 'false' since a value of type 'int' is never equal to 'null' of type 'string?'
{

}

I think this warning should not be made, since the "always 'false'" part is just a lie. 👎

Conditional operator

string? c = Rand()? "hi" : null //CS0173 Type of conditional expression cannot be determined because there is no implicit conversion between 'string' and '<null>'

Isn't this warning super-pedantic?

Even for 'real' Nullable<T> should have been solved since C# 2.0

int? c = Rand()? 1 : null //CS0173 Type of conditional expression cannot be determined because there is no implicit conversion between 'int' and '<null>'

Non-nullable initialization

For local variables is trivial: just like any other variable.

string a;
a.Length; //CS0165  Use of unassigned local variable 'a'

But for members is more tricky:

class Person
{ 
   public string Name;
}

var p = new Person();
p.Name.Length; 

Solving this will be harder, requiring definitely assignment analysis and will still have a lot of holes. Also will require users to create constructors or provide absurd default values. POCOs, object initializers, DataBinding and ORMs will be severely constrained if implemented.

Is better to have a soft and useful solution that communicates intent without creating too much problems, than a Haskell-wannabe that is not going to fit in the language, and has holes anyway.

Finally, for arrays the problem is impossible.

string[] str = new string[100];
//stuff hapens
str[2]; //No way we can know if this value has been initialized.

Casting operator

A simpler operator to cast to not-nullable could be helpful, especially for big generic types, but its hard to come up with a good syntax.

  • Postfix ! is confusing:
!Customers!.Any(c => !c.Name!.Length != 0)
  • Prefix (!) is too cumbersome
!((!)Customers).Any(c => !((!)c.Name).Length != 0)
  • Maybe just notnull(...)
!notnull(Customers).Any(c => !notnull(c.Name).Length != 0)
  • Pretend is a Nullable<T> 👍
!Customers.Value.Any(c => !c.Name.Value.Length != 0)
  • Do nothing
!((List<CustomerEntity>)Customers).Any(c => !((string)c.Name).Length != 0)
  • Or...just let the implicit conversion and make the feature documentation-only?
!Customers.Any(c => !c.Name.Length != 0)

as operator

I've just come up with one reflection about as operator:

object obj = "string";
string s = (string)obj; //OK
string s = obj as string; //Warning! now as returns string?
@olmobrutall

@fschmied, how are you encoding POCOs with your Resharper solution in practice?

  • You add the constructor?
  • Make the properties nullable?
  • Or take advantage of the holes in the analyzer?

Also, when you enabled NullGuard, how many real bugs you found and how many you create?

I assume that you and you're team have a lot of pride for your code and take the necessary time to improve it, but other shops with a more... client/provider relationship with MS will be disappointed if VS2017 introduces 400 new warnings, and 5 new bugs of it's own. Even if is helping to prevent dozens of current bugs and hundred of future ones.

@fschmied

@olmobrutall We don't, since we use implicit not-null and NullGuard only for parameter and return types.

Also, when you enabled NullGuard, how many real bugs you found and how many you create?

For non-nullable parameter types, it's hard to say as we already hat manual argument checks in place. I can say, though, that there are often situations while writing code where the nullability analysis warns us of a potential bug. How many of those would've escaped our attention - I don't know. We haven't seen a new bug introduced by this tooling during the last 10 months.

By enabling non-nullable return types on our existing code base, we've found a few actual issues where we didn't handle null values correctly. We don't know if we created any new bugs yet - our experience with implicitly non-nullable parameter types causes me to hope that we didn't.

In general, the experience was very smooth - we had to add a number of annotations and assertions, but it wasn't that much work actually. But yes, you're right, we have a very high test coverage and put a lot of thought into the quality of our code, so we're likely not a representative team for the whole customer base of Visual Studio. Without coverage, I'd see turning those features on for an existing code base as quite risky. I don't see as much risk on a new code base, though.

@ndykman
ndykman commented Oct 23, 2015

While I get the demand for this feature, the myriad complications just don't seem worthwhile versus a CodeContracts approach or going wholesale and introducing records (ala F#). It seems that for interoperability with existing code libraries you have to treat a T! into a T at some point and you are back to static analysis to determine if a instance of T is never null.

Sure, nulls are annoying, but there are plenty of special cases that are also annoying. Take collections that don't allow indexes < 0. There's a lot of those. Do we need to introduce a int+, long+ syntax for those cases? And so on. Sure, Dependent Types are great, but we know that's an unsolvable problem in general. I'd much rather the emphasis be placed on static analysis and contract rewriting. While I know that the CodeContracts project comes with it's own issues, it's really a more general way of addressing these issues in a standard way.

@yaakov-h

Take collections that don't allow indexes < 0. There's a lot of those. Do we need to introduce a int+, long+ syntax for those cases?

Well, we do have unsigned integer types, which - ignoring their C-based legacy - is effectively a way of specifying a very particular constraint on a number.

While I know that the CodeContracts project comes with it's own issues, it's really a more general way of addressing these issues in a standard way.

Code Contracts is non-standard, not always enforceable, and requires third-party tooling for both the producer and the consumer. It's effectively an optional add-on for .NET.

Integrating into the language and compiler toolchain would make it much simpler to to declare and comply with a particular API's nullability rules, and would likely result in safer code for everyone.

@Joe4evr
Joe4evr commented Oct 24, 2015

how are you encoding POCOs with your Resharper solution in practice?

  • You add the constructor?
  • Make the properties nullable?

This is really the crux of the matter, IMO. I'm relatively sure static null analysis is gonna break down around POCOs, and thus, many application/database models will not co-operate very well. A constructor is near useless, because these POCOs are pretty much always instantiated via Object Initialization or Reflection. Making all properties nullable would work, but then you get into the same Option hell I've read about Swift. Plus, this will just look incredibly silly if you ask me

[Required]
public string? FirstName { get; set; } //Even if you get what it means, it still looks silly

The main reason given for this feature is "to express developer intent", but I say that this example does not express the developer intent too well, if at all.

And the size of these models could make it a real chore. I've recently been working on a new project and the design documents lists almost three dozen tables with many of them having more than a dozen or two dozen fields. And this might even be considered an average sized model.

If you ask me, >90% of the value of this compile-time nullability checking will be at locals and method parameters. On class properties, it would also be a very good fit on classes that do something and have a constructor that sets these properties up (like a Controller), but not on POCOs which just need to hold data.

Because of this, I'd like to suggest adding a new attribute, named maybe [ImplicitlyNullableProperties] or something similar, and the compiler would behave as if all reference type properties of the class it's applied to are declared with nullable behavior when nullability checking is enabled.

[ImplicitlyNullableProperties]
public class Person
{
    [Key]
    public int Id { get; set; }
    //nothing happens around struct properties,
    //since they have a default value to begin with

    [Required]
    public string FirstName { get; set; }
    //compiler behaves as if this was declared as
    //a string? and gives the appropriate 'could be null'
    //info/warning to the dev at call-sites
}

If you're talking about developer intent, this would display the intent very clearly, while still having the compiler co-operate with the dev (instead of working against them) and not looking as silly as the "required-nullable property" example.

@yaakov-h

I don't see any advantage in an [ImplicitlyNullableProperties] attribute over simply declaring each property as nullable.

I also don't see the problem with required-nullable. That simply states that whilst the property can be null at any arbitrary point during the execution of the program, at a certain point (i.e. whatever validates the POCO based on attributes) that value must be non-null. Attributes are not part of the class, they're just annotations. Thus, it's exactly the same as it is today where all reference types are implicitly nullable, except the developer (or an auto-generating tool) has explicitly marked it as nullable.

@psmacchia

@ndykman While I know that the CodeContracts project comes with it's own issues, it's really a more general way of addressing these issues in a standard way.

I'd say forget about Microsoft Code Contracts library. We did the mistake to rely on this library, and now we are stuck with this buggy library that just don't work with Visual Studio 2015, more than 3 months after it turned RTM!

have a look at critics, we are not the only ones in this sad situation https://visualstudiogallery.msdn.microsoft.com/1ec7db13-3363-46c9-851f-1ce455f66970

@roji
roji commented Oct 26, 2015

@psmacchia, CodeContracts does have VS2015 support and works well, check out their github releases page. Whether it's a good idea to start off a new project with CC is another question, as the project doesn't seem to be a very high priority for Microsoft.

@yaakov-h

@psmacchia I had the same issue, and had to maintain a private fork to get a proper VS2015-compatible release.

@roji It claims to have VS2015 support, but it has quite a few VS2015- and Roslyn-related bugs. Some have been fixed in master, but there's no new release yet.

Either way, I don't think it makes sense from any perspective to fob this issue off to Code Contracts or another external party. This ought to be a core feature.

@psmacchia

@roji

Microsoft/CodeContracts#169 (comment)

7 days ago I wrote

Nevertheless installing this fixed version still ends up in VS2015 with the error:*

The command ""C:\Program Files (x86)\Microsoft\Contracts\Bin\ccdocgen.exe" "@obj\Debug\MyDllThatUsesMSCCccdocgen.rsp"" exited with code 255.    MyDllThatUsesMSCC

@tom-englert wrote: there is no fix committed yet for this issue,,,

@yaakov-h This ought to be a core feature. I 100% agree,

The 28/07/2015 public version should not claim it supports VS2015 because it just not, this would save a lot of time and disappointment to users.

https://visualstudiogallery.msdn.microsoft.com/1ec7db13-3363-46c9-851f-1ce455f66970

@tom-englert

@psmacchia If you like to support the projects you could help with testing: https://github.com/tom-englert/CodeContracts/tree/Issue169-3
This now should contain the fix.

@fschmied

See also #119, a proposal to bring method contracts into the language.

@olmobrutall

@yaakov-h, I also don't see any point in [ImplicitNullabePropertiesAttribute], it's a bad solution but the problem remains:

If is encouraged to create POCOs with not-Nullable reference types then a non-Nullable reference type will mean just "I promise I'll fill this property... If I've time".

It is a very week promise, but is probably the best solution nevertheless, a more strong promise, even based on warnings, could be annoying to use.

@psmacchia

@tom-englert I tried again without success, see my answer on the thread Microsoft/CodeContracts#169

@yaakov-h

@olmobrutall the issue I have with enforcing non-null on POCO properties is that they can often be in an invalid state when new, or in the middle of a calculation. It's one of the few places I find the "implicitly unwrapped optional" in Swift to be a good construct.

@olmobrutall

Hey! I didn't know that swift has the concept of implicitly unwrapped optionals. Of course the guarantees for optionals in swift are going to be more strong so they need the concept more.

I see three options for C#:

Follow swift lead:

  • T? is a nullable reference that requires checking,
  • T is strict non-Nullable reference. Is not watertight, but you can expect warnings to get pedantic and maybe even runtime checks. Mainly use for method parameters and return types.
  • T! is a permissive non-Nullable reference. There is an implicit contact between the library and the consumer that the thing should be not-null, but some warnings are disabled to make it more usable in situations like POCOs properties and fields.

Invert swift solution

  • T will be permissive not-Nullable, used in POCOs and legacy code.
  • T! will be a more whatertight version, with errors and exceptions.
    This has the benefit of maiking the legacy code easier to migrate and the semantics easier to understand (I see ! more like the opposite to ? than a watered down version). On the other side the confusion with swift will be big, and the more strict you make the code, the more ugly it becomes.

Week and simple solution:
Take into advantage the fact that the solution is not going to be watertight anyway, to simplify the solution, T will be permissive not-Nullable and the number of warnings will be reduced globally.

@yaakov-h

There is an implicit contact between the library and the consumer that the thing should be null

I think you meant either "should not be null", or "can be null".

The issue I have with the "Invert swift solution" is that as more and more C# code becomes null-safe over time, ! will be absolutely everywhere.

@danieljohnson2

@olmobrutall, I don't think there's going to be much support for the Swift way; that adds a new type distinction (and probably requires CLR changes because of it), yet it remains incompatible with essentially all existing C# code- worst of both worlds. I'm afraid.

The 'inverted Swift' way is more viable, but there are serious technical obstacles. Honestly @MadsTorgersen proposal at the top is problematic also: T? for reference types probably will require some CLR support. Even Nullable<S> required CLR support, after all. But T! would be worse, since that type has no natural default value.

@yaakov-h

@danieljohnson2 the "Swift way" was already discussed in #5033 as an option, including "This seems like a good idea." There's some support already.

T! has a natural default - null, the same as T?. T has no natural default though, best I can tell - what would that be? I dare say you would be required to set it in a constructor or inline assignment (readonly T thing = new Thing()), or it would follow the same initialisation rules as a struct, which may break compatibility.

@scottsteesy

I support the idea of a non-nullable type, and on variables of that type, but I do not think that being able to say you want a variable of just any type to be non-nullable is realistic.
For a non-nullable type all we need is a static constructor MyType!() which will define the default value for the non-nullable type. And just doing so could conceivably automatically create a property named Default that returns that value.
Then because the type is now non-nullable compatible if you define a variable of MyType if would have an initial value of null, but a variable of MyType! would get an initial value equal to the reference returned by the special constructor.
This will obviously require CLR support to have the special constructor called, just as the class's static constructor does now.
Code that is non-nullable unaware would consume all code same as today, because it will just use nullable variables. Conversely, with non-nullable aware code consuming a regular old nullable class we could either have something like NonNullable which could work on any class that has a default constructor, to wrap it in a lightweight non-nullable class, or the developer would have to write their own wrapping class in order to make a non-nullable compatible type.

@danieljohnson2

I think we're in for a lot confusion with the T! syntax; Swift uses this for a "nullable T with dynamic checks", which is just T in current C#. But previous discussion has used T! for "mandatory T".

The big problem with a real "mandatory T", however you spell it, is that it has no reasonable default. The CLR thinks it can zero initialize things like structure and arrays and, by doing so, set everything to its default value.

It would be a big thing to make the CLR initialize certain arrays and fields to some other value, and it would mean that mandatory references can exist only for special types that define these default values. Many classes couldn't really do that. This would really limit the usefulness of the feature, while adding a big pile of complexity.

If you allow "mandatory T" for any reference type T, you have the hard problems- how do you ensure initialization for arrays, structures, and finalizers? If the answer is "you don't", then your "mandatory T" isn't really mandatory anymore.

I still favor an approach that restricts these things to places where we could really enforce it, but that's a pretty big restriction in its own right.

@qrli
qrli commented Oct 28, 2015

I think, non-nullable fields/variables are not a must. The top usage of non-nullable is method/property signature and the code analysis atop of it.

As long as we have it, the nullability of related local variables and fields can be deducted by code analysis, so that we don't have to add special syntax to them.e.g.:

string! GetFoo() { ... }
...
var str = GetFoo();
var parts = str.Split(','); // compiler/analysis-tool knows str is not null.

This could be a small and easier step to start with. And it also keeps most of C# code as it is.

@yaakov-h

@qrli That already exists in the form of Code Contracts, but it's nowhere near as neat a solution as the proposition detailed here to allow variables/parameters to be declared as "definitely some sort of value", as opposed to the traditional "some sort of value, or maybe nothing. Who knows?"

It's a smaller step, it's an easier step, it's been done before, and it's not good enough for massively-large systems.

@qrli
qrli commented Oct 28, 2015

@yaakov-h IMO, the issues with Code Contracts are mostly with its library approach and the tools, and it has many more complicated features which are not sweet yet because of its larger goal.

While at the same time, Resharper's simpler annotation approach gains more practical usage. So, it is one way that we can get the relative mature part into production, leaving other features for future improvement.

The similarity to Code Contracts is also a good thing, because the BCL is already annotated with contracts, which can be used directly, instead of modifying whole BCL to support non-nullable.

@adaptabi

I know that there are a couple of languages that are already implementing this syntax ?. such as Groovy, Swift or .? Ruby.

However, I have already specified an even better solution for ES6 / ES7 / TypeScript and even Ruby.

So instead of using either ?. or .? I propose to simply use .. double dots.

Here is an adaptation from the ES7 / TypeScript proposal (so the below code applies to C# world instead of javascript, but however the idea is the same)


This would be amazing operator!! Especially for ES6/ES7/TypeScript languages -- and why not C# ?

var error = a.b.c.d; //this would fail with error if a, b or c are null or undefined.
var current = a != null && a.b != null && a.b.c != null ? a.b.c.d : null; // the current messy way to handle this
var csharpQuestionMark = a?.b?.c?.d; // The current c# 6.0 way of handling the above mess with no errors

However I propose a more clear one - as not to confuse ? from the a ? b : c statements with a?.b statements:

var doubleDots = a..b..c..d; //this would be ideal to understand that you assume that if any of a, b, c, d is null the result will be null, otherwise d

Two dots, means if its null stop processing further and assume the result of expression is null. (as d would be null).
Perhaps for non-reference types (class instances) and not nullable variables it should mean default(typeof D) assuming d is of type D

And hence .? would not be misunderstood with int? as it applies to both int? (defaulting to null and int defaulting to 0.

Two dots make it more clear, more visible and more space-wise so you understand what's going on.

This is not messing with numbers too - as is not the same case e.g.

1..ToString(); // works returning "1"
var x = new X();
x.1 = 2; //fails currently - since 1 is not a valid C# identifier
x.y = new Y(); //works currently 
var current = x.y.z; //works
var missing= x.w.y; //throws exception since w is null
var assume= x != null && x.w != null ? x.w.y : null; // works but very messy

What do you think folks?

P.S. foo?.bar syntax would work too.

However the using both current ? : operator and ?. might be very confusing on the same line.

e.g. using ?.

var a = new X { x = new Y { y = 1 }, w = null };
var b = condition ? a?.x.?y : a?.w?.y;

as opposed to double dots ..

var a = new X { x = new Y { y = 1 }, w = null };
var b = condition ? a..x..y : a..y..z;
Which one does look more clear to you?
@MgSam
MgSam commented Oct 29, 2015

@dotnetwise I think your suggestion is 6 months too late. C# has already added the ?. operator and there is likely 0 percent chance that they'll add another operator that does exactly the same thing.

@HaloFour

@dotnetwise What makes .. better than ?. other than the fact that you've spammed the same request over a slew of other languages, none of which have adopted your suggestion? As @MgSam had already stated, and you acknowledged, C# 6.0 shipped with ?. which behaves identically to your suggestion. You're considerably too late to throw in your opinion to have that changed, and the chances that a second operator would be adopted when it provides no verbosity or clarity benefits is nil. There are also other proposals (#155) to consider .. for method cascading taking inspiration from the language Dart.

@adaptabi

C# 6.0 is not RTM and even if would be, they can still consider suggestions, if they are better.
I have not spammed anything - is just that it is applicable to all js, Typescript, C# - and perhaps Ruby.

It's not an alternative of ?. - but perhaps is a clearer replacement of it - since it makes code much clearer (in my opinion anyways).

  • It doesn't mix int? nullable types with silent default operator (a?.b?.c) making you to believe that only applies to nullable types and classes, since could work for value type members too - by always applying default(T) - whereas T is the type of the last property in the chain.

e.g.

class X { public Y y { get; set; } } 
class Y { public int? n { get; set; } int i { get; set; } }
var x = new X();
var n = x..y..n; // null default(int?)
var i = x..y..i; //zero default(int)

//whereas 

var n = x?.y?.n; // null since y is null and n is int? anyways
var i = x?.y?.i; // null - or 0 - NOT CLEAR! since ? should return null if y instance is null, as by design ? is used for nullable properties
  • It doesn't mix ? : ternary decision operator with silent default operator (?. that is mixing now vs .. that would not mix)
var a = new X { x = new Y { y = 1 }, w = null }; // which one does look more clear?
var b = condition ? a?.x.?y : a?.w?.y;   //this one?
var c = condition ? a..x..y : a..w..y;       // or this one?
@gafter
Member
gafter commented Oct 29, 2015

@dotnetwise

C# 6.0 is not RTM

C# 6 was RTM in VS2015 this past July.

VS2015 Update 1 is not RTM yet. It won't include any language changes.

@asik
asik commented Oct 29, 2015

This is a well-balanced proposal that provides immediate value to both existing and new code, without introducing much new syntax at all. I am no language designer but after dabbling extensively in both C# and F#, I have a strong feeling that this is the most that can be done in C#; in fact it's surprisingly ambitious.

In particular this is much better than requiring special syntax for non-nullable types like T!, because it reflects developer intent most of the time: most of the time you don't want null, and when you do want null, you'd like the compiler to help you track down potential nullrefs. Having to explicitely specify non-nullability doesn't achieve this - it has to work without you thinking about it.

My only concern is where this proposal may clash with developer intent. I'm thinking about types that map to JSON, where JSON semantics are that everything is optional by default... now you'll have to write ? everywhere. I'd say it's for the best, but maybe some devs will be ticked off and disable the warnings. It's a wide-ranging, optional feature that'll certainly be polarizing and cause pains esp. when trying to enable it on large existing code bases. Lots of projects compile with warnings as errors...

Just food for thought, the feature has great value and I certainly hope it'll make it in C#7 anyhow.

Re.: arrays of non-nullable refs: F# deals with this by syntax (Array.zeroCreate is pretty explicit), not much C# can do about that, not a reason to drop the feature either.

TL;DR: love default non-nullability and explicit nullability, explicit non-nullability wouldn't be nearly as useful, please implement as-is!

@olmobrutall

@asik

I agree 100% with what you said. The original proposal from Mads has a very simple syntax and is practical and ambitious at the same time.

I'm only concern with the list of warnings that will be enabled for explicitly Nullable and not-Nullable references. It is tricky to get a height signal-to-noise ratio, specially when fields or properties are involved.

For example:

if(order.State == OrdeeStatr.Cancelled)
    return order.CancellationDate

CancellationDate is nullable, but I know that it will be set when the order is Cancelled, and I don't want the compiler getting on the way with warnings.

Maybe getting a list of useful warnings end up being to hard and the best solution is just to not implement any. The important thing is that libraries can express nullability in a way that is shown in IntelliSense, and can be reflected by libraries, anything else is secondary.

@yaakov-h

@olmobrutall that would be a good use of postfix-! as outlined in #5033.

@asik
asik commented Oct 30, 2015

@olmobrutall @yaakov-h Better yet, use a hierarchy of record types and pattern matching. #206 So your order can be one of several types: if it's a "cancelled", it has a cancellation date. No need for nullability there.

Anyway, didn't want to suggest getting rid of warnings, they're instrumental to the proposal I think; less warnings will mean more nullrefs. Just concerned over how that'll play out on real world existing code bases.

@MiddleTommy

What is wrong with using Method Parameter Constraints to ease in Non-nullability.

public void Foo(string bar != null)
{
......
}

http://visualstudio.uservoice.com/forums/121579-visual-studio-2015/suggestions/5441721-method-parameter-constraints?tracking_code=e20d2783c87f8f4af488d3973b7ca1bd

@yaakov-h

@MiddleTommy That looks like just a different syntactical take on #119, which still doesn't provide for some cases of non-nullable (e.g. variables within a method).

Ultimately every time you use a reference type there's an implicit "or null" that follows you around. Killing that at the source is, in my experience, the best way to actually solve the problem, instead of just applying guards everywhere whilst maintaining the fundamental cause.

@olmobrutall

@yaakov-h, I agree that more warnings could be possible for nullable types if there is a good notation to cast it back to not-nullable.

Postix ! is tempting but will be confusing with prefix not. What about some virtual .HasValue and .Value members? This way the simetry between value and reference types will be even bigger.

The sweet spot will be when users could forget of a type is a value type or not. At least if you are not programming video games and concerned about GC.

@olmobrutall

@MiddleTommy, the problem of constraints is that they are too general for the compiler to understand.

Dynamic languages are Turing complete languages that the an interpreter can evaluate but not analyze (halting problem, etc...).

Static languages add a rigid structure on top (namespaces, types, members) that is easier for the compiler to verify and provide tooling.

Constraints are in between. They use a powerful general notation for conditions, but then they underperform in compiler analysis and tooling.

For the particular case of not-nullable references, as yaakov noted, the problem is even greater because the default is wrong. Most of the a reference type is used, null is an error. So you will be adding the notation everywhere.

@yaakov-h
yaakov-h commented Nov 1, 2015

@danieljohnson2 C#6's null-conditional operator makes it easier in cases where you don't care that a value may or may not be null. It doesn't help particularly much when you do care, which is what this proposal is all about.

@danieljohnson2

@olmobrutall, I don't like the idea of compiler-injected Value and HasValue properties. If it really works like a struct, we'd be writing s.Value.This and s.Value.That all the damn time. It's the worst thing about current C# nullable-struct implementation.

I'd endorse having Swift-like postfix !, or some syntax like that, and I'd like to use in with nullable structs too.

I think it's worth noticing that the history null in C#, after 1.0 anyway, has been about making null more usable; S?, x ?? y, x?.y and x?[y] and all additions to the language that make null easier to use rather than trying to eliminate it. You might say they encourage it's use as a sentinel value.

We may now feel that null is Just Plain Wrong, but there's a lot of history here and I doubt the broader C# programming community avoids null as much as we might like them to.

@olmobrutall

I don't have any problem with nulls or all the new operators, but we're discussing an operator for asserting not-null on nullable types.

The postfix ! operator could be confusing with not.

One thing that I like about HasValur is that it keeps the logic positive:

if(name) //great
{...}

if(name.HasValue) //good
{...}

if(name != null) //bad
{...}

if(!string.IsNullOrEmpty(name)) //horrible
{...}

And also, will be great if genetic code could be use T? Independently of whether T is a class or a struct. Of course this will be no problem if postfix ! is added in both.

@bbarry
bbarry commented Nov 1, 2015

Consider that you might not need a postfix operator in that condition at all:

if (order is Order { State is OrderState.Cancelled, CancellationDate is DateTime d})
{
    return d;
}

Similarly in today's code:

if (name is string)
{...} // name is not null inside that if

though if name is a field, it is possible another thread might have modified it after the if and before the first usage of it inside the if block... Enter patterns again:

if (name is string n)
{...}

n is a private variable to the method scoped to the true block of the if statement and cannot be null.

@olmobrutall

@bbarry, this approach works (awesomely) as long as the type is simple, but look at this:

Dictionary<string, List<DateTime>>? GetDictionary(){...}

var dic = GetDictionary()


if(dic.HasValue) //good
{...}

if(dic != null) // bad
{...}

if(dic is Dictionary<string, List<DateTime>> dic2) //horrible

Maybe should also work with var?

if(dic is var dic2) //too implicit?
{...}
@yaakov-h
yaakov-h commented Nov 2, 2015

@olmobrutall

I don't have any problem with nulls or all the new operators, but we're discussing an operator for asserting not-null on nullable types.

We already have an operator for asserting not-null on nullable types - .. But it's doing double-duty. I don't see the problem with moving that assertion to an explicit operator.

I agree with you on involving pattern matching for single variables though. Swift has the if-let which can be somewhat cumbersome in complex methods, but has the effect of converting T? into T for the scope of a single block which is good.

I'd personally prefer an option to go the other way, something like:

T? Foo() { /* ... * }

void Bar()
{
    var foo = Foo();
    if (foo == null)
    {
        return;
    }

    // Down here, `foo` is assumed to be of type `T`, not `T?`, without requiring further checks.
}
@olmobrutall

Good point. I also prefer to get rid of the exceptional cases at the beginning without adding one level of indentation. Maybe this should work:

void Method(string? name)
{

   if(!(name is string n))
      throw new ArgumentException(nameof(name));

    OtherMethod(n);

}

void OtherMethod(string s) {...}

Note that in the example, the new is operator declares a variable in the else case/ rest of the block.

Also note that if you pass a string? to a method expecting string you should become a warning, and explicitly assert that the reference is not null, either by using .Value, postfix ! or other alternative, but double-duty . won't help you in this case.

@gafter
Member
gafter commented Nov 20, 2015

See also #227

@gafter gafter assigned mattwar and unassigned MadsTorgersen Nov 20, 2015
@gafter gafter added the 3 - Working label Nov 20, 2015
@olmobrutall

Nice to see that this is on the table again!

@gafter
Member
gafter commented Nov 21, 2015

FYI this has been in active development. #6476 is the first step.

@alrz
Contributor
alrz commented Nov 25, 2015

Currently, operator obj as T would return a T while it has to be T? I don't know how this going to be handled but to be more explicit I think the following would be a good option to consider,

// returns T?
var result = obj as? T;
// returns non-nullable T, throws if cast is invalid
// equivalent to (T)obj, useful with try? (see below)
var result = obj as! T;

I can imagine proposals like #5961 and #6563 can be useful in this context,

// if list were null foreach won't throw
foreach?(var item in list) {}

// nullable dereference, throw if null
var result = obj!; 

// if obj is null F() won't be called and null returns.
var result = try? F(obj!);

// if obj wasn't T, F() won't be called and null returns
var result = try? F(obj as! T);

// this can be used with null-coalescing operator
var result = try? F(foo?.Bar!);

// note: it's not proposed that try? would catch regular exceptions

// if obj was null switch returns null,
// so no further case* would be needed
// this also can help with nullable ADTs
obj switch?(case ...)

etc.

@yaakov-h

Currently, operator obj as T would return a T while it has to be T? I don't know how this going to be handled but to be more explicit I think the following would be a good option to consider,

as is the safe cast - cast if you can, return null if you can't. I can't imagine obj as TNonNullable would be valid in any case - it's already invalid for structs, triggering a CS0077 "The as operator must be used with a reference type or nullable type" error.

I don't see any value in having a second way to do (TNonNullable)unknownObj which also throws. Keeping it as-is, - i.e. as returns object of nullable target type or null, cast returns object of target type or throws - still makes the most sense to me going forwards AFAICT.

@alrz
Contributor
alrz commented Nov 25, 2015

@yaakov-h Yes it's the safe cast, meaning that the object returned is a "nullable reference type" which in this proposal indicated by T? (I'm not talking about structs). As I said, as! will be useful with try?.

@yaakov-h

@alrz Wouldn't your as! be the same as (obj as T?)!? I don't understand based on your example how as! would be valid in any other context than as a parameter for a function (also how would it work with multiple parameters, only some of which are as! or postfix-!?)

@alrz
Contributor
alrz commented Nov 25, 2015

@yaakov-h I do prefer obj as! T instead of (obj as T?)! I mean, that's a lot of punctuations! as! returns a non-nullable T and for try? — if this is what you're asking,

try? F(arg!, arg2!);
// equivalent to
if(arg != null && arg2 != null) F(arg, arg2);
// if F() only accepts non nullable types
// you can pass regular args for other parameters

see #5961.

@HaloFour

I seriously doubt that #5961 is on the table even with the syntactic changes. Way too much voodoo in one place. Given that as very explicitly is capable of producing null there is no reason to have multiple flavors of it, and even less reason to have multiple flavors of is considering that x is type is never true if x is null.

@alrz
Contributor
alrz commented Nov 25, 2015

@HaloFour Currently try? and as! exist in Swift and Kotlin respectively, no voodoo. I'm just trying to combine these two ideas. And as produces regular types e.g. T while in this topic, nullable types proposed as T?, which is my motivation for this as I said in the first comment.

@HaloFour

@alrz I don't see any mention of as returning a non-nullable type per this proposal. It wouldn't make any sense for it to.

@alrz
Contributor
alrz commented Nov 25, 2015

@HaloFour You say that the following means the existing as would return a T??

Add a notion of safe nullable reference types T? to the language, in addition to the current ones T which are from now on non-nullable.

@HaloFour

@alrz Considering that the result of as is supposed to be null when the cast fails, yes.

@alrz
Contributor
alrz commented Nov 25, 2015

@HaloFour The thing that I liked from those languages is that nullables are not just for variables and types (as it is proposed in this thread), you can easily flow your nullable variables throughout the language constructs without any explicit null-checking.

@yaakov-h

@alrz and that has become problematic in other implementations e.g. Objective-C. It's not something I'd want to see ported to c#.

@olmobrutall

Is this still on the table for c# 7.0? I didn't saw anything at Build 2016 for c#, just for Typescript Microsoft/TypeScript#7140

@HaloFour
HaloFour commented Apr 6, 2016

@olmobrutall

#2136 (comment)

No, non-nullable reference types are not dead. They are actively being worked on. They simply won't be done in time for C# 7. We need a lot of experience with a prototype to make sure we have the right shape of the feature.

@tumtumtum

Putting in my vote for implicitly unwrapped optionals and an unwrap operator (the equivalent of ! in Swift).

Would be good if ! could be used with Nullable value types too. x! is more succinct than x.Value because Value is a valid name on any type whereas ! would only work on nullable/optionals.

@aluanhaddad
aluanhaddad commented May 31, 2016 edited

I think it's worth noticing that the history null in C#, after 1.0 anyway, has been about making null more usable; S?, x ?? y, x?.y and x?[y] and all additions to the language that make null easier to use rather than trying to eliminate it. You might say they encourage it's use as a sentinel value.

Actually this proposal if anything cements null as the de-facto way to indicate an optional value thereby encouraging its use. That said, I'm not sure that's a bad thing.

@Eirenarch

How is this proposal supposed to work with patterns like

var customer = GetCustomerFromDB();
ThrowNotFoundIfNull(customer);
//do something with customer here

What happens if a method validates that the argument is non-null and throws an exception otherwise? Will we still get warnings simply because we didn't put the if statement with the null check in the same method?

@NightElfik

Such "check" method should take nullable argument and return not-nullable.
You can use it like this:

var customer = ThrowNotFoundIfNull(GetCustomerFromDB());

Here, customer is guaranteed to be not null. Or even better as extension
method (we use this now):

var customer = GetCustomerFromDB().CheckNotNull();

However, if there is real possibility that the entry is not in the DB, I
would not be throwing exceptions around if possible... Just my view.

Cheers,
Marek

2016-06-29 9:50 GMT-07:00 Stilgar notifications@github.com:

How is this proposal supposed to work with patterns like

var customer = GetCustomerFromDB();
ThrowNotFoundIfNull(customer);
//do something with customer here

What happens if a method validates that the argument is non-null and
throws an exception otherwise? Will we still get warnings simply because we
didn't put the if statement with the null check in the same method?


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#5032 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/AAuMyoOkHX7PriA-JIm7ocickGJTUWXOks5qQqJGgaJpZM4F4VK2
.

@Eirenarch

@NightElfik I know how to make the warning go away but I think your first suggestion is significantly harder to read. The extension method idea is a cool solution I had not considered but still I dislike how the proposal will force a particular code style that is not inherently better.

While I agree that throwing exceptions like this is in general not a good idea the real world case where I use code like this is Web API which provides convenient HttpResponseExceptions which allow you to respond with something that is not in the signature of the method like 404 from a method that returns a Customer. It is a kind of abuse of exceptions but when I think about it the idea is in practice great.

@series0ne

Off the back of this discussion, I would like to direct you to my issue about type aliases, which might add some additional insight into how non-nullable types might be implemented.

[(https://github.com/dotnet/roslyn/issues/12874#issuecomment-239618500)]

@Obertyukh
Obertyukh commented Oct 4, 2016 edited

2 Major Issues is:

  • Back Compatibility wish C#6 code and or libraries
  • Default values for NonNullable fields

Solution Proposal wish non null guarantee:

Back Compatibility wish C#6 code and libraries can be made using Compiler ability to emit method names (for public methods of public classes) with special characters in it so we can not see those methods in code.

So if this method:

public String StrMagic( String s1, String s2 )
{
    //Do some magic and return str
}

C#7 compiler will emit like:

public String StrMagic( String s1, String s2 )
{
    if(s1 == null || s2 == null)
        throw new ArgumentException();

    return StrMagic>>@@NonNull@@( s1, s2 );
}
public String StrMagic>>@@NonNull@@( String s1, String s2 )
{
    //Do some magic and return str
}

than pre C#7 compiler will always use undecorated methods and never see decorated.
And C#7 compiler and VisualStudio can show to us exactly this decorated method instead of Undecorated and apply all rules of non nullability.

So there we always have checks for null for old code and no checks for code from C#7 and all this is transparent for programmer and just work properly.

Library programmer have guarantee of nun nullability in his code and C#6 customer will have Exceptions on invalid arguments pass.

Default values for NonNullable fields can be made by allowing partial initializing of objects (this is matter for loading grapth of object from saved state when we can not construct objects with correct links in time of construction). Partially initialized Objects must have special type like ( Part<SomeType>, or SomeType^ ) and constructor that construct partially initialized Object must return this type so we know that this object dont safe to pass to functions that expect fully initialized object but we can create functions that can operate on partially initialized object, this will be used mostly in loading Code or in some Big and Hard object initialization routine. But after full initialization we can get fully InitializedObject from partRefernce like this:

SomeType^ myObj = SomeType( ); //partially init because need to set non null fields
myObj.field1 = new Object();//Whatever
myObj.Name = "SOme Name";//Whatever

SomeType finalObj = myObj.FinalizeInitialization( );
//or
SomeType? finalObjNullable = myObj.FinalizeInitialization( );

FinalizeInitialization is special function generated by compiler that checks all nonnullable fields to be non null and return initialized reference to SomeType object;

from now we can pass this object to regular code that have guarantee of full initialization of object.

After this additions we can have not nullability warnings but true and safe nullability errors.

@yaakov-h
yaakov-h commented Oct 8, 2016

@Obertyukh The mangled method names aren't needed. AFAIK, overloads can also be based on modopts and modreqs.

In theory, we could have nullability annotations turn into modopts and older compilers would recognise them (and pass nulls), or modreqs and older compilers I believe cannot handle them.

Using modopts or modreqs has been discussed before and I think the biggest argument against it is a complete break of compatibility (particularly for the BCL), but if we could have the compiler generate backwards-compatible stubs, that might help.

e.g.

public void Foo(string a, string? b) { .... }

could get transformed into the following (or similar):

public void Foo (nonnull string a, string b) { .... }

public void Foo (string a, string b)
{
    if (a == null)
        throw new ArgumentNullException(nameof(a));
    Foo(a!, b);
}
@series0ne

I recently watched a video (The Future of C# and VB.NET) where the presenter was a member of the .NET team and was discussing the proposal for non-nullable reference types.

The proposal was

string? x = null; // nullable string
string x = ""; // non-nullable string

This presents a breaking change, and I for one do not think that this is a good idea!

My proposal for the roslyn and .NET guys would be to introduce an operator to handle this suitably.

string x = null; // nullable string, the way it's always been
string! x = ""; //non-nullable string.

I think this flows nicely with nullable value types too.

DateTime? dt; // nullable datetime.
DateTime dt; // non-nullable datetime.

string x; // nullable string.
string! x; // non-nullable string.

BANG (!) brings your attention to THIS IS NOT NULLABLE!

@HaloFour
HaloFour commented Nov 9, 2016

@series0ne

That was actually the original form of this proposal. See CodePlex: non-nullable reference types (the one billion $ mistake) and #227.

The argument behind the approach above is that the vast majority of the time today the developer intends for a parameter to be non-nullable. Requiring an explicit decoration for non-nullable references would require that this vast majority then requires additional adornment while the less common case would not. This would likely hamper the adoption of the feature.

There is the additional argument that the single ? suffix would unify the syntax between nullable references and nullable value types.

The changes above don't affect the type system. Existing code would compile and run just as it always did. Non-nullable references would instead of a feature that can be enabled that would perform flow-analysis and warn on potential null dereferencing. It doesn't have particularly sharp teeth but would hopefully at least aid the developer in avoiding common mistakes.

Note that I'm mostly rehashing the conversation here, not advocating for any particular position.

@Joe4evr
Joe4evr commented Jan 16, 2017 edited

I would like to add something that I've thought of today, which I think could really compliment this feature (and reduce usage of !. to an absolute minimum for opt-ins).

Reading alternate null checks

Some developers use an additional property that indicates whether another property is null or not.

public class Foo
{
    //it is expected that SomeBar can be null
    public Bar? SomeBar { get; }

    //if true, SomeBar is not null
    public bool HasABar { get; }
}

As it stands, the compiler does not know the relation between these two properties, which is why the dammit-operator (!.) is to be introduced alongside this feature.

Proposal: Attribute for these properties/fields

API shape:

[AttributeUsage(AttributeTargets.Property | AttributeTargets.Field)]
public sealed class ChecksNullOnAttribute : Attribute
{
    public string PropertyName { get; }
    public bool TrueMeansNull { get; } //flag for if inverted behavior is wanted
    
    public ChecksNullOnAttribute(string propname, bool trueMeansNull = false);
}

Usage:

[ChecksNullOn(nameof(SomeBar))]
public bool HasABar { get; }

Now the compiler knows there's a relation between HasABar and the property named SomeBar that can be used during the static analysis.

void M(Foo foo)
{
    if (foo.HasABar) //compiler now knows this is equal to `foo.SomeBar != null`
    {
        Console.WriteLine(foo.SomeBar.ToString()); //safe to dereference `SomeBar`, no `!.` required
    }
}

Open questions

  • Is it possible/worthwhile to make the compiler check that the attribute is used only on bool fields/properties?
  • Should the compiler auto-emit this attribute if it encounters the most straightforward of cases:
    bool PropNotNull => SomeProp != null; / bool PropIsNull => SomeProp == null;?
@aluanhaddad
aluanhaddad commented Jan 16, 2017 edited

@Joe4evr that is a good suggestion but unfortunately it introduces a race condition.

@gafter
Member
gafter commented Jan 17, 2017

I would like to add something that I've thought of today, which I think could really compliment this feature (and reduce usage of !. to an absolute minimum for opt-ins).

@Joe4evr This issue does not propose any ! syntax for opt-ins, What feature is your proposal intended to complement?

@Joe4evr
Joe4evr commented Jan 17, 2017 edited

that is a good suggestion but unfortunately it introduces a race condition.

Not if the properties are immutable, but the compiler has no special knowledge of that either. ¯\_(ツ)_/¯
If the property was mutable, shouldn't if (foo.SomeBar != null) { /* deref SomeBar here */ } have the exact same race condition?

This issue does not propose any ! syntax for opt-ins, What feature is your proposal intended to complement?

Really? I could swear that every time I hear Mads talk about Nullable Reference Types, he adds that the plan is to include a "dammit-operator" !. so that when you opt-in to get this static analysis, you can override a compiler warning in cases where a potential null dereference is guarded in a way the compiler can't see.

My proposal is intended to compliment Nullable Reference Types as a whole by making the compiler aware of at least one alternate null checking pattern, so that there can be less cases where !. is genuinely warranted, increasing the source code compatibility when a developer enables these warnings.

@aluanhaddad

Not if the properties are immutable, but the compiler has no special knowledge of that either. ¯_(ツ)_/¯
If the property was mutable, shouldn't if (foo.SomeBar != null) { /* deref SomeBar here */ } have the exact same race condition?

I believe the idea is that if (foo.SomeBar != null) would introduce a temp referring to foo.SomeBar. An immutable property may still change over time.

Really? I could swear that every time I hear Mads talk about Nullable Reference Types, he adds that the plan is to include a "dammit-operator" !. so that when you opt-in to get this static analysis, you can override a compiler warning in cases where a potential null dereference is guarded in a way the compiler can't see.

Yeah I saw a talk where he discussed the "damnit" operator. Perhaps that corresponds to a different issue, but I also thought it was in this issue. I can't find it however...

@Joe4evr
Joe4evr commented Jan 17, 2017

An immutable property may still change over time.

Doesn't setting a readonly backing field after an object has been initialized require full trust Reflection? Which isn't impossible, but seems like the most extremely rare of cases where this could go wrong to me. Otherwise the use of readonly is completely undermined and you might as well teach developers that it's not going to help thread-safety at all.

@gafter
Member
gafter commented Jan 17, 2017

Yes, I forgot the dammit operator x!.y is under consideration as part of this, to silence the compiler when the compiler thinks x might be null and the programmer is OK with a NullReferenceException if it is.

@mattwar
Contributor
mattwar commented Jan 17, 2017

@Joe4evr How common of a pattern is this? It does seem worthy to look for other patterns that prove null/non-null-ness and potentially make the compiler aware of them if they are common.

@Joe4evr
Joe4evr commented Jan 21, 2017

@mattwar Well to be honest, it's not like I surveyed any other developers, I just thought that this pattern seems useful enough that others might have adopted it independently.

Although I have seen a variation in a lib I'm heavily using at the moment, where a message can be put through a pipeline and every step of the pipeline returns a Result object indicating if that step succeeded or not. All these Result objects are consolidated under an IResult interface looking something like this:

public interface IResult
{
    public bool IsSuccess { get; }
    public string ErrorReason { get; }
}

Which gets used like so:

var result = await ProcessMessageAsync(msg);

if (!result.IsSuccess)
{
    //Log result.ErrorReason however you like

    //inside of the pipeline the same thing is checked and
    //either returns `result` or `continue`s if inside a loop
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment