New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Non-nullable type annotation syntax #27231

Open
munificent opened this Issue Sep 2, 2016 · 115 comments

Comments

Projects
None yet
@munificent
Copy link
Member

munificent commented Sep 2, 2016

This is the tracking bug for the real implementation of the postfix ? syntax to mark a type annotation as nullable as part of supporting non-nullable types (#28619). See the "Syntax" section of the proposal for details.

Since we're still only in the prototype phase, consider this bug to be on hold. If we decide to go ahead and ship a real implementation, I'll update this.

Flag

While this is being implemented, to ensure we don't expose users to inconsistency in our tools, it should be put behind a flag named nnbd.

Tracking issues

@crelier

This comment has been minimized.

Copy link
Member

crelier commented Sep 2, 2016

Just curious, why was '?' chosen? It seems very counter-intuitive to me, since a question mark implies some kind of optionality. In this case, we rather want to express a restriction. A more assertive exclamation mark would be more appropriate, in my opinion, but I did not study possible grammar conflicts, if that is the reason of this choice.

@lrhn

This comment has been minimized.

Copy link
Member

lrhn commented Sep 3, 2016

The question mark means that the type is nullable/optionally null/union of type and Null, and its absence means that the type is just itself.

@donny-dont

This comment has been minimized.

Copy link

donny-dont commented Sep 3, 2016

Any chance union types will come along with this? They seem to be natural fits.

@tatumizer

This comment has been minimized.

Copy link

tatumizer commented Sep 3, 2016

Questions:

  • How to declare the type of optional parameters of a function? With question mark or without? By definition, they accept null, so we have to add question marks everywhere?
  • How to declare that, var is nullable? Will it be "var? x" ? Can we write "dynamic? x". (IMO both forms look stupid: "dynamic" means "whatever" already, and var is a synonym of "dynamic")
  • Opinion: too many question marks already. Maybe we don't need to make declaration cryptic. e.g. use some keyword instead? (though "nullable" is probably too long)
  • As soon as we have non-nullable types, we need "is-not-null" assertion operator. E.g. double exclamation mark: expression "x!!" means "I'm sure x is not null, but if it is, throw NPE". This is necessary to prevent monstrosity like:
if (x != null) {
  return x;
} else {
   throw new NullPointerException();
}
@dynaxis

This comment has been minimized.

Copy link

dynaxis commented Sep 4, 2016

This seems just an experiment for now. But I'd like to note that there might be needs for addition of APIs to the standard library. For instance, Kotlin has filterNotNull and mapNotNull on nullable sequence or collection, which return one with non-nullable element type. Dart's Stream and collections would be better to have such new APIs. They can definitely be implemented in user codes, but are really clumsy to do so.

@crelier

This comment has been minimized.

Copy link
Member

crelier commented Sep 4, 2016

@lrhn Oh, I see. The title is misleading and the text does not clarify. It should actually be "Nullable type annotation syntax" instead of "Non-nullable type annotation syntax".

@zoechi

This comment has been minimized.

Copy link
Contributor

zoechi commented Sep 5, 2016

@tantumizer

"is-not-null" assertion operator.

This is shorter

return x ?? (throw new NullPointerException);

See also #24892

I find x!! a bit cryptic.
If the return type for the function that contains this code is a non-nullable type it should throw in checked mode anyway and make the check and throw redundant.

@lrhn

This comment has been minimized.

Copy link
Member

lrhn commented Sep 5, 2016

@tatumizer

Good questions, some not decided yet.

Optional parameters: If nothing else changes, you'll have to make them nullable by adding a ? to the type. After all, that is their type. We may consider allowing you to make them non-nullable if they have a non-null default value. If we do that, we should probably also change how default parameters work by making an explicit null argument be equivalent to not having the argument at all. There is some design-space here to explore.

For a nullable var, it's not really a problem as we are moving towards Strong Mode where var means to infer the type. You can write dynamic? but since null is already assignable to dynamic, I think dynamic? will just mean the same as dynamic (so we may disallow it entirely, there is no reason to have two ways to write the same thing). Same for Object and Null.

Too many question-marks. Likely true, but they are consistently about null (except for the original ?: conditional operator).

A not null assertion (that is: take nullable value, throw if it's null, otherwise we know the type).
That could just be x as Foo if the type of x is Foo?. We will have to change as to not allow null for non-nullable types, you would then write x as Foo? to get the current behavior.
We are also considering shorthands to quickly coerce a value to an expected type. Maybe x! will check for null if x has type Foo? and it's used in a position where a (non-nullable) Foo is needed. Maybe that's just being too clever by half.

@tatumizer

This comment has been minimized.

Copy link

tatumizer commented Sep 5, 2016

@zoechi, @lrhn: "return x ?? (throw new NullPointerException)" is technically the same as "return x!!", but human meaning of it is different. There are cases where you want to say: "I'm SURE x is not null, that's it. And if I'm wrong, it's a bug!" Example:

map["foo"]="Hello";
String foo = map["foo"]!!;  // I'm sure it's not null, I've just assigned it!

Do you really believe the following is an equivalent replacement? To me it looks absurd:

map["foo"]="Hello";
String foo = map["foo"]  ??  throw new NullPointerException();

Although the example might look artificial, the phenomenon is quite common. While programming in Kotlin, I found myself using !! more often than I'd like to, for variety of reasons. It would be a good idea to create a summary of these reasons - scenarios when you are forced to declare nullable vars even though you are sure (and can prove!) that before first use, they are already initialized. Writing anything more verbose than x!! would be just adding noise to the program. (Maybe we can discover some deficiencies of the language by analyzing these reasons).

@lrhn: same argument applies to x as Foo. It's a very roundabout way of saying "I'm sure x is not null", though technically the same. (Occam's razor doesn't apply to the language b/c language is about meaning - the notion which is (almost by definition) impossible to formalize).

BTW, just to avoid misunderstanding, x!! is defined in Kotlin as (x != null ? x : throw NullPointerException()) - so if you are considering one exclamation mark for this, that's fine, it's just Kotlin believes two are better. I have no opinion on that.

@zoechi

This comment has been minimized.

Copy link
Contributor

zoechi commented Sep 5, 2016

If you have

String foo = map["foo"];

then ?? throw new NullPointerException(); is redundant because String already is non-nullable and checked mode should throw.

@tatumizer

This comment has been minimized.

Copy link

tatumizer commented Sep 5, 2016

@zoechi: it depends on how nullable types are implemented in dart. I think the whole purpose of them is to make things like String foo = map["foo"] illegal. Strong mode compiler must complain here. Anything less than that undermines the whole idea of nullables/non-nullables IMO.

@floitschG

This comment has been minimized.

Copy link
Contributor

floitschG commented Sep 5, 2016

As a clarification: if we add only ? (meaning "this type is nullable"), then the non-? type must be non-nullable.
This means that String foo = map["foo"] would statically not be allowed, unless we have implicit downcasts for nullable types, too. (It wouldn't be that awkward, since A is pretty much a subtype of A|Null).

If we don't allow downcast assignments, then there must be a way to go from nullable to non-nullable.
We have the following choices:

  • as, conditions and ifs promoting the type: String foo = map["foo"] ?? ... basically falls into that category. This requires no change to the language.
  • a special operator: String foo = map["foo"]!! which checks for nullability. I guess this would also include !!. as in map["foo"]!!.bar(). As Lasse suggests, this could potentially be more general, coercing more than just nullable types. For example, it could replace the implicit downcast.

@tatumizer: I'm definitely interested in the reasons for why you had to use !! more often than you thought.

@tatumizer

This comment has been minimized.

Copy link

tatumizer commented Sep 5, 2016

@floitschG General description of the situation is "you can prove x!=null, but compiler can't".
Sometimes you can restructure your program so that compiler can figure it out, but as a result, your code becomes worse, so you choose a lesser evil. Because this is subjective, I'm reluctant to use my own code for demonstration - instead, I will use dart's own code. Opened dart sdk code at random place, and in 30 sec got a first example (https://github.com/dart-lang/sdk/blob/master/sdk/lib/collection/linked_list.dart):

E get last {
    if (isEmpty) {
      throw new StateError('No such element');
    }
    return _first._previous; 
}

You know _first is not null here, but compiler doesn't. With !! operator, last line will be written simply as
return _first!!._previous. Without !!, this cute piece will become a mess. I will post more examples later, but the search algo is simple: find any uninitialized var anywhere, and see how it's used later in the code. There's a good chance it's used without explicit "if (x != null)" guard, because it's clear from the context that it can't be null (modulo bugs).

Added later: turns out, isEmpty is used as a guard everywhere in linked_list. You need to write quite a few of !! to pacify compiler. That's basically the effect I wanted to demonstrate: the number of !! in the code is higher than one can naively expect.

@tatumizer

This comment has been minimized.

Copy link

tatumizer commented Sep 5, 2016

Second source I opened (randomly) was splay_tree. Another program - another "universal guard", this time, it's method _splay, which returns -1 in case _root == null. Throughout the code, program "knows" that when _splay returns non-negative value, it's safe to access _root. But compiler, most likely, won't know it. You will have to use _root!! in every place where you currently have _root.

@tatumizer

This comment has been minimized.

Copy link

tatumizer commented Sep 6, 2016

Another question: consider generic class Foo. What is the meaning of E? Is it a nullable type? Or only non-nullable type? If the latter, we would need Foo<E?> to denote nullable E, but this idea is probably not tenable. So E is nullable. Suppose we want to say that in Foo<E>, E must be NOT nullable. How to write this restriction? Maybe Foo<E!!> :) Or what?

@eernstg

This comment has been minimized.

Copy link
Member

eernstg commented Sep 6, 2016

Currently the nullability experiment is about syntax only.

In Patrice Chalin's proposal (
https://github.com/chalin/DEP-non-null/blob/master/doc/dep-non-null-AUTOGENERATED-DO-NOT-EDIT.md),
nullability is a property of a type (T? is essentially a shorthand for T | Null: B.3.1), and Foo<C> would be the generic class Foo instantiated
with an actual type argument C, which is a non-null type, assuming that
C is the name of a class. The type argument of Foo<E> where E is a
type variable of an enclosing generic entity such as a generic class Bar<E> .. could be a nullable type or a non-null type, depending on the
instantiation of that generic class of which this is an instance. For
instance, if we consider an instance of Bar<int?> and Bar contains an
occurrence of Foo<E> then E is a nullable type, namely int?. If you
want to make sure that a given type is non-null then you may or may not
have an operator for it: For instance, Foo<E!> could be a Foo of
non-null E, which would in this case be int (int? with the ?
stripped off).

When a type variable can stand for a nullable as well as a non-null type it
is necessary to be a little bit smarter in code generation, such that it
will work in both cases, with good performance.

Because of complexities like this, there are quite a number of issues that
we haven't decided on, so we can't promise anything specific about these
design choices. But currently we won't even promise that there will be
anything like a ! operator for stripping ? off of a given type, we are
just looking for a syntax that will work well for ?.

On Tue, Sep 6, 2016 at 3:14 AM, Tatumizer notifications@github.com wrote:

Another question: consider generic class Foo. What is the meaning of E?
Is it a nullable type? Or only non-nullable type? If the latter, we would
need Foo<E?> to denote nullable E, but this idea is probably not tenable.
So E is nullable. Suppose we want to say that for Foo, we accept generic
type parameter E only if NOT nullable? How to write this restriction? Maybe
Foo<E!!> :) Or what?


You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#27231 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AJKXUhdiciCSV8iIfIi2pqtBdS5pIByZks5qnL5kgaJpZM4J0Ezv
.

Erik Ernst - Google Danmark ApS
Skt Petri Passage 5, 2 sal, 1165 København K, Denmark
CVR no. 28866984

@munificent

This comment has been minimized.

Copy link
Member

munificent commented Sep 6, 2016

Replying to a few random things that weren't already covered:

Just curious, why was '?' chosen?

It's the same syntax used to represent nullable types in C#, Ceylon, Fantom, Kotlin, and Swift.

Any chance union types will come along with this?

We are interested in exploring union types too, but they're a big feature with a lot of consequences, so we aren't working on them right now. There's only so many hours in the day. :)

It should actually be "Nullable type annotation syntax" instead of "Non-nullable type annotation syntax".

I considered that, but the obvious response is that Dart 1.0 already has nullable type annotations—all type annotations are nullable. So this is really about adding a way to express non-nullable types. And the way we do that is by adding new syntax for nullable types and changing the existing syntax to mean non-nullable.

I find x!! a bit cryptic.

Me too, but something along these lines might be worth doing. A big part of why we are doing an experiment around non-nullable types is to get answers to usability questions like this. How often do users need to assert that they know something isn't null when the compiler doesn't? We're hoping to implement enough of the static checking to be able to answer that confidently.

String foo = map["foo"]!!;  // I'm sure it's not null, I've just assigned it!

Another part of this experiment is determining how we need to change our core libraries to make it pleasant to work with non-nullable code. In this case, I think Map should support two accessor methods. One returns V? and returns null if the key isn't present. The other returns V and throws if the key isn't found. In this case, you'd use the latter and wouldn't need !!.

There's some interesting API design questions about which of those operations should be [] versus a named method, which is more commonly used, etc. but we need to start trying things out to get a feel for that.

Opened dart sdk code at random place, and in 30 sec got a first example (https://github.com/dart-lang/sdk/blob/master/sdk/lib/collection/linked_list.dart)

That entire class was designed around the idea that E is nullable. Once that assumption is no longer true, there are probably systemic changes you could make to the entire class so that you don't need to sprinkle !! everywhere.

It's also probably true that core low-level collection classes like this will bear the brunt of the manual null checking. They are closer to the metal and need to do things a little more manually. Higher-level application code should hopefully be able to use non-nullable types more easily.

Another question: consider generic class Foo. What is the meaning of E?

I have an answer in mind for this, which I think lines up with Patrice's proposal, but I haven't verified that or written mine down in detail yet. (That's why this issue is about non-nullable syntax. :) ).

The short answer is that here, since you have no constraint, E can be instantiated with either a nullable or non-nullable type. If E was constrained to some non-nullable type, it could only be instantiated with a non-nullable type.

Suppose we want to say that in Foo, E must be NOT nullable. How to write this restriction? Maybe Foo<E!!>

If you want to say E is some specific type, you can give it a constraint and that implicitly constrains it to be non-nullable too, unless the constraint is nullable:

class Point<T extends num> {
  T x, y; // <-- These are non-nullable.
  Point(this.x, this.y);
}

new Point<int>(); // Fine.
new Point<int?>(); // Error! Constraint is non-nullable.

class Pointish<T extends num?> {
  T x, y; // <-- These may or may not be nullable.
  Pointish(this.x, this.y);
}

new Pointish<int>(); // Fine.
new Pointish<int?>(); // Also fine.

I don't currently plan to support a constraint that says, "The type must be non-nullable, but I don't care anything else about the type." I could be wrong, but it doesn't seem very useful to me.

@tatumizer

This comment has been minimized.

Copy link

tatumizer commented Oct 4, 2016

Before this proposal, the syntax for types was indistinguishable from the syntax for referring to a class

You can always keep it this way. Like you currently do for function types via typedef. It's a structural type, so tuple type can be introduced exactly like that - you always have a simple name. And this is true for the union types and any of the hypothetical panoply, too. This is just a very convenient syntactic device to keep syntax simple.

Your argument is not very convincing IMO because it lacks a massive use case that could justify it.
But it's your call.

@chalin

This comment has been minimized.

Copy link
Contributor

chalin commented Oct 4, 2016

@eernstg @munificent: no worries; the original proposal is a bit lengthy, but nullity (as we all know) can be tricky to do "right" in the context of Dart.

All: thanks for keeping the discussions moving forward.

@tatumizer

This comment has been minimized.

Copy link

tatumizer commented Oct 6, 2016

@munificent : is this a valid syntax:

var map=new Map<String?, String>();

Assuming the answer is "yes": suppose I want to implement my own map that only works with String keys, but I want null to be a valid key. How to write definition of this generic class:

class StringMap<K extends WHAT???, V> {...}

Is there a syntax for that?

@munificent

This comment has been minimized.

Copy link
Member

munificent commented Oct 6, 2016

@munificent : is this a valid syntax:

var map=new Map<String?, String>();

Yup, that's fine.

Assuming the answer is "yes": suppose I want to implement my own map that only works with String keys, but I want null to be a valid key. How to write definition of this generic class:

class StringMap<K extends WHAT???, V> {...}

Well, in this case, the answer would be just class StringMap<V> { ... }. String is a sealed type, so there's no reason to make it generic on the key type. The only valid type argument would be String. :)

But let's say you want to define a number set that can be used with ints or doubles and you also want null to be a valid member. You would do:

class NumSet<T extends num> {
  void add(T? value) { ... }
  void remove(T? value) { ... }
  // etc...
}
@tatumizer

This comment has been minimized.

Copy link

tatumizer commented Oct 6, 2016

String is a sealed type, so there's no reason to make it generic on the key type

I made it generic precisely to show that it can handle null keys - so that it can be instantiated with both String and String?

var map=new StringMap<String?,String>();
var map1=new StringMap<String,String>();

Anyway, you answered the same question with num example.

@munificent

This comment has been minimized.

Copy link
Member

munificent commented Oct 6, 2016

I made it generic precisely to show that it can handle null keys - so that it can be instantiated with both String and String?

Ah, sorry. I thought you meant you wanted it to always support null keys, regardless of the type parameter type. In my NumSet example, you can not do NumSet<int?> because the constraint is num, which is non-nullable. If you want to allow that, you'd do:

class NumSet<T extends num?> { ... }

This means that in the body of NumSet, T is now a nullable num type, so before you can call methods on it, you have to test for null first.

@tatumizer

This comment has been minimized.

Copy link

tatumizer commented Oct 6, 2016

This was precisely my question, and now I understand that extends num? is allowed in this context

class NumSet<T extends num?> { ... }

Then let's come back to the earlier question:
Which of the following definitions are syntactically correct:

class Sorter<T extends Ordered?<T>> {}
class Sorter<T extends Ordered<T?>> {}
class Sorter<T extends Ordered?<T?>> {}

You said only the second one. But why? Your example with num shows that it would be fine (syntactically) to say

class Sorter<T extends Ordered?> {}

Which makes all 3 definitions syntactically correct by implication, no?

@munificent

This comment has been minimized.

Copy link
Member

munificent commented Oct 6, 2016

But why?

The other two would be:

class Sorter<T extends Ordered<T>?> {}
class Sorter<T extends Ordered<T?>> {}
class Sorter<T extends Ordered<T?>?> {}

The ? was in the wrong place. (Also, I think when I first commented I may have forgotten that our tentative plan does allow ? in constraints.)

@tatumizer

This comment has been minimized.

Copy link

tatumizer commented Oct 7, 2016

Thus, we established that both following definitions are valid:

class MyGenericClass<T extends Object> {...}
class MyGenericClass<T extends Object?> {...}

(The latter can be instantiated with any concrete type, but the former - only with non-nullable concrete types)

The question is: what is the default? When I write

class MyGenericClass<T> {...}

is it equivalent to the first, or to the second one?

@munificent

This comment has been minimized.

Copy link
Member

munificent commented Oct 7, 2016

class MyGenericClass<T extends Object> {...}
class MyGenericClass<T extends Object?> {...}

These would be equivalent since Null is a subtype of Object. Null|Object collapses to Object.

(The latter can be instantiated with any concrete type, but the former - only with non-nullable concrete types)

My current idea for the semantics does not give you a way to express "any type, but not nullable". If the only thing you know about the type parameter is that it's Object, it's not the end of the world to permit null—it supports all of the methods that Object does. It is an object.

All of these questions would be answered by a proposal for the semantics, which I have not yet written down. This issue is just for the syntax.

Do you think we can table this discussion until I have a real proposal to go on? Right now, we're sort of doing a breadth-first traversal through the semantics one comment at a time, which isn't an efficient use of either of our time.

@eernstg

This comment has been minimized.

Copy link
Member

eernstg commented Oct 7, 2016

If we wish to support a distinction between Object with and without null we can remodel the type hierarchy such that Null is not a subtype of Object any more, but we haven't decided that this is a good idea.

@tatumizer

This comment has been minimized.

Copy link

tatumizer commented Oct 7, 2016

Do you think we can table this discussion until I have a real proposal to go on?

Sure. Thanks. Looking forward to semantics proposal :)

@donny-dont

This comment has been minimized.

Copy link

donny-dont commented Jan 14, 2017

@munificent the link to the dart2js bug is wrong up top. Off by one error 😉

@osdiab

This comment has been minimized.

Copy link

osdiab commented Jun 27, 2018

Is this not happening? :/ I find this to be a really useful concept for ensuring type safety of my code, and along with the lack of union types is making me wary of switching from TypeScript/React Native to Dart/Flutter in future projects.

@munificent

This comment has been minimized.

Copy link
Member

munificent commented Jun 27, 2018

It's not happening yet. Moving Dart to strong mode is a necessary pre-condition for non-nullable types. We are doing that in Dart 2. We hoped to get non-nullable types in at the same time, but it proved to be too big of a change to fit into 2.0, so it's going to have to wait until a later version.

@vanesyan

This comment has been minimized.

Copy link
Contributor

vanesyan commented Sep 9, 2018

So now as Dart 2.0 is released would be there any changes so it finally get happened?

@munificent

This comment has been minimized.

Copy link
Member

munificent commented Sep 10, 2018

Now all that's left is to design and implement non-nullable types, figure out a migration plan, add language features to make them more usable, etc. :) Basically, we have to do all the work. It's a giant feature.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment