Null Safety: Sound non-nullable types with incremental migration (NNBD)

[leafpetersen]

This is a solution proposal for #71. The proposal covers the intended final language design, and the language, tooling, and ecosystem support required for the migration.

• Roadmap
• Feature specification
• NNBD related issues

Topic specific discussion issues for resolution:

• Throw/catch static errors
• Null assertion operator
  • Resolved, yes, !.
• Should the NNBD opt in also remove implicit casts
  • Resolved, yes
• Is Object nullable?
  • Resolved, no
• Do we need T!?
  • Resolved, no
• Do we need generic constraints?
  • Resolved, no
• Grammar issues
• Fields initialized after creation
  • Resolved, we will use late
• Chaining null-aware operators
  • Resolved, yes.
• Default parameter values
• Can object methods be called on nullable types
  • Resolved, yes.
• Precedence of await vs !
  • Resolved, ! binds more tightly.
• Querying the state of late variables
• Null short-circuiting spread operator

[leafpetersen]

I've added a draft of the roadmap for this proposal, linked from the header comment.

[yjbanov]

@jmesserly I'm not sure how to constructively move the NNBD discussion from #125 here, because the my suggestion is all about linking the two issues. But I'll try:

I suggest that, as a first step, we introduce sound non-nullable types as part of immutable types (#125). This has several advantages:

• It would be a non-breaking change: immutable types is a new feature so having them NNBD doesn't break any existing code.
• It could have the biggest performance impact: if non-nullability is implemented such that many highly-used types can be inlined, such as double, int, bool, enum, String, we could significantly improve memory layout efficiency.
• It would give us a chance to get the syntax and semantics right and educate users before rolling NNBD to the "normal" classes, which does require a bigger migration.

[jmesserly]

IMO, linking the two proposals would be counterproductive. We are working hard on NNBD, there is a lot of design & planning work going on. There aren't any major open questions on syntax or desired semantics. Linking the two would prevent us from pushing forward on NNBD, because immutable types has a lot of open design questions, and we'd have to delay work on NNBD until that's resolved.

(Also it could be confusing if different parts of the language treat int as having different non-nullability. It would be really nice if int always means non-null, and int? always means mean nullable. The NNBD proposal achieves that.)

[yjbanov]

That's fine too, it was just an idea. I don't have enough context to know how best to proceed, so if you believe that NNBD and immutables should proceed completely independently, then they should :)

[jmesserly]

Makes sense :). Yeah, we seem to be gathering a lot of momentum behind Leaf's proposal. It's really exciting! I think we'll be able to apply the many lessons of Strong Mode to NNBD, to make the migration as easy & rewarding as possible. (e.g. ideally it's non-breaking for code that isn't migrated. Until you turn on NNBD, you don't break. Once you turn it on, you have to fix some errors, but you gain some degree of null safety, even if everything else isn't migrated yet. Once everything in an app is migrated, then it has sound null safety. It's pretty neat.)

[leafpetersen]

I've updated the header with some sub-issues for discussing specific topics.

[leafpetersen]

Added links to two new discussion issues in the header comment.

[Levi-Lesches]

And then, they will just assume the analyzer is broken and add exclamation marks everywhere when using “foo”.

A very good point. I saw on some other issue that someone proposed to fix this by putting implicit !s during compilation, which defeats the purpose of null safety. I, for one, am a fan of #1201 as it entirely solves this problem.

[leafpetersen]

So, even with sealed classes, the cases where we could choose to promote an instance getter would still be fairly restricted — declared as a (final only?) field in the same class (or at least in the same library, so any breakage will still be immediately visible), and only using this. access

I don't think the this. access is necessary, if the class is sealed. In any case, while I agree that this is a limited case, I am still struck by the fact that this is pretty much exactly the case that Kotlin allows, and people don't seem to find this painful there.

[Timmmm]

I, for one, am a fan of "if-vars" as it entirely solves this problem.

I don't think it entirely solves the problem. How do you use if-vars for code like this?

if (field == null) {
  throw ...
}
var x = field.foo;

[Levi-Lesches]

The author of that proposal included a "negative if-var", which covers exactly that case. It would look exactly like you wrote:

if (var field == null) {
  throw "Error, not allowed to be null";
}
var x = field.foo;

The gist is that when checking to see if an if-var is null (or when using is!), instead of creating a new variable in scope of the curly brackets, it creates a new variable outside the scope of the if statement, and requires that the "then" block exit abnormally (throws, returns, break, continue, etc.). It's an interesting idea, even though I have trouble getting over the non-intuitive scope. See my comment there for more about that.

[derolf]

Honestly, I think the language should be designed in a way to avoid using the ! whenever possible!

If devs are forced into using ! at "strange" places and to need to scratch their head, they will scratch their head once or twice and then just put ! whenever the compiler complains.

This is why I wrote a linter for a bigger TypeScript project that disallowed the usage of ! completely.

[Timmmm]

I have trouble getting over the non-intuitive scope

Yeah that is confusing. And var field == null is quite close to var field = null which also seems confusing. I think I'd still rather give up field/getter symmetry.

This is why I wrote a linter for a bigger TypeScript project that disallowed the usage of ! completely.

I'd say you wasted some time writing a linter! In any case this isn't a problem in Typescript because Typescript's null checking isn't sound. They just ignore this problem. For example the following code type checks fine but will throw a type error at runtime.

let i = 1;

class Foo {
    public get maybeString(): string | null {
        return ++i % 2 === 0 ? "Dave" : null;
    }
}

function maybeStringLen(f: Foo): number {
    if (f.maybeString === null) {
        return 0;
    }
    return f.maybeString.length;
}

let x = maybeStringLen(new Foo());

[jodinathan]

I, for one, am a fan of "if-vars" as it entirely solves this problem.

I don't think it entirely solves the problem. How do you use if-vars for code like this?

if (field == null) {
  throw ...
}
var x = field.foo;

I really think that this is a case for a new local variable:

final notNullField = field;

if (notNullField == null) {
  throw ...
}
var x = notNullField.foo;

I am also in favor for simple type and non-null scoped var:

// not null scoped var
if (field != null use notNullField) {
  var x = notNullField.foo;
}

// type scoped var
if (field is SomeClass use fieldAsSomeClass) {
  print(fieldAsSomeClass.foo);
}

the semantics don't bother me as long as I have some simple and intuitive as above

[Timmmm]

I really think that this is a case for a new local variable

But that is what we have to do at the moment. The point of this thread is to find better solutions.

if (field != null use notNullField) {
  var x = notNullField.foo;
}

Another case if-vars don't solve is multiple chained conditions. E.g. in Typescript you can easily do this:

if (field !== null && field.subfield !== null && field.subfield.subsubfield !== null) {
  doStuff(field.subfield.subsubfield);
}

Would you advocate this?

if (field != null use a && a.subfield != null use b && b.subsubfield != null use c) {
  doStuff(c);
}

Surely we can do better than that!

[jodinathan]

I really think that this is a case for a new local variable

But that is what we have to do at the moment. The point of this thread is to find better solutions.

if (field != null use notNullField) {
  var x = notNullField.foo;
}

Another case if-vars don't solve is multiple chained conditions. E.g. in Typescript you can easily do this:

if (field !== null && field.subfield !== null && field.subfield.subsubfield !== null) {
  doStuff(field.subfield.subsubfield);
}

Would you advocate this?

if (field != null use a && a.subfield != null use b && b.subsubfield != null use c) {
  doStuff(c);
}

Surely we can do better than that!

TS is a great language and a very good evolution of JS stuff, but I don't think we should try to strictly follow it.

your example could be simple as:

if (field?.subfield?.subsubfield != null use c) {
  doStuff(c);
}

[Levi-Lesches]

And with if-vars (positive if-vars -- the thread seems to agree that negative if-vars are too messy for the moment), that would be:

if (var field?.subfield?.subfield != null) {
  doStuff(subfield);
}

Also,

This has now launched in tech preview

I just noticed that all 14 tasks at the top of this issue are complete. (Well, one is not checked off yet but the issue it's linked to is closed.) Does that mean this issue is ready to be closed? Oh how far it's come.

[derolf]

At least, for private final fields, it should be sound to propagate the null-safety:

class Foo {
  ...
  final Bar? _bar;
  ...
  void baz() {
    if (_bar != null) {
      _bar.somemethod(); <-- is always safe here
    }
  }
}

[jodinathan]

At least, for private final fields, it should be sound to propagate the null-safety:

class Foo {
  ...
  final Bar? _bar;
  ...
  void baz() {
    if (_bar != null) {
      _bar.somemethod(); <-- is always safe here
    }
  }
}

and if not overridden in any other class in the same file/part

[derolf]

@jodinathan

and if not overridden in any other class in the same file/part

I know, it's not perfect. But, I am currently migrating a bigger Flutter project with 150k Dart LOC.

The most annoying piece are exactly these constructs where you have final nullable fields. Typical pattern is a widget wrapping some arguments:

class Bla extends StatelessWidget {
  Bla({this.text});
  final String? text;

  @override
  Widget build(BuildContext) => text == null ? SizedBox.shrink() : Text(text);
}

[jodinathan]

@jodinathan

and if not overridden in any other class in the same file/part

I know, it's not perfect. But, I am currently migrating a bigger Flutter project with 150k Dart LOC.

The most annoying piece are exactly these constructs where you have final nullable fields. Typical pattern is a widget wrapping some arguments:

class Bla extends StatelessWidget {
  Bla({this.text});
  final String? text;

  @override
  Widget build(BuildContext) => text == null ? SizedBox.shrink() : Text(text);
}

I agree.
One of the reasons that we are not migrating to NNBD is the non-null promotion issue

[rrousselGit]

Sealed classes or structures would make this a lot less painful

The problem exists because of inheritance. By being able to store state in an inheritance-free manner, the type-inference could be much better

In the end it isn't just about null-testing here. All kinds of testing suffer from the same issue, such as:

final Object something
...

if (something is MyClass) {
  (something as MyClass).doSomething();
}

[derolf]

While migrating a 150k project, I run into the following problem:

///
void foo<T>(T bar) {
  switch (T) {
    case String:
      break;
    case double:
      break;
    default:
      throw Exception('Unsupported type $T');
  }
}

If T is nullable, it throws, so how can I add nullable Ts to the case?

[rrousselGit]

You can do:

Type _typeOf<T>() => T;

if (T == _typeOf<String?>())

That will require to not use a switch-case though

[lrhn]

@derolf You cannot add nullable types to the switch. Only constants can be used as switch case expressions, and there is no syntax for creating constant nullable type literals.

You'll have to use a non-constant approach instead.
One alternative:

Type typeOf<T>() => T; // Helper function.
final Type _nullableString = typeOf<String?>();
final Type _nullableDouble = typeOf<String?>();
void foo<T>(T bar) {
  var type = T;
  if (type == String || type == _nullableString || type == double || type == _nullableDouble) return;
  throw Exception("Unsupported type $type");
}

This will allow you to check against any type.

I'd probably rewrite it to avoid the Type equality checks. If you want to support both String? and String, maybe just check that T is a subtype of String?. Then I'd do:

bool _isSubType<S, T>() => <S>[] is List<T>;
void foo<T>(T bar) {
  if ((_isSubtype<T, String?>() || _isSubtype<T, double?>()) &&
      !(_isSubtype<T, Null>()) {
    return;
  }
  throw ....
}

[derolf]

@rrousselGit @lrhn thanks for the hints, I am using T.toString() and switch on it. After fixing ~4000 nullability issues, the app now runs. Btw I tried to use the "migration tool", but had to completely revert it because it was adding way too many "!" everywhere.

So, I am fixing the tests now.

Does anyone has an idea how to run tests from Visual Code while passing --no-sound-null-safety to Dart???

Options:

• Adding "@Dart = 2.11" to all tests -> doesn't work because the tests themselves are null safe
• Environment variable to turn on "no-sound-null-safety"?
• Something in pubspec.yaml to turn on "no-sound-null-safety"?
• Some option in the visual code plugin?

[derolf]

.vscode/settings.json:

{
    "dart.flutterTestAdditionalArgs": [
        "--no-sound-null-safety"
    ],
}

[Levi-Lesches]

You cannot add nullable types to the switch. Only constants can be used as switch case expressions, and there is no syntax for creating constant nullable type literals.

Was there talk about having something like T is Object? a while ago? (I get that it still wouldn't fit in a switch-case, just curious).

[olfek]

Hello, when is this feature being released?

[leafpetersen]

This feature has been released in the beta channel and should land in stable shortly.

[olfek]

@leafpetersen What is shortly? Few days? weeks? months? I am looking to start a new web service project in Dart.

[leafpetersen]

@olfek I suspect that the PMs would be sad if I stole their thunder, so I think I'll leave it at shortly. :) The feature has been released in a beta build, and the general cadence of stable releases is public info, so you can draw your own conclusions.

That said, I would discourage you from starting a new project with null safety enabled unless you're fairly confident that you will not subsequently need to take on a dependency which is not yet null safe. We are strongly encouraging folks to migrate in "waterfall" order (dependencies first). Starting with a null safe app and then adding a non-null safe dependency will require you to jump through some hoops and switch back to unsound mode. So if you're expecting to not have many package dependencies (or you can validate that all of your expected dependencies already have null safe releases), then by all means, grab a beta-channel SDK and go for it, but otherwise I might suggest waiting until the package ecosystem has a bit more time to migrate.

[mit-mit]

Please see this blog post for our latest update on migrating to null safety:

https://medium.com/dartlang/preparing-the-dart-and-flutter-ecosystem-for-null-safety-e550ce72c010

[mit-mit]

As of Dart 2.12 -- launched to the stable channel today -- null safety is now available!
https://medium.com/dartlang/announcing-dart-2-12-499a6e689c87

[gmpassos]

What a nice thread, 2+ years of hard work! I'm enjoying Dart Null Safety, it's the future of modern languages. ;-)