Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SEVERE: dartdevc can't compile closure/local function returning FutureOr<T?> #48631

Closed
gmpassos opened this issue Mar 21, 2022 · 19 comments
Closed
Assignees
Labels
area-front-end Use area-front-end for front end / CFE / kernel format related issues. type-bug Incorrect behavior (everything from a crash to more subtle misbehavior) web-dev-compiler

Comments

@gmpassos
Copy link
Contributor

I have isolated the bug with code examples to reproduce it at:
https://github.com/gmpassos/dev_compiler_test

The error happens with Dart 2.16.1 (current stable) and also Dart 2.17.0-182.1.beta (current beta). Today is 2022/03/21.

Specific code that triggers the compiler error:
https://github.com/gmpassos/dev_compiler_test/blob/master/example/example_error.dart

import 'dart:async';

void main() {
  _f1([123]);
}

FutureOr<T?> _f1<T>(dynamic o) {
  subF(e) {
    if (e != null) return e as T; // This is the line that triggers the bug.
    // if (e != null) return e as T?; // This version works.
    return _f2<T>(o);
  }

  return subF(o);
}

FutureOr<T?> _f2<T>(dynamic o) => o as T?;

Error:

$> dartdevc -o example/example_error.js example/example_error.dart
Warning: this 'dartdevc' script is deprecated and will be removed. 
More details at https://github.com/dart-lang/sdk/issues/46100.
We're sorry, you've found a bug in our compiler.
You can report this bug at:
    https://github.com/dart-lang/sdk/issues/labels/web-dev-compiler
Please include the information below in your report, along with
any other information that may help us track it down. Thanks!
-------------------- %< --------------------
    dartdevc -k arguments: -o example/example_error.js example/example_error.dart
    dart --version: 2.17.0-182.1.beta (beta) (Mon Mar 14 13:49:22 2022 +0100) on "macos_x64"

Unsupported operation: Undetermined Nullability
#0      ProgramCompiler._emitFutureOrType (package:dev_compiler/src/kernel/compiler.dart:3056:7)
#1      ProgramCompiler.visitFutureOrType (package:dev_compiler/src/kernel/compiler.dart:2954:11)
#2      FutureOrType.accept (package:kernel/ast.dart:11527:42)
#3      ProgramCompiler._emitType (package:dev_compiler/src/kernel/compiler.dart:2853:54)
#4      ProgramCompiler.visitFunctionType (package:dev_compiler/src/kernel/compiler.dart:3137:22)
#5      ProgramCompiler._emitFunctionTagged (package:dev_compiler/src/kernel/compiler.dart:2809:19)
#6      ProgramCompiler.visitFunctionDeclaration (package:dev_compiler/src/kernel/compiler.dart:4545:9)
#7      FunctionDeclaration.accept (package:kernel/ast.dart:10523:43)
#8      ProgramCompiler._visitStatement (package:dev_compiler/src/kernel/compiler.dart:3775:20)
#9      MappedListIterable.elementAt (dart:_internal/iterable.dart:413:31)
#10     ListIterator.moveNext (dart:_internal/iterable.dart:342:26)
#11     new _GrowableList._ofEfficientLengthIterable (dart:core-patch/growable_array.dart:189:27)
#12     new _GrowableList.of (dart:core-patch/growable_array.dart:150:28)
#13     new List.of (dart:core-patch/array_patch.dart:51:28)
#14     ListIterable.toList (dart:_internal/iterable.dart:213:44)
#15     ProgramCompiler.visitBlock (package:dev_compiler/src/kernel/compiler.dart:3963:62)
#16     Block.accept (package:kernel/ast.dart:8906:43)
#17     ProgramCompiler._visitStatement (package:dev_compiler/src/kernel/compiler.dart:3775:20)
#18     ProgramCompiler._emitFunctionScopedBody (package:dev_compiler/src/kernel/compiler.dart:3808:18)
#19     ProgramCompiler._emitSyncFunctionBody.<anonymous closure> (package:dev_compiler/src/kernel/compiler.dart:3577:17)
#20     ProgramCompiler._withLetScope (package:dev_compiler/src/kernel/compiler.dart:2382:25)
#21     ProgramCompiler._withCurrentFunction (package:dev_compiler/src/kernel/compiler.dart:3611:18)
#22     ProgramCompiler._emitSyncFunctionBody (package:dev_compiler/src/kernel/compiler.dart:3573:17)
#23     ProgramCompiler._emitFunction (package:dev_compiler/src/kernel/compiler.dart:3390:11)
#24     ProgramCompiler._emitLibraryFunction (package:dev_compiler/src/kernel/compiler.dart:2764:14)
#25     MappedIterator.moveNext (dart:_internal/iterable.dart:391:20)
#26     new _GrowableList._ofOther (dart:core-patch/growable_array.dart:202:26)
#27     new _GrowableList.of (dart:core-patch/growable_array.dart:152:26)
#28     new List.of (dart:core-patch/array_patch.dart:51:28)
#29     Iterable.toList (dart:core/iterable.dart:470:12)
#30     ProgramCompiler._emitLibraryProcedures (package:dev_compiler/src/kernel/compiler.dart:2729:10)
#31     ProgramCompiler._emitLibrary (package:dev_compiler/src/kernel/compiler.dart:578:7)
#32     List.forEach (dart:core-patch/growable_array.dart:416:8)
#33     ProgramCompiler.emitModule (package:dev_compiler/src/kernel/compiler.dart:431:15)
#34     _compile (package:dev_compiler/src/kernel/command.dart:441:27)
<asynchronous suspension>
#35     compile (package:dev_compiler/src/kernel/command.dart:50:12)
<asynchronous suspension>
#36     internalMain (package:dev_compiler/ddc.dart:38:18)
<asynchronous suspension>
@gmpassos gmpassos changed the title dartdevc can't compile closure/local function returning FutureOr<T?> Severe: dartdevc can't compile closure/local function returning FutureOr<T?> Mar 21, 2022
@gmpassos gmpassos changed the title Severe: dartdevc can't compile closure/local function returning FutureOr<T?> SEVERE: dartdevc can't compile closure/local function returning FutureOr<T?> Mar 21, 2022
@gmpassos
Copy link
Contributor Author

@gmpassos
Copy link
Contributor Author

FYI: @nshahan

@nshahan
Copy link
Contributor

nshahan commented Mar 21, 2022

@gmpassos Thanks for the minimal reproduction, it really helps us resolve these bugs!

@johnniwinther @chloestefantsova This looks like another case of the undetermined nullability leaking through to the DDC backend. The text dump of the .dill file shows undetermined nullability coming through in a few places. The crash is specifically coming from the the return type of the local method subf.

static method main() → void {
  tes::_f1<dynamic>(<core::int>[123]);
}
static method _f1<T extends core::Object? = dynamic>(dynamic o) → FutureOr<tes::_f1::T?> {
  function subF(dynamic e) → FutureOr<tes::_f1::T?>% {
    if(!(e == null))
      return e as{ForNonNullableByDefault} tes::_f1::T%;
    return tes::_f2<tes::_f1::T%>(o);
  }
  return subF(o){(dynamic) → FutureOr<tes::_f1::T?>%};
}
static method _f2<T extends core::Object? = dynamic>(dynamic o) → FutureOr<tes::_f2::T?>
  return o as{ForNonNullableByDefault} tes::_f2::T?;
}

@nshahan nshahan added area-front-end Use area-front-end for front end / CFE / kernel format related issues. web-dev-compiler type-bug Incorrect behavior (everything from a crash to more subtle misbehavior) labels Mar 21, 2022
@gmpassos
Copy link
Contributor Author

If you need some help to test some fix, just let me know.

The actual error happened in a much bigger code, and a complex method. I spent 2h to isolate the bug, since the ProgramCompiler is not giving any information about the Class or method that it was generating.

Actually I had to clone the Dart SDK, and change de internal dev_compiler package to run without build all the Dart SDK, and then be able to debug it with my project. After that I had to guess the statement the it was trying to generate by the stack variables, then I had to try to modify my project code to isolate the line that generated the bug!

I strongly recommend to add some context to the exceptions of the compiler. At least the file, class and method. Since even the reported file not generated by the compiler was totally different from the file containing the error.

@gmpassos
Copy link
Contributor Author

gmpassos commented Mar 21, 2022

Another thing. I think that the package dev_compiler at pub.dev should be archived/Unlisted:

https://pub.dev/packages/dev_compiler

It's 5 years old!

@nshahan
Copy link
Contributor

nshahan commented Mar 21, 2022

@kevmoo Do you have any thoughts on archiving/unlisting the dev_compiler package? It was last published before my time but I'm not aware of any need for it on pub.

gmpassos added a commit to Colossus-Services/bones_api that referenced this issue Mar 21, 2022
- `EntityHandler.resolveValueByType`:
  - Avoid `dev_compiler` bug dart-lang/sdk#48631 when generating JS code.
@jonasfj
Copy link
Member

jonasfj commented Mar 22, 2022

@gmpassos the package is flagged discontinued, this omits it from search, highlights the fact that it is unused, and warns anyone who installs it using dart pub get.

Not that anyone can install this with a modern Dart SDK, when it has an SDK constraint of sdk: ">=1.12.0 <2.0.0" 😄

@perret123
Copy link

I encountered the exact same exception, but didn't take the time to isolate the code-block that caused it. Kudos to @gmpassos that you figured that out. I just continued using iOS/Android for debugging purposes.

But of course, I would also be happy if debugging on the web works again for me, so now that this ticket is closed, I wondered if my exception was also gone - so I updated to the master channel in which the fix should already be applied;

[√] Flutter (Channel master, 2.13.0-0.0.pre.329, on Microsoft Windows [Version 10.0.19042.1586], locale de-CH)
    • Flutter version 2.13.0-0.0.pre.329 at C:\src\flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 5c6918933e (57 minutes ago), 2022-03-30 20:30:09 -0700
    • Engine revision a1106daced
    • Dart version 2.17.0 (build 2.17.0-249.0.dev)
    • DevTools version 2.11.4

But I still encounter the same exception:


Unhandled exception:
Unsupported operation: Undetermined Nullability
#0      ProgramCompiler._emitFutureOrType (package:dev_compiler/src/kernel/compiler.dart:3056:7)
#1      ProgramCompiler.visitFutureOrType (package:dev_compiler/src/kernel/compiler.dart:2954:11)

#2      FutureOrType.accept (package:kernel/ast.dart:11542:42)
#3      ProgramCompiler._emitType (package:dev_compiler/src/kernel/compiler.dart:2853:54)
#4      ProgramCompiler._emitArgumentList (package:dev_compiler/src/kernel/compiler.dart:5616:41)
#5      ProgramCompiler._emitMethodCall (package:dev_compiler/src/kernel/compiler.dart:4890:16)

#6      ProgramCompiler.visitInstanceInvocation (package:dev_compiler/src/kernel/compiler.dart:4804:12)
#7      InstanceInvocation.accept (package:kernel/ast.dart:5525:44)
#8      ProgramCompiler._visitExpression (package:dev_compiler/src/kernel/compiler.dart:3869:20)
#9      ProgramCompiler.visitReturnStatement (package:dev_compiler/src/kernel/compiler.dart:4403:38)

#10     ReturnStatement.accept (package:kernel/ast.dart:9889:43)
#11     ProgramCompiler._visitStatement (package:dev_compiler/src/kernel/compiler.dart:3775:20)
#12     MappedListIterable.elementAt (dart:_internal/iterable.dart:413:31)

#13     ListIterator.moveNext (dart:_internal/iterable.dart:342:26)
#14     new _GrowableList._ofEfficientLengthIterable (dart:core-patch/growable_array.dart:189:27)
#15     new _GrowableList.of (dart:core-patch/growable_array.dart:150:28)
#16     new List.of (dart:core-patch/array_patch.dart:51:28)
#17     ListIterable.toList (dart:_internal/iterable.dart:213:44)
#18     ProgramCompiler.visitBlock (package:dev_compiler/src/kernel/compiler.dart:3963:62)
#19     Block.accept (package:kernel/ast.dart:8910:43)

#20     ProgramCompiler._visitStatement (package:dev_compiler/src/kernel/compiler.dart:3775:20)
#21     ProgramCompiler._emitFunctionScopedBody (package:dev_compiler/src/kernel/compiler.dart:3808:18)
#22     ProgramCompiler._emitSyncFunctionBody.<anonymous closure> (package:dev_compiler/src/kernel/compiler.dart:3577:17)
#23     ProgramCompiler._withLetScope (package:dev_compiler/src/kernel/compiler.dart:2382:25)
#24     ProgramCompiler._withCurrentFunction (package:dev_compiler/src/kernel/compiler.dart:3611:18)
#25     ProgramCompiler._emitSyncFunctionBody (package:dev_compiler/src/kernel/compiler.dart:3573:17)
#26     ProgramCompiler._emitFunction (package:dev_compiler/src/kernel/compiler.dart:3390:11)
#27     ProgramCompiler._emitArrowFunction (package:dev_compiler/src/kernel/compiler.dart:6202:13)
#28     ProgramCompiler.visitFunctionExpression (package:dev_compiler/src/kernel/compiler.dart:6195:14)
#29     FunctionExpression.accept (package:kernel/ast.dart:8304:44)
#30     ProgramCompiler._visitExpression (package:dev_compiler/src/kernel/compiler.dart:3869:20)

#31     ProgramCompiler.visitLet (package:dev_compiler/src/kernel/compiler.dart:6244:16)
#32     Let.accept (package:kernel/ast.dart:8415:44)
#33     ProgramCompiler._visitExpression (package:dev_compiler/src/kernel/compiler.dart:3869:20)
#34     ProgramCompiler.visitLet (package:dev_compiler/src/kernel/compiler.dart:6245:16)
#35     Let.accept (package:kernel/ast.dart:8415:44)
#36     ProgramCompiler._visitExpression (package:dev_compiler/src/kernel/compiler.dart:3869:20)

#37     ProgramCompiler._visitInitializer (package:dev_compiler/src/kernel/compiler.dart:1792:11)
#38     ProgramCompiler._emitStaticFieldInitializer.<anonymous closure> (package:dev_compiler/src/kernel/compiler.dart:2372:23)
#39     ProgramCompiler._withLetScope (package:dev_compiler/src/kernel/compiler.dart:2382:25)
#40     ProgramCompiler._emitStaticFieldInitializer (package:dev_compiler/src/kernel/compiler.dart:2370:40)
#41     ProgramCompiler._emitLazyFields (package:dev_compiler/src/kernel/compiler.dart:2345:43)

#42     ProgramCompiler._emitTopLevelFields (package:dev_compiler/src/kernel/compiler.dart:2329:21)
#43     ProgramCompiler._emitLibrary (package:dev_compiler/src/kernel/compiler.dart:579:7)
#44     List.forEach (dart:core-patch/growable_array.dart:416:8)
#45     ProgramCompiler.emitModule (package:dev_compiler/src/kernel/compiler.dart:431:15)

#46     JavaScriptBundler.compile ([file:///C:/b/s/w/ir/cache/builder/sdk/pkg/frontend_server/lib/src/javascript_bundle.dart:152:33]())
<asynchronous suspension>
#47     FrontendCompiler.writeJavascriptBundle ([file:///C:/b/s/w/ir/cache/builder/sdk/pkg/frontend_server/lib/frontend_server.dart:693:32]())
<asynchronous suspension>
#48     FrontendCompiler.compile ([file:///C:/b/s/w/ir/cache/builder/sdk/pkg/frontend_server/lib/frontend_server.dart:601:9]())
<asynchronous suspension>

#49     listenAndCompile.<anonymous closure> ([file:///C:/b/s/w/ir/cache/builder/sdk/pkg/frontend_server/lib/frontend_server.dart:1198:11]())
<asynchronous suspension>

the Dart compiler exited unexpectedly.

Now, the question is;
Was my problem really the same problem as the one in the opening post? Is the OP's problem solved with this fix?

And I would really like to support the suggestion of @gmpassos:

I strongly recommend to add some context to the exceptions of the compiler. At least the file, class and method. Since even the reported file not generated by the compiler was totally different from the file containing the error.

Because it would be quite a task for me to find the cause of this error. Funnily, in my full code, I don't even have "FutureOr" used - so I suspect a package, but as I use 60 packages I can't really point my finger in any direction.

Of course, maybe at some point I will be forced to go through this in any case - in the meantime I can still use the release-version for web-testing - as this error is "only" in the debug version.

Anyway, thank you all for the time spent on solving flutter problems, it is really fun to work with :-)

@gmpassos
Copy link
Contributor Author

Can you test the code at?:

https://github.com/gmpassos/dev_compiler_test

With the last Dart at dev channel?

This fix entered in version 2.17.0-258.0.dev

@nshahan
Copy link
Contributor

nshahan commented Mar 31, 2022

This fix entered in version 2.17.0-258.0.dev

I see the fix is first present in 2.17.0-247.0.dev according to the list of tags shown here
ab62c34. That makes me think there might be a different code pattern triggering the issue again.

Because it would be quite a task for me to find the cause of this error. Funnily, in my full code, I don't even have "FutureOr" used - so I suspect a package, but as I use 60 packages I can't really point my finger in any direction.

I totally agree that we need the error message to provide more context to be helpful in tracking down these bugs. The trickiest part is that when the invalid state is recognized, we don't have any source location information. We have been discussing how we can add enough context to be helpful without having to touch every visitor in the compiler. I think I can pretty easily include the name of the library and at least that could help us narrow it down among many packages.

@gmpassos
Copy link
Contributor Author

gmpassos commented Mar 31, 2022

Note: the dev compiler compiles all the imported packages and related Dart files (there's no tree shaking). This is why a separated package (not your own code) can trigger any dev compiler bug.

When the bug happened with my project I had a lot of work to identify which package and method was triggering the bug, and it was in a 3rd part package (in a file that wasn't being used by my project, even indirectly, but was part of the exported files of the package).

@nshahan
Copy link
Contributor

nshahan commented Mar 31, 2022

I'm landing a quick fix that should help at least narrow down the library and the type that contains the undetermined nullability https://dart-review.googlesource.com/c/sdk/+/239722.

To get more source location information we will need to follow up with a larger change.

@chloestefantsova
Copy link
Contributor

I'm working an experiment to restrict the use of .declaredNullability: https://dart-review.googlesource.com/c/sdk/+/239670. Is the error still manifests with the CL patched in?

@nshahan
Copy link
Contributor

nshahan commented Apr 1, 2022

@chloestefantsova

I'm working an experiment to restrict the use of .declaredNullability: https://dart-review.googlesource.com/c/sdk/+/239670. Is the error still manifests with the CL patched in?

I tried the original reproduction from the top of this thread with your new change and did not see any regressions. I don't yet have a repro for the issue that @perret123 ran into so I'm not sure about that one.

@perret123
Copy link

Thank you for the quick fixes @nshahan @chloestefantsova

The issue was caused by my code after all - in combination with the riverpod 1.0.3 package.

As the riverpod code is quite complex, I only made a reproducible example with usage of the riverpod package and flutter. Is this already helpful for you? If not, feel free to tell me.

import 'dart:async';

import 'package:riverpod/riverpod.dart';

final myFutureProvider = FutureProvider<int>((ref) {
  return ref.watch(myStreamProvider.future);
});

final myStreamProvider = StreamProvider<int>((ref) {
  return Stream<int>.periodic(const Duration(seconds: 1), (x) => x).take(15);
});

void main() {}

Running this example with Flutter 2.13.0-0.0.pre.362 with Dart 2.17.0-263.0.dev on Chrome gives the following output:

PS C:\src\nullabilityexception> flutter run -d chrome
Launching lib\main.dart on Chrome in debug mode...
Unhandled exception:
Unsupported operation: Undetermined Nullability encounted while compiling file:///C:/src/nullabilityexception/lib/main.dart, which contains the type: FutureOrType(FutureOr<int>%).
#0      ProgramCompiler._undeterminedNullabilityError (package:dev_compiler/src/kernel/compiler.dart:3055:7)
#1      ProgramCompiler._emitFutureOrType (package:dev_compiler/src/kernel/compiler.dart:3040:7)
.....

The newly added output of the file/type is very helpful in finding the cause of the exception, thank you!

@chloestefantsova
Copy link
Contributor

I made a repro.

import 'dart:async';

abstract class A {
  T foo<T>(B<T> b);
}

class B<X> {
  B(X x);
}

class C<Y> {
  final Bar<FutureOr<Y>, D<Y>> bar;

  C(this.bar);
}

abstract class D<W> implements A {}

typedef Bar<V, U extends A> = V Function(U);

final baz = C<int>((a) {
    return a.foo(B(Future.value(0)));
});

When compiled to Kernel by the CFE it emits the following code with a suspicious FutureOr<int>%, which I believe to be the root of the issue.

$ ./fasta compile --dump-ir -o /dev/null /tmp/asdf.dart
library /*isNonNullableByDefault*/;
import self as self;
import "dart:core" as core;
import "dart:async" as asy;

import "dart:async";

typedef Bar<V extends core::Object? = dynamic, contravariant U extends self::A> = (U) → V%;
abstract class A extends core::Object {
  synthetic constructor •() → self::A
    : super core::Object::•()
    ;
  abstract method foo<T extends core::Object? = dynamic>(self::B<self::A::foo::T%> b) → self::A::foo::T%;
}
class B<X extends core::Object? = dynamic> extends core::Object {
  constructor •(self::B::X% x) → self::B<self::B::X%>
    : super core::Object::•()
    ;
}
class C<Y extends core::Object? = dynamic> extends core::Object {
  final field (self::D<self::C::Y%>) → FutureOr<self::C::Y%>% bar;
  constructor •((self::D<self::C::Y%>) → FutureOr<self::C::Y%>% bar) → self::C<self::C::Y%>
    : self::C::bar = bar, super core::Object::•()
    ;
}
abstract class D<W extends core::Object? = dynamic> extends core::Object implements self::A {
  synthetic constructor •() → self::D<self::D::W%>
    : super core::Object::•()
    ;
}
static final field self::C<core::int> baz = new self::C::•<core::int>((self::D<core::int> a) → FutureOr<core::int>% {
  return a.{self::A::foo}<FutureOr<core::int>%>(new self::B::•<FutureOr<core::int>%>(asy::Future::value<core::int>(0))){(self::B<FutureOr<core::int>%>) → FutureOr<core::int>%};
});

That type does seem suspicious, so I'll work on eliminating that (it should be just FutureOr<int>). Hopefully, it will solve the issue.

@chloestefantsova
Copy link
Contributor

I just landed the CL https://dart-review.googlesource.com/c/sdk/+/240049 that should fix that particular second case of undetermined nullability. I'm currently working on a bigger change that should work for similar cases arising in future.

copybara-service bot pushed a commit that referenced this issue Apr 5, 2022
Bug: #48631
Change-Id: I329a70386d59425cf3f157dc4b6316649f8aa389
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/240049
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Chloe Stefantsova <cstefantsova@google.com>
copybara-service bot pushed a commit that referenced this issue Apr 25, 2022
Bug: #48631
Change-Id: I329a70386d59425cf3f157dc4b6316649f8aa389
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/240049
Reviewed-by: Johnni Winther <johnniwinther@google.com>
Commit-Queue: Chloe Stefantsova <cstefantsova@google.com>
@gmpassos
Copy link
Contributor Author

Has this bug been fixed?

@chloestefantsova
Copy link
Contributor

Yes, I believe the issue can be closed. Feel free to reopen if needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-front-end Use area-front-end for front end / CFE / kernel format related issues. type-bug Incorrect behavior (everything from a crash to more subtle misbehavior) web-dev-compiler
Projects
None yet
Development

No branches or pull requests

6 participants