New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue with IKG (incremental kernel generator) #31085

Open
mkustermann opened this Issue Oct 12, 2017 · 1 comment

Comments

Projects
None yet
3 participants
@mkustermann
Member

mkustermann commented Oct 12, 2017

When recently working om ikg benchmarks, I noticed a new issue must have been introduced recently:

sdk $ tools/build.py -mrelease -ax64 runtime_kernel
...

# This works!
sdk $ out/ReleaseX64/dart pkg/front_end/tool/incremental_perf.dart --target=vm --sdk-summary=out/ReleaseX64/vm_outline.dill --sdk-library-specification=out/ReleaseX64/patched_sdk/sdk/libraries.json pkg/front_end/benchmarks/ikg/hello.{dart,edits.json}  
Libraries changed: 1
Initial compilation took: 545ms
edit Edit(org-dartlang-overlay:///usr/local/google/home/kustermann/repositories/sdk-gclient/sdk/pkg/front_end/benchmarks/ikg/hello.dart, "world" -> "fisk")
Change 'mainlib' - Libraries changed: 1
Change 'mainlib' - Incremental compilation took: 82ms

# This doesn't work!
sdk $ out/ReleaseX64/dart pkg/front_end/tool/incremental_perf.dart --target=vm --sdk-summary=out/ReleaseX64/vm_outline.dill --sdk-library-specification=out/ReleaseX64/patched_sdk/sdk/libraries.json pkg/front_end/benchmarks/ikg/dart2js.{dart,edits.json}
Unhandled exception:
Missing canonical name for Reference to dart.collection::MapBase::addAll
#0      BinaryPrinter.writeReference (package:kernel/binary/ast_to_binary.dart:411:9)
#1      BinaryPrinter.visitDirectMethodInvocation (package:kernel/binary/ast_to_binary.dart:879:5)
#2      DirectMethodInvocation.accept (package:kernel/ast.dart:2136:36)
#3      LimitedBinaryPrinter.writeNode (package:kernel/binary/limited_ast_to_binary.dart:55:10)
#4      BinaryPrinter.writeOptionalNode (package:kernel/binary/ast_to_binary.dart:156:7)
#5      BinaryPrinter.visitReturnStatement (package:kernel/binary/ast_to_binary.dart:1229:5)
#6      ReturnStatement.accept (package:kernel/ast.dart:3802:35)
#7      LimitedBinaryPrinter.writeNode (package:kernel/binary/limited_ast_to_binary.dart:55:10)
#8      BinaryPrinter.writeOptionalNode (package:kernel/binary/ast_to_binary.dart:156:7)
#9      BinaryPrinter.visitFunctionNode (package:kernel/binary/ast_to_binary.dart:751:5)
#10     FunctionNode.accept (package:kernel/ast.dart:1667:30)
#11     LimitedBinaryPrinter.writeNode (package:kernel/binary/limited_ast_to_binary.dart:55:10)
#12     BinaryPrinter.writeOptionalNode (package:kernel/binary/ast_to_binary.dart:156:7)
#13     BinaryPrinter.visitProcedure (package:kernel/binary/ast_to_binary.dart:676:5)
#14     Procedure.accept (package:kernel/ast.dart:1368:32)
#15     LimitedBinaryPrinter.writeNode (package:kernel/binary/limited_ast_to_binary.dart:55:10)
#16     List.forEach (dart:core-patch/dart:core/growable_array.dart:269)
#17     BinaryPrinter.writeList (package:kernel/binary/ast_to_binary.dart:137:11)
#18     BinaryPrinter.writeNodeList (package:kernel/binary/ast_to_binary.dart:141:5)
#19     BinaryPrinter.visitClass (package:kernel/binary/ast_to_binary.dart:624:5)
#20     Class.accept (package:kernel/ast.dart:824:30)
#21     LimitedBinaryPrinter.writeNode (package:kernel/binary/limited_ast_to_binary.dart:55:10)
#22     List.forEach (dart:core-patch/dart:core/growable_array.dart:269)
#23     BinaryPrinter.writeList (package:kernel/binary/ast_to_binary.dart:137:11)
#24     BinaryPrinter.writeNodeList (package:kernel/binary/ast_to_binary.dart:141:5)
#25     BinaryPrinter.visitLibrary (package:kernel/binary/ast_to_binary.dart:492:5)
#26     Library.accept (package:kernel/ast.dart:384:30)
#27     LimitedBinaryPrinter.writeNode (package:kernel/binary/limited_ast_to_binary.dart:55:10)
#28     List.forEach (dart:core-patch/dart:core/growable_array.dart:269)
#29     BinaryPrinter.writeList (package:kernel/binary/ast_to_binary.dart:137:11)
#30     LimitedBinaryPrinter.writeLibraries (package:kernel/binary/limited_ast_to_binary.dart:49:5)
#31     BinaryPrinter.writeProgramFile (package:kernel/binary/ast_to_binary.dart:214:5)
#32     serializeProgram (package:front_end/src/fasta/kernel/utils.dart:52:11)
#33     KernelDriver._compileCycle.<anonymous closure>.<anonymous closure> (package:front_end/src/incremental/kernel_driver.dart:292:13)
#34     PerformanceLog.run (package:front_end/src/base/performace_logger.dart:34:15)
#35     KernelDriver._compileCycle.<anonymous closure> (package:front_end/src/incremental/kernel_driver.dart:290:15)
<asynchronous suspension>
#36     PerformanceLog.runAsync (package:front_end/src/base/performace_logger.dart:52:21)
<asynchronous suspension>
#37     KernelDriver._compileCycle (package:front_end/src/incremental/kernel_driver.dart:214:20)
<asynchronous suspension>
#38     KernelDriver.getKernel.<anonymous closure>.<anonymous closure> (package:front_end/src/incremental/kernel_driver.dart:165:21)
<asynchronous suspension>
#39     PerformanceLog.runAsync (package:front_end/src/base/performace_logger.dart:52:21)
<asynchronous suspension>
#40     KernelDriver.getKernel.<anonymous closure> (package:front_end/src/incremental/kernel_driver.dart:162:21)
<asynchronous suspension>
#41     PerformanceLog.runAsync (package:front_end/src/base/performace_logger.dart:52:21)
<asynchronous suspension>
#42     KernelDriver.runWithFrontEndContext.<anonymous closure> (package:front_end/src/incremental/kernel_driver.dart:195:22)
#43     CompilerContext.runInContext.<anonymous closure> (package:front_end/src/fasta/compiler_context.dart:89:33)
#44     _rootRun (dart:async/zone.dart:1124)
#45     _CustomZone.run (dart:async/zone.dart:1021)
#46     runZoned (dart:async/zone.dart:1499)
#47     CompilerContext.runInContext (package:front_end/src/fasta/compiler_context.dart:89:12)
#48     CompilerContext.runWithOptions (package:front_end/src/fasta/compiler_context.dart:96:41)
#49     KernelDriver.runWithFrontEndContext (package:front_end/src/incremental/kernel_driver.dart:193:34)
<asynchronous suspension>
#50     KernelDriver.getKernel (package:front_end/src/incremental/kernel_driver.dart:130:18)
<asynchronous suspension>
#51     IncrementalKernelGeneratorImpl.computeDelta.<anonymous closure> (package:front_end/src/incremental_kernel_generator_impl.dart:102:51)
<asynchronous suspension>
#52     PerformanceLog.runAsync (package:front_end/src/base/performace_logger.dart:52:21)
<asynchronous suspension>
#53     IncrementalKernelGeneratorImpl.computeDelta (package:front_end/src/incremental_kernel_generator_impl.dart:100:20)
#54     main (file:///usr/local/google/home/kustermann/repositories/sdk-gclient/sdk/pkg/front_end/tool/incremental_perf.dart:88:31)
<asynchronous suspension>
#55     _startIsolate.<anonymous closure> (dart:isolate-patch/dart:isolate/isolate_patch.dart:275)
#56     _RawReceivePortImpl._handleMessage (dart:isolate-patch/dart:isolate/isolate_patch.dart:163)

As can be seen above, IKG on hello world works, but on dart2js causes a strange issue :-/

As another side note: The pkg/front_end/tool/incremental_perf.dart script was supposed to measure IKG time only, not serialization. Yet the stack above shows some kind of serialization is involved. Why is that?

/cc @scheglov @sigmundch

@sigmundch

This comment has been minimized.

Show comment
Hide comment
@sigmundch

sigmundch Oct 12, 2017

Member

There are some general issues that IKG is running into with the the mixin transformer. The transformer is not modular and breaks IKG in various ways. Normally it results in invalid code rather than a crash though. Relevant bugs if you are curious: #29881 #30111 and #31043 .

If you remove this transformer (commenting it out in pkg/kernel/target/vm.dart), you'll see the benchmark running again. Another way to work around this is to provide vm_platform instead of vm_outline. That's because one the transformer is not meant to run against outlines in the first place.

BTW - it might be worth changing incremental_perf_test.dart to use the data from the benchmarks/ikg folder. We are currently testing a small program and dart2js, but it would be nice to test the exact same configuration as the one you are using.

You may note that the test uses vm_platform.dill if it's available. We were however able to run with an outline file if checked-mode was off (.status file marks the failure in checked mode), so your crash surprises me a bit. I would expect this to happen on the unit test as well :-/. We need to take a closer look at that.

Member

sigmundch commented Oct 12, 2017

There are some general issues that IKG is running into with the the mixin transformer. The transformer is not modular and breaks IKG in various ways. Normally it results in invalid code rather than a crash though. Relevant bugs if you are curious: #29881 #30111 and #31043 .

If you remove this transformer (commenting it out in pkg/kernel/target/vm.dart), you'll see the benchmark running again. Another way to work around this is to provide vm_platform instead of vm_outline. That's because one the transformer is not meant to run against outlines in the first place.

BTW - it might be worth changing incremental_perf_test.dart to use the data from the benchmarks/ikg folder. We are currently testing a small program and dart2js, but it would be nice to test the exact same configuration as the one you are using.

You may note that the test uses vm_platform.dill if it's available. We were however able to run with an outline file if checked-mode was off (.status file marks the failure in checked mode), so your crash surprises me a bit. I would expect this to happen on the unit test as well :-/. We need to take a closer look at that.

@kmillikin kmillikin assigned kmillikin and unassigned sigmundch Nov 24, 2017

@kmillikin kmillikin added this to Incoming in Dart Front End Jan 3, 2018

@kmillikin kmillikin added the p3-low label Jan 5, 2018

@kmillikin kmillikin removed this from Incoming Untriaged in Dart Front End Jan 5, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment