Webdev build daemon crashes on windows due to removal of pub.dart.snapshot from the SDK

[annagrin]

pub.dart.snapshot was removed from the SDK starting from SDK version 2.15.0-49.0.dev

SDK commit: dart-lang/sdk@ae81cd5

Webdev is using dart pub.dart.snapshot build_runner build command to make sure we can communicate to the build daemon - see:

https://github.com/dart-lang/webdev/blob/a0b8e2b062fa8cfafc3c410259e934322cc4dde6/webdev/lib/src/daemon_client.dart#L19

In particular, the comment:

// On Windows we need to call the snapshot directly otherwise
// the process will start in a disjoint cmd without access to
// STDIO. We also want to ensure the version of pub is consistent with
// the SDK that was used to launch webdev.

I tried using dart pub or dart dartdev.dart.snapshot pub instead of the original command, but unfortunately the disjoint cmd still happens and prevents the daemon from communicating to webdev correctly, see test failures on windows in:

dart-lang/webdev#1394

Changing the process start mode to ProcessStartMode.normal in the code below seems to solve the problem in my experiments (in tests at via VSCode) - is there anything preventing us from changing that?

build/build_daemon/lib/client.dart

Line 171 in 2159c90

mode: ProcessStartMode.detachedWithStdio,

• Dart SDK version: 2.15.0-51.0.dev
• Webdev version: 2.7.5
• build 2.1.0
• build_config 1.0.0
• build_daemon 3.0.0
• build_resolvers 2.0.4
• build_runner 2.1.1
• build_runner_core 7.1.0
• build_verify 2.0.0
• build_version 2.1.0

TL;DR the crash:

Building

[INFO] Connecting to the build daemon...
[SEVERE] Could not find a command named "/Users/annagrin/.dart-sdks/2.15.0-51.0.dev/bin/snapshots/pub.dart.snapshot".
[SEVERE]
[SEVERE] Usage: dart [vm-options] <command|dart-file> [arguments]
[SEVERE]
[SEVERE] Global options:
[SEVERE] -h, --help                 Print this usage information.
[SEVERE] -v, --verbose              Show additional command output.
[SEVERE]     --version              Print the Dart SDK version.
[SEVERE]     --enable-analytics     Enable analytics.
[SEVERE]     --disable-analytics    Disable analytics.
[SEVERE]     --diagnostics          Show tool diagnostic output.
[SEVERE]
[SEVERE] Available commands:
[SEVERE]   analyze           Analyze Dart code in a directory.
[SEVERE]   compile           Compile Dart to various formats.
[SEVERE]   create            Create a new Dart project.
[SEVERE]   devtools          Open DevTools (optionally connecting to an existing application).
[SEVERE]   fix               Apply automated fixes to Dart source code.
[SEVERE]   format            Idiomatically format Dart source code.
[SEVERE]   language-server   Start Dart's analysis server.
[SEVERE]   migrate           Perform null safety migration on a project.
[SEVERE]   pub               Work with packages.
[SEVERE]   run               Run a Dart program.
[SEVERE]   test              Run tests for a project.
[SEVERE]
[SEVERE] Run "dart help <command>" for more information about a command.
[SEVERE] See https://dart.dev/tools/dart-tool for detailed documentation.
Unhandled exception:
Bad state: Unable to start build daemon.
#0      _handleDaemonStartup.<anonymous closure>
#1      _runUserCode (dart:async/stream_pipe.dart:11:23)
#2      Stream.firstWhere.<anonymous closure> (dart:async/stream.dart:1338:9)
#3      _RootZone.runGuarded (dart:async/zone.dart:1608:10)
#4      _BufferingStreamSubscription._sendDone.sendDone (dart:async/stream_impl.dart:394:13)
#5      _BufferingStreamSubscription._sendDone (dart:async/stream_impl.dart:404:7)
#6      _BufferingStreamSubscription._close (dart:async/stream_impl.dart:291:7)
#7      _SyncBroadcastStreamController._sendDone.<anonymous closure> (dart:async/broadcast_stream_controller.dart:400:22)
#8      _BroadcastStreamController._forEachListener (dart:async/broadcast_stream_controller.dart:323:15)
#9      _SyncBroadcastStreamController._sendDone (dart:async/broadcast_stream_controller.dart:399:7)
#10     _BroadcastStreamController.close (dart:async/broadcast_stream_controller.dart:269:5)
#11     _AsBroadcastStreamController.close (dart:async/broadcast_stream_controller.dart:505:27)
#12     _RootZone.runGuarded (dart:async/zone.dart:1608:10)
#13     _BufferingStreamSubscription._sendDone.sendDone (dart:async/stream_impl.dart:394:13)
#14     _BufferingStreamSubscription._sendDone (dart:async/stream_impl.dart:404:7)
#15     _BufferingStreamSubscription._close (dart:async/stream_impl.dart:291:7)
#16     _SinkTransformerStreamSubscription._close (dart:async/stream_transformers.dart:87:11)
#17     _EventSinkWrapper.close (dart:async/stream_transformers.dart:21:11)
#18     _StringAdapterSink.close (dart:convert/string_conversion.dart:251:11)
#19     _LineSplitterSink.close (dart:convert/line_splitter.dart:135:11)
#20     _SinkTransformerStreamSubscription._handleDone (dart:async/stream_transformers.dart:132:24)
#21     _RootZone.runGuarded (dart:async/zone.dart:1608:10)
#22     _BufferingStreamSubscription._sendDone.sendDone (dart:async/stream_impl.dart:394:13)
#23     _BufferingStreamSubscription._sendDone (dart:async/stream_impl.dart:404:7)
#24     _BufferingStreamSubscription._close (dart:async/stream_impl.dart:291:7)
#25     _SinkTransformerStreamSubscription._close (dart:async/stream_transformers.dart:87:11)
#26     _EventSinkWrapper.close (dart:async/stream_transformers.dart:21:11)
#27     _StringAdapterSink.close (dart:convert/string_conversion.dart:251:11)
#28     _Utf8ConversionSink.close (dart:convert/string_conversion.dart:305:20)
#29     _ConverterStreamEventSink.close (dart:convert/chunked_conversion.dart:81:18)
#30     _SinkTransformerStreamSubscription._handleDone (dart:async/stream_transformers.dart:132:24)
#31     _RootZone.runGuarded (dart:async/zone.dart:1608:10)
#32     _BufferingStreamSubscription._sendDone.sendDone (dart:async/stream_impl.dart:394:13)
#33     _BufferingStreamSubscription._sendDone (dart:async/stream_impl.dart:404:7)
#34     _BufferingStreamSubscription._close (dart:async/stream_impl.dart:291:7)
#35     _SyncStreamControllerDispatch._sendDone (dart:async/stream_controller.dart:741:19)
#36     _StreamController._closeUnchecked (dart:async/stream_controller.dart:596:7)
#37     _StreamController.close (dart:async/stream_controller.dart:589:5)
#38     _Socket._onData (dart:io-patch/socket_patch.dart:2173:21)
#39     _RootZone.runUnaryGuarded (dart:async/zone.dart:1620:10)
#40     _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:341:11)
#41     _BufferingStreamSubscription._add (dart:async/stream_impl.dart:271:7)
#42     _SyncStreamControllerDispatch._sendData (dart:async/stream_controller.dart:733:19)
#43     _StreamController._add (dart:async/stream_controller.dart:607:7)
#44     _StreamController.add (dart:async/stream_controller.dart:554:5)
#45     new _RawSocket.<anonymous closure> (dart:io-patch/socket_patch.dart:1709:35)
#46     _NativeSocket.issueReadEvent.issue (dart:io-patch/socket_patch.dart:1206:18)
#47     _microtaskLoop (dart:async/schedule_microtask.dart:40:21)
#48     _startMicrotaskLoop (dart:async/schedule_microtask.dart:49:5)
#49     _runPendingImmediateCallback (dart:isolate-patch/isolate_patch.dart:119:13)
#50     _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:188:5)

Exited (255)

[annagrin]

/cc @natebosch @jakemac53 @sigmundch - I'd appreciate your advice on that, the issue prevents webdev from working on the latest dev SDK, as well as breaks webdev CI

[jakemac53]

@annagrin did you have a chance to try just dart run build_runner build?

[sigmundch]

@jakemac53 - to clarify, is your suggestion to bypass pub? at a first glance I thought you were suggesting dart pub run build_runner build, which @annagrin tried but didn't work (see dart-lang/webdev#1394 (comment))

/cc @bkonyi @sigurdm , in case you have any insights on the issue above.

One random idea to try is to see whether disabling the default dartdev cli with --disable-dart-dev helps in this picture:

dart --disable-dart-dev dartdev.dart.snapshot pub run

[jakemac53]

dart run works the same as pub run used to - but it doesn't use a bat script on windows which was the problem IIRC there. If you try to use dart pub run it tells you it is deprecated and to instead use dart run.

[jakemac53]

cc @grouma who I think did all the original investigation here and might remember the details of exactly why we had to use this process mode.

[natebosch]

the details of exactly why we had to use this process mode.

https://github.com/dart-lang/webdev/blob/f6b8457d1be971fd9cf8b89178290bb76369bf34/webdev/lib/src/daemon_client.dart#L21-L24

I think you're right that it was the batch script causing issues with stdio.

I also would expect that using dart run build_runner should work.

[natebosch]

More discussion in dart-lang/webdev#111

[annagrin]

Thanks for help, removing pub from the command fixed the windows issue:

dart-lang/webdev#1394

Closing this.