Running tests causes an `RPCError` fairly often.

[gspencergoog]

What happened

When running tests in the Flutter framework, but right clicking next to the main() declaration and clicking "Run tests in ...", this RPC error occurs fairly often now, probably every fourth time I try it or so. It may have something to do with timing: it doesn't seem to happen if I'm not running tests one right after another. Maybe something is crashing and restarting when each test finishes, and it takes a moment to restart?

Version information

IntelliJ IDEA 2020.2.2 • Flutter plugin io.flutter 50.0 • Dart plugin 202.7319.5

Flutter 1.23.0-19.0.pre.26 • channel nnbd_widgets_2 • git@github.com:gspencergoog/flutter.git
Framework • revision e46ec16b01 (4 hours ago) • 2020-10-14 10:47:44 -0700
Engine • revision adf5b59485
Tools • Dart 2.11.0 (build 2.11.0-217.0.dev)

Exception

null

java.lang.Throwable: org.dartlang.vm.service.element.RPCError@178b3b7e
	at com.intellij.openapi.diagnostic.Logger.error(Logger.java:143)
	at com.intellij.openapi.diagnostic.Logger.error(Logger.java:147)
	at com.intellij.idea.IdeaLogger.error(IdeaLogger.java:71)
	at io.flutter.run.test.FlutterTestRunner$1$1.onError(FlutterTestRunner.java:119)
	at org.dartlang.vm.service.VmServiceBase.processResponse(VmServiceBase.java:629)
	at org.dartlang.vm.service.VmServiceBase.processMessage(VmServiceBase.java:422)
	at org.dartlang.vm.service.VmServiceBase$1.onMessage(VmServiceBase.java:86)
	at de.roderick.weberknecht.WebSocketReceiver.run(WebSocketReceiver.java:65)

[devoncarew]

Thanks for the report!

Looks like you're on the latest Flutter plugin, and on an up-to-date version of the framework.

I'll see if I can repro locally. Is this with all tests? With tests in a certain project? Are there tests / projects where it doesn't repro?

If we can't repro I may put in more logging logic, and ask you to do something like switch to the plugin's dev channel (which auto-publishes from master every sunday morning).

[gspencergoog]

It doesn't seem to depend on which test, and I don't see a pattern to which ones succeed and which don't. The same test will usually succeed if I try it again after it throws the exception.

[gspencergoog]

The tests I was running were various widget tests in the Flutter repo: e.g. packages/flutter/test/widgets/router_test.dart

[devoncarew]

Great, thanks for the info. Will see if I can repro here.

[devoncarew]

It looks like this is happening when we're trying to resume isolates paused at VM start - on the initial connect to flutter test. When you see this error, do tests run to completion as normal, or does it prevent tests from running? It's not clear to me from the code that we're not reporting a harmless error - trying to resume an isolate that's already running.

I believe that the latency around isolate creation and notification has changed a little bit - we've recently landed some debugging middleware between the VM and IDEs (DDS).

[gspencergoog]

No, the test never runs, even if I wait five minutes for it to run. If I hit "stop" and run the test again, it runs after a few seconds (as normal). It's possible that it only reports the error once I hit "stop", I'll try and pay more attention to that next time.

[devoncarew]

Thanks! It sounds from your description that we are not resuming the isolate properly at the start of the test (that the reported error does indicate an issue). I can make a speculative fix here but would be more comfortable if it could repo this locally. Perhaps it repros if the transitive set of referenced code is large (if the isolate takes a long time to be ready).