Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run All Test fails with SIGINT #1673

Closed
ComplexCarbos opened this issue May 1, 2019 · 14 comments
Closed

Run All Test fails with SIGINT #1673

ComplexCarbos opened this issue May 1, 2019 · 14 comments
Labels
in flutter Relates to running Flutter apps in testing Relates to test execution of Dart/Flutter tests for end users is bug
Milestone

Comments

@ComplexCarbos
Copy link

I'm seeing a strange issue where I can no longer execute all my tests using Dart: Run All Tests.

All tests run successfully when invoking via Visual Studio Code terminal: flutter test.

When executing Dart: Run All Tests from the command palette some tests execute, but then the process fails:

✓ Game News Item Get Contents ✓ Game News Item Get Author Shell subprocess terminated by ^C (SIGINT, -2) after tests finished. Test: /home/rich/Documents/visual_studio_code_projects/GamerNews/test/screens/news_list_test.dart Shell: /home/rich/Development/flutter/bin/cache/artifacts/engine/linux-x64/flutter_tester dart:async/stream_controller.dart 595:43 _StreamController.addError dart:async/stream_controller.dart 862:13 _StreamSinkWrapper.addError package:stream_channel/src/guarantee_channel.dart 140:14 _GuaranteeSink._addError package:stream_channel/src/guarantee_channel.dart 131:5 _GuaranteeSink.addError package:flutter_tools/src/test/flutter_platform.dart 576:29 _FlutterPlatform._startTest.<fn> ===== asynchronous gap =========================== dart:async/zone.dart 1053:19 _CustomZone.registerUnaryCallback dart:async/runtime/libasync_patch.dart 77:23 _asyncThenWrapperHelper package:flutter_tools/src/test/flutter_platform.dart _FlutterPlatform._startTest.<fn> package:flutter_tools/src/test/flutter_platform.dart 782:26 _FlutterPlatform._startTest ===== asynchronous gap =========================== dart:async/zone.dart 1053:19 _CustomZone.registerUnaryCallback dart:async/runtime/libasync_patch.dart 77:23 _asyncThenWrapperHelper package:flutter_tools/src/test/flutter_platform.dart _FlutterPlatform._startTest package:flutter_tools/src/test/flutter_platform.dart 462:36 _FlutterPlatform.loadChannel package:flutter_tools/src/test/flutter_platform.dart 417:46 _FlutterPlatform.load ===== asynchronous gap =========================== dart:async/zone.dart 1053:19 _CustomZone.registerUnaryCallback dart:async/runtime/libasync_patch.dart 77:23 _asyncThenWrapperHelper package:test_core/src/runner/loader.dart Loader.loadFile.<fn> package:test_core/src/runner/load_suite.dart 98:31 new LoadSuite.<fn>.<fn> ===== asynchronous gap =========================== dart:async/zone.dart 1045:19 _CustomZone.registerCallback dart:async/zone.dart 962:22 _CustomZone.bindCallbackGuarded dart:async/timer.dart 52:45 new Timer dart:async/timer.dart 87:9 Timer.run dart:async/future.dart 174:11 new Future package:test_api/src/backend/invoker.dart 399:21 Invoker._onRun.<fn>.<fn>.<fn> ✓ NewsList Screen with no news. Exited (sigint)

Sometimes, a notification pops up that says: "No Debug Adapter Found"

Flutter doctor output:

`rich@rich-GS73VR-7RF:~/Documents/visual_studio_code_projects/GamerNews$ flutter doctor
Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel stable, v1.2.1, on Linux, locale en_US.UTF-8)
[✓] Android toolchain - develop for Android devices (Android SDK version 28.0.3)
[!] Android Studio (version 3.3)
✗ Flutter plugin not installed; this adds Flutter specific functionality.
✗ Dart plugin not installed; this adds Dart specific functionality.
[✓] VS Code (version 1.33.1)
[!] Connected device
! No devices available

! Doctor found issues in 2 categories.`

Capture Log output is attached.

Dart-Code-Log-2019-04-03 08-15-03.txt

The logs suggest the test process was interrupted because it couldn't find one of the test files. But I can see that file in the editor and it does exist on the file system:

rich@rich-GS73VR-7RF:~$ ls -l /home/rich/Documents/visual_studio_code_projects/GamerNews/test/screens/game_list_test.dart -rw-r--r-- 1 rich rich 2728 Apr 30 21:05 /home/rich/Documents/visual_studio_code_projects/GamerNews/test/screens/game_list_test.dart rich@rich-GS73VR-7RF:~$

@DanTup
Copy link
Member

DanTup commented May 1, 2019

Hmmm, the sigint is being sent because the debug adapter says it was told to terminate:

[08:15:13 GMT-0400 (EDT)] [FlutterTest] [Info] <== [{"event":"test.startedProcess","params":{"observatoryUri":"http://127.0.0.1:32947/"}}]
[08:15:13 GMT-0400 (EDT)] [Observatory] [Info] Connecting to VM Service at http://127.0.0.1:32947/ws
[08:15:13 GMT-0400 (EDT)] [Observatory] [Info] Disconnect requested!

The last line here is in the disconnectRequest handler which is normally called by VS Code when it wants to stop a session (for ex. if you click the Stop button on the debug toolbar):

protected async disconnectRequest(
response: DebugProtocol.DisconnectResponse,
args: DebugProtocol.DisconnectArguments,
): Promise<void> {
this.log(`Disconnect requested!`);

I think the "failed to load" message is just a result of us starting to terminate processes as a result of being told to quit.

Does this happen only for tests? Does it happen if you run a single test? Does it happen if you create the flutter create empty project and then try to run it from there?

@DanTup DanTup added the awaiting info Requires more information from the customer to progress label May 1, 2019
@ComplexCarbos
Copy link
Author

I only see this behavior when attempting to run all my tests by executing:

Dart: Run All Tests.

I can run individual tests (clicking run or debug in the source file).

I can execute all tests in a single file by right clicking and selecting "Start Debugging".

I just created an empty project through the IDE. I made no changes to the default project. I was able to execute Dart: Run All Tests.

✓ Counter increments smoke test
Exited

@DanTup DanTup added in testing Relates to test execution of Dart/Flutter tests for end users is bug and removed awaiting info Requires more information from the customer to progress labels May 1, 2019
@DanTup DanTup added this to the v2.27.0 milestone May 1, 2019
@DanTup
Copy link
Member

DanTup commented May 1, 2019

Thanks, I'll see what I can find. Based on your comment above, I believe this has worked for you in the past sometime? Do you know when it might've become broken?

@ComplexCarbos
Copy link
Author

Yep, you are correct. It was working fine until last night. All I did last night was add a test file with several new tests. Deleting the new test file doesn't do anything - still see the same error. Since I can run all tests with no errors from the command line, I thought I might have stumbled on some obscure issue with the VS Code - flutter integration.

The only change that sticks out in my mind is running an Ubuntu software update in the morning.

Thank you for looking!

@DanTup
Copy link
Member

DanTup commented May 2, 2019

Weird, I can't repro here, and the logs make no sense. I don't see why termination is being requested.

Could you try making a copy of your project and trimming your tests down, to see if you can make a small reproducible project that you could share? You can email it to logs@dartcode.org if you don't want to attach publicly.

Also - do you have easy access to a Windows or macOS machine to see whether it repros there? (I'm currently testing on macOS, though we do have automated tests on Travis that include Linux).

If that gets us nowhere, I can make a custom build of the extension with more verbose logging to help track it down.

@DanTup
Copy link
Member

DanTup commented May 2, 2019

Hmmm, looking at your log again, I see something a bit weird going on that maybe we've never accounted for. When we run multiple files like this, we actually just run a single command:

[08:15:10 GMT-0400 (EDT)] [FlutterTest] [Info] Spawning /home/rich/Development/flutter/bin/flutter with args ["test","--machine","--start-paused","test"]

We expect a single VM debugger connection, however the logs show that we were given multiple - one for each file. I wonder if what's happening here is that one file is finishing, which disconnects our VM, which causes us to signal the end of the run. I'll try making some longer-running test files so they don't all finish quickly and see if I can find anything.

@DanTup
Copy link
Member

DanTup commented May 2, 2019

Shell subprocess terminated by ^C (SIGINT, -2) after tests finished.

Aha, looks like I've reproduced it. We might have to change how this works since my assumptions about how flutter test would run multiple files don't seem to be accurate.

@DanTup
Copy link
Member

DanTup commented May 2, 2019

Right, I think I've tracked this down. In 7a2a372 we enabled connecting to the VM service even when running "without debugging", in order to allow VM-service features to work (toggle debug banners, performance overlays, etc.).

This resulted in us connecting the VM when running tests too - and based on the commit message for 5d10d7f I previously knew this wouldn't work for Flutter.

So, I should be able to fix by not trying to connect to the VM service for Flutter Test runs in noDebug mode. First I'd like to try and get some failing tests around this though. I don't know how quick I'll ship a fix, but once I have one I could at least make you a build that includes it.

@DanTup DanTup added the in flutter Relates to running Flutter apps label May 2, 2019
DanTup added a commit that referenced this issue May 2, 2019
DanTup added a commit that referenced this issue May 2, 2019
DanTup added a commit that referenced this issue May 2, 2019
@ComplexCarbos
Copy link
Author

Awesome. No hurry on this - I have a work around with flutter test. Anything more you need from me?

Really appreciate you looking into the issue.

@DanTup
Copy link
Member

DanTup commented May 2, 2019

I don't think so - I've got tests that fail now (runs all tests if given a folder), and a fix running that makes them go green locally, so hopefully all sorted :-)

A version with the fix is likely mid-end of May. If it gets annoying enough that you want the fix sooner, just shout - it's not a big deal to make a build.

Thanks for reporting it and capturing the logs to help me track it down!

DanTup added a commit that referenced this issue May 6, 2019
DanTup added a commit that referenced this issue May 6, 2019
DanTup added a commit that referenced this issue May 6, 2019
DanTup added a commit that referenced this issue May 6, 2019
DanTup added a commit that referenced this issue May 6, 2019
DanTup added a commit that referenced this issue May 6, 2019
DanTup added a commit that referenced this issue May 6, 2019
DanTup added a commit that referenced this issue May 6, 2019
DanTup added a commit that referenced this issue May 6, 2019
DanTup added a commit that referenced this issue May 15, 2019
@DanTup DanTup closed this as completed in 4d53e8c May 15, 2019
@jpv123
Copy link

jpv123 commented Jan 11, 2020

Sorry to reopen the issue, it's happening to me in VSCode with a launch.json file configured like this:

{
        "name": "All Tests",
        "program": "test/",
        "request": "launch",
        "type": "dart",
}

Error launches

Failed to load "/Users/........../test/Widgets/ProjectCell_test.dart":
Shell subprocess terminated by ^C (SIGINT, -2) before connecting to test harness.

Running flutter test works correctly

@jpv123
Copy link

jpv123 commented Jan 11, 2020

Forgot to add some important info sorry:

In test/Widgets folder I got only two testing files. One works correctly, the other one fails with the launch.json file. Both works with flutter test

I got the last version of VSCode 1.41.1
And the last version of flutter 1.12.13 + hotfix.5

@DanTup
Copy link
Member

DanTup commented Jan 13, 2020

@jpv123 thanks, I can repro this intermittently. I think this is what #2082 is for, so let's use that issue for this.

@DanTup
Copy link
Member

DanTup commented Jan 13, 2020

As a workaround, if you add "noDebug": true to the launch.json, it should fix it (albeit at the expense of debugging, and a warning from VS Code that it's not a valid option).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
in flutter Relates to running Flutter apps in testing Relates to test execution of Dart/Flutter tests for end users is bug
Projects
None yet
Development

No branches or pull requests

3 participants