[Codelab] Example failing for Scrolling test in integration testing.

[chonghorizons]

Expected Result

https://codelabs.developers.google.com/codelabs/flutter-app-testing/ The codelab integration test examples run and the tests pass.

Actual Result

The Scrolling test fails in https://github.com/flutter/codelabs/blob/master/testing_codelab/step_07/integration_test/app_test.dart

The other two tests pass.

This is the error log

shell$ flutter drive --driver integration_test/driver.dart --target integration_test/app_test.dart --profile
Running "flutter pub get" in testing_app...                      1,543ms
Running Gradle task 'assembleProfile'...                           56.9s
✓  Built build/app/outputs/flutter-apk/app-profile.apk (11.8MB).
Installing build/app/outputs/flutter-apk/app.apk...                 3.5s
VMServiceFlutterDriver: Connecting to Flutter application at http://127.0.0.1:62747/iRiezfewcsE=/
VMServiceFlutterDriver: Isolate found with number: 2816534710512779
VMServiceFlutterDriver: Isolate is paused at start.
VMServiceFlutterDriver: Attempting to resume isolate
I/flutter ( 5749): 00:00 +0: Testing App Performance Tests Favorites operations test
VMServiceFlutterDriver: Connected to Flutter application.
I/flutter ( 5749): 00:09 +1: Testing App Performance Tests (tearDownAll)
W/ple.testing_ap( 5749): Accessing hidden method
Lsun/misc/Unsafe;->compareAndSwapObject(Ljava/lang/Object;JLjava/lang/Object;Ljava/lang/Object;)Z (greylist, linking,
allowed)
W/ple.testing_ap( 5749): Accessing hidden method
Lsun/misc/Unsafe;->compareAndSwapObject(Ljava/lang/Object;JLjava/lang/Object;Ljava/lang/Object;)Z (greylist, linking,
allowed)
I/flutter ( 5749): 00:09 +2: All tests passed!
All tests passed.
Macs-MacBook-Air:testing_app howardchong$ flutter drive --driver integration_test/driver.dart --target integration_test/app_test.dart --profile
Running "flutter pub get" in testing_app...                      1,470ms
Running Gradle task 'assembleProfile'...                           65.0s
✓  Built build/app/outputs/flutter-apk/app-profile.apk (14.6MB).
Installing build/app/outputs/flutter-apk/app.apk...                 3.8s
VMServiceFlutterDriver: Connecting to Flutter application at http://127.0.0.1:62934/OSWzRBQSSno=/
VMServiceFlutterDriver: Isolate found with number: 2692430830870515
VMServiceFlutterDriver: Isolate is paused at start.
VMServiceFlutterDriver: Attempting to resume isolate
I/flutter ( 6418): 00:00 +0: Testing App Performance Tests Scrolling test
VMServiceFlutterDriver: Connected to Flutter application.
I/flutter ( 6418): SocketException: OS Error: Connection refused, errno = 111, address = localhost, port = 40625
I/flutter ( 6418): #0      _NativeSocket.startConnect (dart:io-patch/socket_patch.dart:677)
I/flutter ( 6418): #1      _RawSocket.startConnect (dart:io-patch/socket_patch.dart:1680)
I/flutter ( 6418): #2      RawSocket.startConnect (dart:io-patch/socket_patch.dart:27)
I/flutter ( 6418): #3      Socket._startConnect (dart:io-patch/socket_patch.dart:1891)
I/flutter ( 6418): #4      Socket.startConnect (dart:io/socket.dart:738)
I/flutter ( 6418): #5      _ConnectionTarget.connect (dart:_http/http_impl.dart:2439)
I/flutter ( 6418): #6      _HttpClient._getConnection.connect (dart:_http/http_impl.dart:2805)
I/flutter ( 6418): #7      _HttpClient._getConnection (dart:_http/http_impl.dart:2810)
I/flutter ( 6418): #8      _HttpClient._openUrl (dart:_http/http_impl.dart:2694)
I/flutter ( 6418): #9      _HttpClient.openUrl (dart:_http/http_impl.dart:2568)
I/flutter ( 6418): #10     _WebSocketImpl.connect (dart:_http/websocket_impl.dart:1027)
I/flutter ( 6418): #11     WebSocket.connect (dart:_http/websocket.dart:379)
I/flutter ( 6418): #12     vmServiceConnectUri (package:vm_service/vm_service_io.dart:32)
I/flutter ( 6418): #13     IntegrationTestWidgetsFlutterBinding.enableTimeline
(package:integration_test/integration_test.dart:244)
I/flutter ( 6418): <asynchronous suspension>
I/flutter ( 6418): #14     IntegrationTestWidgetsFlutterBinding.traceTimeline
(package:integration_test/integration_test.dart:269)
I/flutter ( 6418): <asynchronous suspension>
I/flutter ( 6418): #15     IntegrationTestWidgetsFlutterBinding._runAndGetGCInfo
(package:integration_test/integration_test.dart:321)
I/flutter ( 6418): <asynchronous suspension>
I/flutter ( 6418): #16     IntegrationTestWidgetsFlutterBinding.watchPerformance
(package:integration_test/integration_test.dart:376)
I/flutter ( 6418): <asynchronous suspension>
I/flutter ( 6418): #17     main.<anonymous closure>.<anonymous closure>
(file:///Users/howardchong/Proj2021/testing_app/integration_test/app_test.dart:22)
I/flutter ( 6418): <asynchronous suspension>
I/flutter ( 6418): #18     testWidgets.<anonymous closure>.<anonymous closure>
(package:flutter_test/src/widget_tester.dart:176)
I/flutter ( 6418): <asynchronous suspension>
I/flutter ( 6418): #19     TestWidgetsFlutterBinding._runTestBody (package:flutter_test/src/binding.dart:817)
I/flutter ( 6418): <asynchronous suspension>
I/flutter ( 6418): 00:02 +0: Testing App Performance Tests Scrolling test [E]
I/flutter ( 6418):   Test failed. See exception logs above.
I/flutter ( 6418):   The test description was: Scrolling test
I/flutter ( 6418):   
I/flutter ( 6418): Consider enabling the flag chain-stack-traces to receive more detailed exceptions.
I/flutter ( 6418): For example, 'dart test --chain-stack-traces'.
I/flutter ( 6418): 00:02 +0 -1: Testing App Performance Tests Favorites operations test
I/flutter ( 6418): 00:11 +1 -1: Testing App Performance Tests (tearDownAll)
W/ple.testing_ap( 6418): Accessing hidden method
Lsun/misc/Unsafe;->compareAndSwapObject(Ljava/lang/Object;JLjava/lang/Object;Ljava/lang/Object;)Z (greylist, linking,
allowed)
W/ple.testing_ap( 6418): Accessing hidden method
Lsun/misc/Unsafe;->compareAndSwapObject(Ljava/lang/Object;JLjava/lang/Object;Ljava/lang/Object;)Z (greylist, linking,
allowed)
I/flutter ( 6418): 00:11 +2 -1: Some tests failed.
Failure Details:
Failure in method: Scrolling test

end of failure 1

This is flutter doctor

flutter doctor -v
[✓] Flutter (Channel stable, 2.5.3, on macOS 12.0.1 21A559 darwin-x64, locale en-US)
    • Flutter version 2.5.3 at /Users/howardchong/development/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 18116933e7 (6 weeks ago), 2021-10-15 10:46:35 -0700
    • Engine revision d3ea636dc5
    • Dart version 2.14.4

[✓] Android toolchain - develop for Android devices (Android SDK version 31.0.0)
    • Android SDK at /Users/howardchong/Library/Android/sdk
    • Platform android-31, build-tools 31.0.0
    • Java binary at: /Applications/Android Studio.app/Contents/jre/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7281165)
    • All Android licenses accepted.

[✗] Xcode - develop for iOS and macOS
    ✗ Xcode installation is incomplete; a full installation is necessary for iOS development.
      Download at: https://developer.apple.com/xcode/download/
      Or install Xcode via the App Store.
      Once installed, run:
        sudo xcode-select --switch /Applications/Xcode.app/Contents/Developer
        sudo xcodebuild -runFirstLaunch
    ✗ CocoaPods installed but not working.
        You appear to have CocoaPods installed but it is not working.
        This can happen if the version of Ruby that CocoaPods was installed with is different from the one being used to
        invoke it.
        This can usually be fixed by re-installing CocoaPods.
      To re-install see https://guides.cocoapods.org/using/getting-started.html#installation for instructions.

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2020.3)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7281165)

[✓] Connected device (2 available)
    • PH 1 (mobile) • PM1LHMA7B1300032 • android-arm64  • Android 10 (API 29)
    • Chrome (web)  • chrome           • web-javascript • Google Chrome 96.0.4664.55

Running on physical device (PH1).

[danagbemava-nc]

@chonghorizons, thanks for reporting this.

I looked into it and the issue appears to be coming from the binding.watchPerformance() method.

logs

without binding.watchPerformance

nexus@Daniels-MBP testing_app % flutter drive --driver integration_test/driver.dart --target integration_test/app_test.dart --profile
Running "flutter pub get" in testing_app...                        989ms
Multiple devices found:
M2007J20CG (mobile)       • 5dd3be00      • android-arm64  • Android 11 (API 30)
sdk gphone arm64 (mobile) • emulator-5554 • android-arm64  • Android 11 (API 30) (emulator)
macOS (desktop)           • macos         • darwin-arm64   • macOS 11.6 20G165 darwin-arm
Chrome (web)              • chrome        • web-javascript • Google Chrome 96.0.4664.55
[1]: M2007J20CG (5dd3be00)
[2]: sdk gphone arm64 (emulator-5554)
[3]: macOS (macos)
[4]: Chrome (chrome)
Please choose one (To quit, press "q/Q"): 1
Warning: Mapping new ns http://schemas.android.com/repository/android/common/02 to old ns http://schemas.android.com/repository/android/common/01
Warning: Mapping new ns http://schemas.android.com/repository/android/generic/02 to old ns http://schemas.android.com/repository/android/generic/01
Warning: Mapping new ns http://schemas.android.com/sdk/android/repo/addon2/02 to old ns http://schemas.android.com/sdk/android/repo/addon2/01
Warning: Mapping new ns http://schemas.android.com/sdk/android/repo/repository2/02 to old ns http://schemas.android.com/sdk/android/repo/repository2/01
Warning: Mapping new ns http://schemas.android.com/sdk/android/repo/sys-img2/02 to old ns http://schemas.android.com/sdk/android/repo/sys-img2/01
Running Gradle task 'assembleProfile'...                           26.7s
✓  Built build/app/outputs/flutter-apk/app-profile.apk (11.8MB).
Installing build/app/outputs/flutter-apk/app.apk...                 5.6s
VMServiceFlutterDriver: Connecting to Flutter application at http://127.0.0.1:55581/KiNkCzrDoWQ=/
VMServiceFlutterDriver: Isolate found with number: 3842243781551043
VMServiceFlutterDriver: Isolate is paused at start.
VMServiceFlutterDriver: Attempting to resume isolate
I/flutter (13678): 00:00 +0: Testing App Performance Tests Scrolling test
VMServiceFlutterDriver: Connected to Flutter application.
I/ple.testing_ap(13678): ProcessProfilingInfo new_methods=538 is saved saved_to_disk=1 resolve_classes_delay=8000
I/flutter (13678): 00:04 +1: Testing App Performance Tests Favorites operations test
I/flutter (13678): 00:13 +2: Testing App Performance Tests (tearDownAll)
I/flutter (13678): 00:13 +3: All tests passed!
All tests passed.

with binding.watchPerformance

nexus@Daniels-MBP testing_app % flutter drive --driver integration_test/driver.dart --target integration_test/app_test.dart --profile
Running "flutter pub get" in testing_app...                        993ms
Multiple devices found:
M2007J20CG (mobile)       • 5dd3be00      • android-arm64  • Android 11 (API 30)
sdk gphone arm64 (mobile) • emulator-5554 • android-arm64  • Android 11 (API 30) (emulator)
macOS (desktop)           • macos         • darwin-arm64   • macOS 11.6 20G165 darwin-arm
Chrome (web)              • chrome        • web-javascript • Google Chrome 96.0.4664.55
[1]: M2007J20CG (5dd3be00)
[2]: sdk gphone arm64 (emulator-5554)
[3]: macOS (macos)
[4]: Chrome (chrome)
Please choose one (To quit, press "q/Q"): 1
Warning: Mapping new ns http://schemas.android.com/repository/android/common/02 to old ns http://schemas.android.com/repository/android/common/01
Warning: Mapping new ns http://schemas.android.com/repository/android/generic/02 to old ns http://schemas.android.com/repository/android/generic/01
Warning: Mapping new ns http://schemas.android.com/sdk/android/repo/addon2/02 to old ns http://schemas.android.com/sdk/android/repo/addon2/01
Warning: Mapping new ns http://schemas.android.com/sdk/android/repo/repository2/02 to old ns http://schemas.android.com/sdk/android/repo/repository2/01
Warning: Mapping new ns http://schemas.android.com/sdk/android/repo/sys-img2/02 to old ns http://schemas.android.com/sdk/android/repo/sys-img2/01
Running Gradle task 'assembleProfile'...                           28.3s
✓  Built build/app/outputs/flutter-apk/app-profile.apk (14.6MB).
Installing build/app/outputs/flutter-apk/app.apk...                 6.2s
VMServiceFlutterDriver: Connecting to Flutter application at http://127.0.0.1:56222/HgzzoqWRc2o=/
VMServiceFlutterDriver: Isolate found with number: 1412602457582979
VMServiceFlutterDriver: Isolate is paused at start.
VMServiceFlutterDriver: Attempting to resume isolate
I/flutter (15854): 00:00 +0: Testing App Performance Tests Scrolling test
VMServiceFlutterDriver: Connected to Flutter application.
I/ple.testing_ap(15854): WaitForGcToComplete blocked RunEmptyCheckpoint on ProfileSaver for 7.963ms
I/ple.testing_ap(15854): ProcessProfilingInfo new_methods=542 is saved saved_to_disk=1 resolve_classes_delay=8000
I/flutter (15854): SocketException: OS Error: Connection refused, errno = 111, address = localhost, port = 39192
I/flutter (15854): #0      _NativeSocket.startConnect (dart:io-patch/socket_patch.dart:677)
I/flutter (15854): #1      _RawSocket.startConnect (dart:io-patch/socket_patch.dart:1680)
I/flutter (15854): #2      RawSocket.startConnect (dart:io-patch/socket_patch.dart:27)
I/flutter (15854): #3      Socket._startConnect (dart:io-patch/socket_patch.dart:1891)
I/flutter (15854): #4      Socket.startConnect (dart:io/socket.dart:738)
I/flutter (15854): #5      _ConnectionTarget.connect (dart:_http/http_impl.dart:2439)
I/flutter (15854): #6      _HttpClient._getConnection.connect (dart:_http/http_impl.dart:2805)
I/flutter (15854): #7      _HttpClient._getConnection (dart:_http/http_impl.dart:2810)
I/flutter (15854): #8      _HttpClient._openUrl (dart:_http/http_impl.dart:2694)
I/flutter (15854): #9      _HttpClient.openUrl (dart:_http/http_impl.dart:2568)
I/flutter (15854): #10     _WebSocketImpl.connect (dart:_http/websocket_impl.dart:1027)
I/flutter (15854): #11     WebSocket.connect (dart:_http/websocket.dart:379)
I/flutter (15854): #12     vmServiceConnectUri (package:vm_service/vm_service_io.dart:32)
I/flutter (15854): #13     IntegrationTestWidgetsFlutterBinding.enableTimeline (package:integration_test/integration_test.dart:244)
I/flutter (15854): <asynchronous suspension>
I/flutter (15854): #14     IntegrationTestWidgetsFlutterBinding.traceTimeline (package:integration_test/integration_test.dart:269)
I/flutter (15854): <asynchronous suspension>
I/flutter (15854): #15     IntegrationTestWidgetsFlutterBinding._runAndGetGCInfo (package:integration_test/integration_test.dart:321)
I/flutter (15854): <asynchronous suspension>
I/flutter (15854): #16     IntegrationTestWidgetsFlutterBinding.watchPerformance (package:integration_test/integration_test.dart:376)
I/flutter (15854): <asynchronous suspension>
I/flutter (15854): #17     main.<anonymous closure>.<anonymous closure>
(file:///Users/nexus/projects/nevercode/testing_app/integration_test/app_test.dart:18)
I/flutter (15854): <asynchronous suspension>
I/flutter (15854): #18     testWidgets.<anonymous closure>.<anonymous closure> (package:flutter_test/src/widget_tester.dart:176)
I/flutter (15854): <asynchronous suspension>
I/flutter (15854): #19     TestWidgetsFlutterBinding._runTestBody (package:flutter_test/src/binding.dart:817)
I/flutter (15854): <asynchronous suspension>
I/flutter (15854): 00:02 +0: Testing App Performance Tests Scrolling test [E]
I/flutter (15854):   Test failed. See exception logs above.
I/flutter (15854):   The test description was: Scrolling test
I/flutter (15854):   
I/flutter (15854): Consider enabling the flag chain-stack-traces to receive more detailed exceptions.
I/flutter (15854): For example, 'dart test --chain-stack-traces'.
I/flutter (15854): 00:02 +0 -1: Testing App Performance Tests Favorites operations test
I/flutter (15854): 00:10 +1 -1: Testing App Performance Tests (tearDownAll)
I/flutter (15854): 00:10 +2 -1: Some tests failed.
Failure Details:
Failure in method: Scrolling test

end of failure 1

/cc @domesticmouse

[chonghorizons]

Small Update:

I tried to run this on the flutter web, and I got all tests passed. In --no-headless mode, I was able to see the two "fling" actions.

Log for `-d web-server`

testing_app howardchong$ flutter drive --driver integration_test/driver.dart --target integration_test/app_test.dart -d web-server --no-headless
Running "flutter pub get" in testing_app...                      2,383ms
Launching integration_test/app_test.dart on Web Server in debug mode...
Waiting for connection from debug service on Web Server...         35.0s
integration_test/app_test.dart is being served at http://localhost:53303
The web-server device requires the Dart Debug Chrome extension for debugging. Consider using the Chrome or Edge devices
for an improved development workflow.
[INFO]: http://localhost:53303/ 61:26 "Installed new service worker."
[WARNING]: http://localhost:53303/dart_sdk.js 49581:16 "registerExtension() from dart:developer is only supported in build/run/test environments where the developer event method hooks have been set."
[WARNING]: http://localhost:53303/dart_sdk.js 49563:16 "postEvent() from dart:developer is only supported in build/run/test environments where the developer event method hooks have been set."
[INFO]: http://localhost:53303/dart_sdk.js 28067:14 "00:00 +0: Testing App Performance Tests Scrolling test"
All tests passed.
Application finished.

Therefore this may be a physical device error. I get the binding.watchPerformance() error you get and we both tested on physical devices.

Current knowledge

• Error triggers on your physical device and my physical device.
• No error when testing for web.

Suggestions

See if the same error triggers when running in the Android Emulator or an iOS device.

[danagbemava-nc]

It failed on iOS too.

logs

Running "flutter pub get" in testing_app...                        949ms
Automatically signing iOS for device deployment using specified development team in Xcode project: HGS96587A5
Running pod install...                                             909ms
Running Xcode build...                                                  
 └─Compiling, linking and signing...                        12.3s
Xcode build done.                                           49.5s
Installing and launching...                                        38.7s
VMServiceFlutterDriver: Connecting to Flutter application at http://127.0.0.1:49672/YeaX-qIB1wA=/
VMServiceFlutterDriver: Isolate found with number: 15265507526239
VMServiceFlutterDriver: Isolate is paused at start.
VMServiceFlutterDriver: Attempting to resume isolate
flutter: 00:00 +0: Testing App Performance Tests Scrolling test
fopen failed for data file: errno = 2 (No such file or directory)
Errors found! Invalidating cache...
fopen failed for data file: errno = 2 (No such file or directory)
Errors found! Invalidating cache...
VMServiceFlutterDriver: Connected to Flutter application.
flutter: SocketException: OS Error: Connection refused, errno = 61, address = localhost, port = 51806
flutter: #0      _NativeSocket.startConnect (dart:io-patch/socket_patch.dart:677)
flutter: #1      _RawSocket.startConnect (dart:io-patch/socket_patch.dart:1680)
flutter: #2      RawSocket.startConnect (dart:io-patch/socket_patch.dart:27)
flutter: #3      Socket._startConnect (dart:io-patch/socket_patch.dart:1891)
flutter: #4      Socket.startConnect (dart:io/socket.dart:738)
flutter: #5      _ConnectionTarget.connect (dart:_http/http_impl.dart:2439)
flutter: #6      _HttpClient._getConnection.connect (dart:_http/http_impl.dart:2805)
flutter: #7      _HttpClient._getConnection (dart:_http/http_impl.dart:2810)
flutter: #8      _HttpClient._openUrl (dart:_http/http_impl.dart:2694)
flutter: #9      _HttpClient.openUrl (dart:_http/http_impl.dart:2568)
flutter: #10     _WebSocketImpl.connect (dart:_http/websocket_impl.dart:1027)
flutter: #11     WebSocket.connect (dart:_http/websocket.dart:379)
flutter: #12     vmServiceConnectUri (package:vm_service/vm_service_io.dart:32)
flutter: #13     IntegrationTestWidgetsFlutterBinding.enableTimeline (package:integration_test/integration_test.dart:244)
flutter: <asynchronous suspension>
flutter: #14     IntegrationTestWidgetsFlutterBinding.traceTimeline (package:integration_test/integration_test.dart:269)
flutter: <asynchronous suspension>
flutter: #15     IntegrationTestWidgetsFlutterBinding._runAndGetGCInfo (package:integration_test/integration_test.dart:321)
flutter: <asynchronous suspension>
flutter: #16     IntegrationTestWidgetsFlutterBinding.watchPerformance (package:integration_test/integration_test.dart:376)
flutter: <asynchronous suspension>
flutter: #17     main.<anonymous closure>.<anonymous closure>
(file:///Users/nexus/projects/nevercode/testing_app/integration_test/app_test.dart:22)
flutter: <asynchronous suspension>
flutter: #18     testWidgets.<anonymous closure>.<anonymous closure> (package:flutter_test/src/widget_tester.dart:176)
flutter: <asynchronous suspension>
flutter: #19     TestWidgetsFlutterBinding._runTestBody (package:flutter_test/src/binding.dart:817)
flutter: <asynchronous suspension>
flutter: 00:02 +0: Testing App Performance Tests Scrolling test [E]
flutter:   Test failed. See exception logs above.
  The test description was: Scrolling test
flutter:
flutter: Consider enabling the flag chain-stack-traces to receive more detailed exceptions.
For example, 'dart test --chain-stack-traces'.
flutter: 00:02 +0 -1: Testing App Performance Tests Favorites operations test
flutter: 00:10 +1 -1: Testing App Performance Tests (tearDownAll)
flutter: Warning: integration_test plugin was not detected.


If you're running the tests with `flutter drive`, please make sure your tests
are in the `integration_test/` directory of your package and use
`flutter test $path_to_test` to run it instead.


If you're running the tests with Android instrumentation or XCTest, this means
that you are not capturing test results properly! See the following link for
how to set up the integration_test plugin:


https://flutter.dev/docs/testing/integration-tests#testing-on-firebase-test-lab
flutter: 00:10 +2 -1: Some tests failed.
Failure Details:
Failure in method: Scrolling test

end of failure 1

[abhaysood]

It fails on Android emulator too - Pixel 3 API 28

logs

VMServiceFlutterDriver: Connecting to Flutter application at http://127.0.0.1:56869/pB8P3j2LbFA=/
VMServiceFlutterDriver: Isolate found with number: 651184184411879
VMServiceFlutterDriver: Isolate is paused at start.
VMServiceFlutterDriver: Attempting to resume isolate
D/eglCodecCommon( 7683): setVertexArrayObject: set vao to 0 (0) 1 0
I/flutter ( 7683): 00:00 +0: Testing App Performance Tests Scrolling test
VMServiceFlutterDriver: Connected to Flutter application.
I/flutter ( 7683): ══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
I/flutter ( 7683): The following SocketException was thrown running a test:
I/flutter ( 7683): OS Error: Connection refused, errno = 111, address = localhost, port = 60582
I/flutter ( 7683):
I/flutter ( 7683): When the exception was thrown, this was the stack:
I/flutter ( 7683): #0      _NativeSocket.startConnect (dart:io-patch/socket_patch.dart:677:35)
I/flutter ( 7683): #1      _RawSocket.startConnect (dart:io-patch/socket_patch.dart:1680:26)
I/flutter ( 7683): #2      RawSocket.startConnect (dart:io-patch/socket_patch.dart:27:23)
I/flutter ( 7683): #3      Socket._startConnect (dart:io-patch/socket_patch.dart:1891:22)
I/flutter ( 7683): #4      Socket.startConnect (dart:io/socket.dart:738:21)
I/flutter ( 7683): #5      _ConnectionTarget.connect (dart:_http/http_impl.dart:2439:18)
I/flutter ( 7683): #6      _HttpClient._getConnection.connect (dart:_http/http_impl.dart:2805:12)
I/flutter ( 7683): #7      _HttpClient._getConnection (dart:_http/http_impl.dart:2810:12)
I/flutter ( 7683): #8      _HttpClient._openUrl (dart:_http/http_impl.dart:2694:12)
I/flutter ( 7683): #9      _HttpClient.openUrl (dart:_http/http_impl.dart:2568:7)
I/flutter ( 7683): #10     _WebSocketImpl.connect (dart:_http/websocket_impl.dart:1027:24)
I/flutter ( 7683): #11     WebSocket.connect (dart:_http/websocket.dart:379:22)
I/flutter ( 7683): #12     vmServiceConnectUri (package:vm_service/vm_service_io.dart:32:44)
I/flutter ( 7683): #13     IntegrationTestWidgetsFlutterBinding.enableTimeline (package:integration_test/integration_test.dart:244:26)
I/flutter ( 7683): <asynchronous suspension>
I/flutter ( 7683): #14     IntegrationTestWidgetsFlutterBinding.traceTimeline (package:integration_test/integration_test.dart:269:5)
I/flutter ( 7683): <asynchronous suspension>
I/flutter ( 7683): #15     IntegrationTestWidgetsFlutterBinding._runAndGetGCInfo (package:integration_test/integration_test.dart:321:34)
I/flutter ( 7683): <asynchronous suspension>
I/flutter ( 7683): #16     IntegrationTestWidgetsFlutterBinding.watchPerformance (package:integration_test/integration_test.dart:376:43)
I/flutter ( 7683): <asynchronous suspension>
I/flutter ( 7683): #17     main.<anonymous closure>.<anonymous closure> (file:///Users/abhaysood/experiments/untitled/integration_test/app_test.dart:18:7)
I/flutter ( 7683): <asynchronous suspension>
I/flutter ( 7683): #18     testWidgets.<anonymous closure>.<anonymous closure> (package:flutter_test/src/widget_tester.dart:176:15)
I/flutter ( 7683): <asynchronous suspension>
I/flutter ( 7683): #19     TestWidgetsFlutterBinding._runTestBody (package:flutter_test/src/binding.dart:817:5)
I/flutter ( 7683): <asynchronous suspension>
I/flutter ( 7683):
I/flutter ( 7683): The test description was:
I/flutter ( 7683):   Scrolling test
I/flutter ( 7683): ════════════════════════════════════════════════════════════════════════════════════════════════════
I/flutter ( 7683): 00:05 +0: Testing App Performance Tests Scrolling test [E]
I/flutter ( 7683):   Test failed. See exception logs above.
I/flutter ( 7683):   The test description was: Scrolling test
I/flutter ( 7683):
I/flutter ( 7683): Consider enabling the flag chain-stack-traces to receive more detailed exceptions.
I/flutter ( 7683): For example, 'dart test --chain-stack-traces'.
I/flutter ( 7683): 00:05 +0 -1: Testing App Performance Tests Favorites operations test
I/flutter ( 7683): 00:14 +1 -1: Testing App Performance Tests (tearDownAll)
Failure Details:
Failure in method: Scrolling test
══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞═════════════════
The following SocketException was thrown running a test:
OS Error: Connection refused, errno = 111, address = localhost,
port = 60582

When the exception was thrown, this was the stack:
#0      _NativeSocket.startConnect (dart:io-patch/socket_patch.dart:677:35)
#1      _RawSocket.startConnect (dart:io-patch/socket_patch.dart:1680:26)
#2      RawSocket.startConnect (dart:io-patch/socket_patch.dart:27:23)
#3      Socket._startConnect (dart:io-patch/socket_patch.dart:1891:22)
#4      Socket.startConnect (dart:io/socket.dart:738:21)
#5      _ConnectionTarget.connect (dart:_http/http_impl.dart:2439:18)
#6      _HttpClient._getConnection.connect (dart:_http/http_impl.dart:2805:12)
#7      _HttpClient._getConnection (dart:_http/http_impl.dart:2810:12)
#8      _HttpClient._openUrl (dart:_http/http_impl.dart:2694:12)
#9      _HttpClient.openUrl (dart:_http/http_impl.dart:2568:7)
#10     _WebSocketImpl.connect (dart:_http/websocket_impl.dart:1027:24)
#11     WebSocket.connect (dart:_http/websocket.dart:379:22)
#12     vmServiceConnectUri (package:vm_service/vm_service_io.dart:32:44)
#13     IntegrationTestWidgetsFlutterBinding.enableTimeline (package:integration_test/integration_test.dart:244:26)
<asynchronous suspension>
#14     IntegrationTestWidgetsFlutterBinding.traceTimeline (package:integration_test/integration_test.dart:269:5)
<asynchronous suspension>
#15     IntegrationTestWidgetsFlutterBinding._runAndGetGCInfo (package:integration_test/integration_test.dart:321:34)
<asynchronous suspension>
#16     IntegrationTestWidgetsFlutterBinding.watchPerformance (package:integration_test/integration_test.dart:376:43)
<asynchronous suspension>
#17     main.<anonymous closure>.<anonymous closure> (file:///Users/abhaysood/experiments/untitled/integration_test/app_test.dart:18:7)
<asynchronous suspension>
#18     testWidgets.<anonymous closure>.<anonymous closure> (package:flutter_test/src/widget_tester.dart:176:15)
<asynchronous suspension>
#19     TestWidgetsFlutterBinding._runTestBody (package:flutter_test/src/binding.dart:817:5)
<asynchronous suspension>

The test description was:
  Scrolling test
═════════════════════════════════════════════════════════════════

end of failure 1



I/flutter ( 7683): 00:14 +2 -1: Some tests failed.

[domesticmouse]

/cc @miquelbeltran

[miquelbeltran]

@domesticmouse I can confirm this issue when running on mobile devices, including emulators, but not as desktop app.

The thing I noticed is that IntegrationTestWidgetsFlutterBinding.watchPerformance is trying to connect to a localhost service, which does not exist on the device but on the local desktop.

In flutter/flutter#90410 the same issue is discussed, and the solution is to add --no-dds to the command, so to run:

flutter drive --driver integration_test/driver.dart --target integration_test/app_test.dart --no-dds

[abhaysood]

I can confirm the codelab works fine when running with --no-dds flag. Thanks @miquelbeltran.

[chonghorizons]

I think this can be closed as a duplicate, then of flutter/flutter#90410. Thanks @miquelbeltran .