WidgetTester.dragUntilVisible() to show helpful errors #89336

mleonhard · 2021-09-02T02:40:27Z

Use case

I am developing integration tests for a mobile app. The tests use WidgetTester.dragUntilVisible() (inherited from WidgetController.dragUntilVisible to scroll a ListView until a widget is visible. The method failed in my tests in several ways:

When it cannot find the target widget, it logs a useless exception. It does the same thing when the widget is scrolled up out of view.

Bad state: No element

When the exception was thrown, this was the stack:
#0      Iterable.single (dart:core/iterable.dart:498:25)
#1      WidgetController.element (package:flutter_test/src/controller.dart:112:30)
#2      WidgetController.dragUntilVisible.<anonymous closure> (package:flutter_test/src/controller.dart:1181:38)
<asynchronous suspension>
<asynchronous suspension>
(elided one frame from package:stack_trace)

integration_test/example_test.dart

import 'package:flutter/cupertino.dart' show CupertinoApp;
import 'package:flutter/widgets.dart'
    show ListView, Offset, SizedBox, Text, ValueKey, Widget;
import 'package:flutter_test/flutter_test.dart'
    show find, testWidgets, WidgetTester;
import 'package:integration_test/integration_test.dart'
    show IntegrationTestWidgetsFlutterBinding;

void main() {
  IntegrationTestWidgetsFlutterBinding.ensureInitialized();
  testWidgets("test1", (WidgetTester tester) async {
    await tester.pumpWidget(
      CupertinoApp(
        home: ListView(
          key: const ValueKey('table'),
          children: <Widget>[
            SizedBox(width: 10, height: 2000),
            Text('hello', key: ValueKey('target')),
          ],
        ),
      ),
    );
    await tester.dragUntilVisible(
      find.byKey(ValueKey('NONEXISTENT')),
      find.byKey(ValueKey('table')),
      Offset(0.0, -50.0),
    );
  });
}

When the target widget is not visible and it fails to find the view widget, it throws a good exception:

The finder "zero widgets with key [<'NONEXISTENT'>] (ignoring
offstage widgets)" (used in a call to "drag()") could not find
any matching widgets.

integration_test/example_test.dart

import 'package:flutter/cupertino.dart' show CupertinoApp;
import 'package:flutter/widgets.dart'
    show ListView, Offset, SizedBox, Text, ValueKey, Widget;
import 'package:flutter_test/flutter_test.dart'
    show find, testWidgets, WidgetTester;
import 'package:integration_test/integration_test.dart'
    show IntegrationTestWidgetsFlutterBinding;

void main() {
  IntegrationTestWidgetsFlutterBinding.ensureInitialized();
  testWidgets("test1", (WidgetTester tester) async {
    await tester.pumpWidget(
      CupertinoApp(
        home: ListView(
          key: const ValueKey('table'),
          children: <Widget>[
            SizedBox(width: 10, height: 2000),
            Text('hello', key: ValueKey('target'))
          ],
        ),
      ),
    );
    await tester.dragUntilVisible(
      find.byKey(ValueKey('target')),
      find.byKey(ValueKey('NONEXISTENT')),
      Offset(0.0, -50.0),
    );
  });
}

When the target widget is already visible, it silently ignores a bad view finder. Layout changes can make the test fail later.

integration_test/example_test.dart

import 'package:flutter/cupertino.dart' show CupertinoApp;
import 'package:flutter/widgets.dart'
    show ListView, Offset, Text, ValueKey, Widget;
import 'package:flutter_test/flutter_test.dart'
    show find, testWidgets, WidgetTester;
import 'package:integration_test/integration_test.dart'
    show IntegrationTestWidgetsFlutterBinding;

void main() {
  IntegrationTestWidgetsFlutterBinding.ensureInitialized();
  testWidgets("test1", (WidgetTester tester) async {
    await tester.pumpWidget(
      CupertinoApp(
        home: ListView(
          key: const ValueKey('table'),
          children: <Widget>[
            Text('hello', key: ValueKey('target')),
          ],
        ),
      ),
    );
    await tester.dragUntilVisible(
      find.byKey(ValueKey('target')),
      find.byKey(ValueKey('NONEXISTENT')),
      Offset(0.0, -50.0),
    );
  });
}

When the view finder matches two widgets, the error message is useful:

The finder "2 widgets with key [<'table'>] (ignoring offstage
widgets): [Container-[<'table'>],
ListView-[<'table'>](scrollDirection: vertical, primary: using
primary controller, AlwaysScrollableScrollPhysics, dependencies:
[MediaQuery, PrimaryScrollController])]" (used in a call to
"drag()") ambiguously found multiple matching widgets. The
"drag()" method needs a single target.

When the exception was thrown, this was the stack:
#0      WidgetController._getElementPoint (package:flutter_test/src/controller.dart:900:7)
#1      WidgetController.getCenter (package:flutter_test/src/controller.dart:836:12)
#2      WidgetController.drag (package:flutter_test/src/controller.dart:533:7)
#3      WidgetController.dragUntilVisible.<anonymous closure> (package:flutter_test/src/controller.dart:1177:15)
#4      WidgetController.dragUntilVisible.<anonymous closure> (package:flutter_test/src/controller.dart:1175:39)
#7      TestAsyncUtils.guard (package:flutter_test/src/test_async_utils.dart:71:41)
#8      WidgetController.dragUntilVisible (package:flutter_test/src/controller.dart:1175:27)
#9      main.<anonymous closure> (file:///Users/user/example4/integration_test/example_test.dart:26:18)
<asynchronous suspension>
<asynchronous suspension>
(elided 3 frames from dart:async and package:stack_trace)

integration_test/example_test.dart

import 'package:flutter/cupertino.dart' show CupertinoApp;
import 'package:flutter/widgets.dart'
    show Container, ListView, Offset, SizedBox, Text, ValueKey, Widget;
import 'package:flutter_test/flutter_test.dart'
    show find, testWidgets, WidgetTester;
import 'package:integration_test/integration_test.dart'
    show IntegrationTestWidgetsFlutterBinding;

void main() {
  IntegrationTestWidgetsFlutterBinding.ensureInitialized();
  testWidgets("test1", (WidgetTester tester) async {
    await tester.pumpWidget(
      CupertinoApp(
        home: Container(
          key: const ValueKey('table'),
          child: ListView(
            key: const ValueKey('table'),
            children: <Widget>[
              SizedBox(width: 10, height: 2000),
              Text('hello', key: ValueKey('target')),
            ],
          ),
        ),
      ),
    );
    await tester.dragUntilVisible(
      find.byKey(ValueKey('target')),
      find.byKey(ValueKey('table')),
      Offset(0.0, -50.0),
    );
  });
}

Steps to reproduce:

flutter create example4
Write integration_test/example_test.dart with one of the examples above.

Update these files:

pubspec.yaml

name: example4
description: A new Flutter project.
publish_to: 'none'
version: 1.0.0+1
environment:
  sdk: ">=2.12.0 <3.0.0"
dependencies:
  flutter:
    sdk: flutter
dev_dependencies:
  flutter_test:
    sdk: flutter
  integration_test:
    sdk: flutter
flutter:

test_driver/integration_test.dart

import 'package:integration_test/integration_test_driver.dart'
    show integrationDriver;

Future<void> main() => integrationDriver();

Execute the test:
flutter drive --driver=test_driver/integration_test.dart --target=integration_test/example_test.dart -d test-sim

flutter doctor -v

[✓] Flutter (Channel stable, 2.2.3, on macOS 11.5.1 20G80 darwin-x64, locale en-US)
    • Flutter version 2.2.3 at /Users/user/.flutter
    • Framework revision f4abaa0735 (9 weeks ago), 2021-07-01 12:46:11 -0700
    • Engine revision 241c87ad80
    • Dart version 2.13.4

[✓] Android toolchain - develop for Android devices (Android SDK version 31.0.0)
    • Android SDK at /Users/user/Library/Android/sdk
    • Platform android-31, build-tools 31.0.0
    • Java binary at: /Applications/Android Studio.app/Contents/jre/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 11.0.10+0-b96-7281165)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.5.1, Build version 12E507
    • CocoaPods version 1.10.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[!] Android Studio (version 2020.3)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    ✗ Unable to find bundled Java version.
    • Try updating or re-installing Android Studio.

[✓] Connected device (3 available)
    • test-sim (mobile) • E1A7325E-D299-42D4-A89C-B5DD7CF7DB06 • ios            • com.apple.CoreSimulator.SimRuntime.iOS-14-5 (simulator)
    • iPhone 8 (mobile) • 87606CD6-83B2-433B-9458-FD6944930D2F • ios            • com.apple.CoreSimulator.SimRuntime.iOS-14-5 (simulator)
    • Chrome (web)      • chrome                               • web-javascript • Google Chrome 92.0.4515.159
    ! Error: User's iPhone is not connected. Xcode will continue when User's iPhone is connected. (code -13)

! Doctor found issues in 1 category.

Proposal

When the target widget is not found, throw an exception that includes information about the target widget. This will help users debug failing tests.
Always check the view finder and throw an exception if it cannot find the view widget. This will make tests more reliable.

The text was updated successfully, but these errors were encountered:

darshankawar · 2021-09-02T07:37:57Z

Thanks for the detailed report and code samples. Used the first example and ran it on latest master and stable and get the same exception as below:

`Bad state: No element

When the exception was thrown, this was the stack:
#0      Iterable.single (dart:core/iterable.dart:498:25)
#1      WidgetController.element (package:flutter_test/src/controller.dart:112:30)
#2      WidgetController.dragUntilVisible.<anonymous closure> (package:flutter_test/src/controller.dart:1181:38)
<asynchronous suspension>
<asynchronous suspension>
(elided one frame from package:stack_trace)`

stable, master flutter doctor -v

[✓] Flutter (Channel stable, 2.2.3, on Mac OS X 10.15.4 19E2269 darwin-x64,
    locale en-GB)
    • Flutter version 2.2.3 at /Users/dhs/documents/fluttersdk/flutter
    • Framework revision f4abaa0735 (4 days ago), 2021-07-01 12:46:11 -0700
    • Engine revision 241c87ad80
    • Dart version 2.13.4

[✓] Android toolchain - develop for Android devices (Android SDK version 30)
    • Android SDK at /Users/dhs/Library/Android/sdk
    • Platform android-30, build-tools 30.0.3
    • ANDROID_HOME = /Users/dhs/Library/Android/sdk
    • Java binary at: /Users/dhs/Library/Application Support/JetBrains/Toolbox/apps/AndroidStudio/ch-0/202.7486908/Android
      Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6915495)
    • All Android licenses accepted.    

[✓] Xcode - develop for iOS and macOS
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.3, Build version 12C33
    • CocoaPods version 1.10.1

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 4.1)
    • Android Studio at /Users/dhs/Library/Application Support/JetBrains/Toolbox/apps/AndroidStudio/ch-0/202.7486908/Android
      Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build
      1.8.0_242-release-1644-b3-6915495)        

[✓] VS Code (version 1.57.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.21.0

[✓] Connected device (4 available)
    • SM G975F (mobile) • RZ8M802WY0X • android-arm64 • Android 10 (API 29)
    • iPhone 12 Pro Max (mobile) • A5473606-0213-4FD8-BA16-553433949729 • ios
      • com.apple.CoreSimulator.SimRuntime.iOS-14-3 (simulator)
    • macOS (desktop)            • macos                                •
      darwin-x64     • Mac OS X 10.15.4 19E2269 darwin-x64
    • Chrome (web)               • chrome                               •
      web-javascript • Google Chrome 91.0.4472.114


• No issues found!

[✓] Flutter (Channel master, 2.6.0-1.0.pre.160, on Mac OS X 10.15.4 19E2269
    darwin-x64, locale en-GB)
    • Flutter version 2.6.0-1.0.pre.160 at
      /Users/dhs/documents/fluttersdk/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision f77c453ab6 (2 hours ago), 2021-09-02 00:58:19 -0400
    • Engine revision d877a4bbcd
    • Dart version 2.15.0 (build 2.15.0-68.0.dev)

[✓] Android toolchain - develop for Android devices (Android SDK version 30)
    • Android SDK at /Users/dhs/Library/Android/sdk
    • Platform android-30, build-tools 30.0.3
    • ANDROID_HOME = /Users/dhs/Library/Android/sdk
    • Java binary at: /Users/dhs/Library/Application Support/JetBrains/Toolbox/apps/AndroidStudio/ch-0/202.7486908/Android
      Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6915495)
    • All Android licenses accepted.    

[✓] Xcode - develop for iOS and macOS
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.5.1, Build version 12E507
    • CocoaPods version 1.10.1

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 4.1)
    • Android Studio at /Users/dhs/Library/Application Support/JetBrains/Toolbox/apps/AndroidStudio/ch-0/202.7486908/Android
      Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build
      1.8.0_242-release-1644-b3-6915495)        

[✓] VS Code (version 1.57.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.21.0

[✓] Connected device (4 available)
    • SM G975F (mobile) • RZ8M802WY0X • android-arm64 • Android 10 (API 29)
    • iPhone 12 Pro Max (mobile) • A5473606-0213-4FD8-BA16-553433949729 • ios
      • com.apple.CoreSimulator.SimRuntime.iOS-14-3 (simulator)
    • macOS (desktop)            • macos                                •
      darwin-x64     • Mac OS X 10.15.4 19E2269 darwin-x64
    • Chrome (web)               • chrome                               •
      web-javascript • Google Chrome 91.0.4472.114

Labeling it as a proposal / feature request to show meaningful / helpful errors to properly debug the failing tests.

bislerium · 2022-04-16T16:59:23Z

Same issue. Solution yet?

darshankawar added the in triage Presently being triaged by the triage team label Sep 2, 2021

goderbauer added the P3 Issues that are less important to the Flutter project label Mar 23, 2022

bartekpacia mentioned this issue Apr 16, 2023

Improve error messages from Patrol methods leancodepl/patrol#1044

Open

2 tasks

flutter-triage-bot bot added team-framework Owned by Framework team triaged-framework Triaged by Framework team labels Jul 8, 2023

Hixie removed the passed first triage label Jul 8, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WidgetTester.dragUntilVisible() to show helpful errors #89336

WidgetTester.dragUntilVisible() to show helpful errors #89336

mleonhard commented Sep 2, 2021

darshankawar commented Sep 2, 2021

bislerium commented Apr 16, 2022

WidgetTester.dragUntilVisible() to show helpful errors #89336

WidgetTester.dragUntilVisible() to show helpful errors #89336

Comments

mleonhard commented Sep 2, 2021

Use case

Proposal

darshankawar commented Sep 2, 2021

bislerium commented Apr 16, 2022