Flutter Integration testing fail when moving between tests #101031
Labels
a: tests
"flutter test", flutter_test, or one of our tests
c: crash
Stack traces logged to the console
f: integration_test
The flutter/packages/integration_test plugin
found in release: 2.10
Found to occur in 2.10
found in release: 2.13
Found to occur in 2.13
framework
flutter/packages/flutter repository. See also f: labels.
has reproducible steps
The issue has been confirmed reproducible and is ready to work on
P2
Important issues not at the top of the work list
platform-linux
Building on or for Linux specifically
platform-mac
Building on or for macOS specifically
team-framework
Owned by Framework team
tool
Affects the "flutter" command-line tool. See also t: labels.
triaged-framework
Triaged by Framework team
Comments
|
Hi @HugoCSWork, thanks for filing the issue. Can you please provide your Thank you |
danagbemava-nc
added
the
waiting for customer response
|
Hi @danagbemava-nc thanks for getting back to me so quickly. For the minimal complete product I have created a minimal project let me know if there's a better way to represent the issue. The link for that is here https://github.com/HugoCSWork/integration_config. The information below will be using a Linux machine let me know if you also want the macOS one. flutter doctor -v
flutter test integration --verboseThe image only contains information about the error I've copied the whole log of the second test underneath
Second test log (let me know if there's a better way to format this) |
github-actions
bot
removed
the
waiting for customer response
|
Issue is reproducible on I tested on macOS and iOS. iOS works just fine whereas macOS fails when switching to the second test. Steps to reproduce
Labeling for further investigation logsflutter doctor -vHi @HugoCSWork, kindly note that for logs, we prefer text to screenshots because screenshots are not searchable. If the logs are too large for github, put it into a .txt/.log file and attach it. |
danagbemava-nc
added
a: tests
danagbemava-nc
changed the title
gspencergoog
added
platform-linux
|
cc @cbracken |
darshankawar
changed the title
|
Potentially related: |
|
Is there a workaround to this issue? My team has resorted to putting all our integration tests into one file, which works, but is undesirable for a complicated application. |
|
Thanks @HugoCSWork for posting this. I am facing the same issue since long time. |
|
I've got a similar issue, please help me |
|
I had the same problem but solved it. In my case, it was failing at My product dynamically changes the BundleID with dart-define. When the Integration Test terminates, I want it to terminate like this xcrun simctl terminate 304CA91D-XXXX-XXXX-XXXX-ECFC57AFD011 com.example.app.devBut in fact, xcrun simctl terminate 304CA91D-XXXX-XXXX-XXXX-ECFC57AFD011 com.example.appThe solution was to create a Flavor for Integration Test with reference to Creating flavors for Flutter |Flutter and hardcode the BundleID as As a result, the test now succeeds when Scheme is specified with flutter test --flavor integration_testIn my case, this occurred after upgrading to Flutter 3.7.0. |
|
I experienced the same issue running multiple tests using the command : |
|
I'm not sure I have the same problem, but facing sporadic failures of the entire suite when running a command e.g. As you can see in the log, as soon as testStart event is printed, the test already errored, i.e. nothing happened. |
|
A possible workaround (but it breaks IntelliJ test reporting UI): run a single test file which calls all the main() methods from the other files. import 'package:integration_test/integration_test.dart';
import 'bookflight_test.dart' as bookflight;
import 'flightlog_test.dart' as flightlog;
import 'onboarding_test.dart' as onboarding;
void main() {
IntegrationTestWidgetsFlutterBinding.ensureInitialized();
bookflight.main();
flightlog.main();
onboarding.main();
}Maybe there is a way to load other test files dynamically but I haven't tried it yet - it would still be a workaround though :-) |
a-wallen
added a commit
to a-wallen/AppFlowy
that referenced
this issue
Mar 28, 2023
a-wallen
added a commit
to a-wallen/AppFlowy
that referenced
this issue
Mar 31, 2023
a-wallen
added a commit
to a-wallen/AppFlowy
that referenced
this issue
Apr 3, 2023
appflowy
pushed a commit
to AppFlowy-IO/AppFlowy
that referenced
this issue
Apr 4, 2023
* feat: add archive for compression * feat: add service to manage zipped work spaces * feat: export service in barrel file * feat: ignore .ephemeral directory * feat: add first compressed workspace file * fix: directory path was wrong * feat: add a somewhat useful test * fix: move to same file (delete later) * fix: use script path vs. working directory for CI * fix: read from asset bundle instead of file system * fix: workaround to run integration in multiple files on desktop (flutter/flutter#101031 * feat: remove .ephemeral from .gitignore, no longer created * feat: document test changes * fix: lucas suggestion * feat: mark assets as excluded in pubspec.yaml * feat: add class for build utilities * feat: add script runner for release builds * feat: add build script as task in flowy project * fix: typo in pubspec.yaml * chore: use constants for exclude tag * feat: add appversion as argument to build tool * feat: use dart script in release.yml * chore: remove task * fix: careless error Co-authored-by: Mihir <84044317+squidrye@users.noreply.github.com> * feat: add translations for view of * fix: typo in getAllDatabase * feat: add view of database * fix: remove unused import * fix: use effective dart typing * fix: insertPage marked as async, should return future * fix: Remove multi-line string * fix: ref can be null * fix: unused imports caused analyzer to fail * feat: also fix. Add empty document as option and change name to _name * chore: move referenced database tests to empty document test file * feat: add test utilities * feat: add new integration test on an empty document * feat: register test in runner * fix: missing reference in insert_page_command * fix: analyzer errors --------- Co-authored-by: Mihir <84044317+squidrye@users.noreply.github.com>
LucasXu0
pushed a commit
to LucasXu0/AppFlowy
that referenced
this issue
Apr 6, 2023
…lowy-IO#2083) * feat: add archive for compression * feat: add service to manage zipped work spaces * feat: export service in barrel file * feat: ignore .ephemeral directory * feat: add first compressed workspace file * fix: directory path was wrong * feat: add a somewhat useful test * fix: move to same file (delete later) * fix: use script path vs. working directory for CI * fix: read from asset bundle instead of file system * fix: workaround to run integration in multiple files on desktop (flutter/flutter#101031 * feat: remove .ephemeral from .gitignore, no longer created * feat: document test changes * fix: lucas suggestion * feat: mark assets as excluded in pubspec.yaml * feat: add class for build utilities * feat: add script runner for release builds * feat: add build script as task in flowy project * fix: typo in pubspec.yaml * chore: use constants for exclude tag * feat: add appversion as argument to build tool * feat: use dart script in release.yml * chore: remove task * fix: careless error Co-authored-by: Mihir <84044317+squidrye@users.noreply.github.com> * feat: add translations for view of * fix: typo in getAllDatabase * feat: add view of database * fix: remove unused import * fix: use effective dart typing * fix: insertPage marked as async, should return future * fix: Remove multi-line string * fix: ref can be null * fix: unused imports caused analyzer to fail * feat: also fix. Add empty document as option and change name to _name * chore: move referenced database tests to empty document test file * feat: add test utilities * feat: add new integration test on an empty document * feat: register test in runner * fix: missing reference in insert_page_command * fix: analyzer errors --------- Co-authored-by: Mihir <84044317+squidrye@users.noreply.github.com>
|
Any updates on this? I am still running into this error (on macOS) when using |
flutter-triage-bot
bot
added
multiteam-retriage-candidate
team-framework
sku0x20
added a commit
to sku0x20/flutter_spike_state_management
that referenced
this issue
Oct 29, 2023
…ation tests. anyways, that was a long way. this was in such a bad shape. learned things, again. still so much. but with this, state idea i got. ui i need to see. ui is never ending. but now i have an idea how to approach and can always look back here for insights. quick look for further lookup. starting idea. and 'this exists'. still this has multiple issues. like that delay of 8secs. idk, i have no idea how to solve that rn. fine for now tbh. we will see. i liked working with windows. and how just easily it is to add support for android or windows to existing project. flutter create --platforms=android . but in any case i liked this spike. learned quite good. was doing stuff badly last time. wow. still so much to learn. flutter/flutter#101031 flutter/samples#1000 jtdLab/rapid#114
This comment was marked as off-topic.
This comment was marked as off-topic.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
There seems to be issues running multiple integration tests in separate files on desktop applications, where it crashes when moving between tests. It opens the second application to start testing but will be blank and the test will fail.
I have tested this on both macOS (M1) and Linux (ARM). The tests do work on an iOS emulator so it seems to definitely be a desktop specific issue.
Steps to Reproduce
flutter integration testError Results
The text was updated successfully, but these errors were encountered: