Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request] Ability to run tests sequentially (scenarios AND devices) #181

Open
mstagitis opened this issue Sep 27, 2023 · 1 comment

Comments

@mstagitis
Copy link

Hello all - First off, thanks for the library! It's great.

I have a FR to allow the tests to run sequentially across test cases and devices. In the case where we have 2 scenarios and 2 devices, we should have 4 tests run sequentially.

I'm in need of this due to some limitations of test libraries/dependencies that I'm using. Thanks!

@mstagitis
Copy link
Author

mstagitis commented Oct 3, 2023

A follow-up here: It looks like there is state leaking between test cases when using GoldenToolkit. When I only run a single scenario for a single device, my test passes. Once I add an additional scenario or a second device, the tests fail with the following error:

══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
The following assertion was thrown running a test:
setState() or markNeedsBuild() called during build.
This SearchScreen widget cannot be marked as needing to build because the framework is already in
the process of building widgets. A widget can be marked as needing to be built during the build
phase only if one of its ancestors is currently building. This exception is allowed because the
framework builds parent widgets before children, which means a dirty descendant will always be
built. Otherwise, the framework might not visit this widget during this build phase.
The widget on which setState() or markNeedsBuild() was called was:
  SearchScreen
The widget which was currently being built when the offending call was made was:
  UncontrolledProviderScope

I'm using RiverPod to maintain state. In a nutshell, the test case updates a query field, which triggers a new provider value and therefore tries to rebuild the SearchScreen (mentioned in the error above). My guess is that the SearchScreen for the second device is already being built, so this error is thrown. But shouldn't these two tests have totally disjoint widget/state trees?

Note: I am able to run end-to-end tests with no issues and cannot reproduce this with the app running locally; the issue exists solely in these tests.

@mstagitis mstagitis reopened this Oct 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant