New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run multiple UI tests without restarting ControlGallery #539

Merged
merged 21 commits into from Nov 16, 2016

Conversation

Projects
None yet
5 participants
@hartez
Member

hartez commented Nov 16, 2016

These changes are aimed at running the UI test suite more quickly and efficiently.

  • Update the UI Test package to 2.0.0 (from 2.0.0-beta5)
  • Remove empty UI tests
  • Fix race conditions in some tests which cause intermittent failures
  • Consolidate Issues UI tests to the "Issues" namespace
  • Update the base test fixtures to launch a single instance of the ControlGallery to run tests against
  • Fix up some tests which don't behave well when run before/after other tests
  • Add exported back door method to navigate to Issues tests without going through the "enter text into search box, hit search button" process

Most of the changed files are simply namespace moves; by putting all of the "issues" under a single namespace, we can run a single NUnit SetUpFixture to install and launch the ControlGallery and then run all the "issues" tests against that single instance, rather than shutting down and restarting the ControlGallery between every single test. This drastically shortens the length of a test run.

The rest of the changes to issues are either bug fixes, removal of empty tests, or adjustments to make the tests work well when run in sequence (rather than in complete isolation as they did before).

The main changes are to TestPages.cs, BaseTestFixture.cs, and App.cs.

hartez added some commits Nov 1, 2016

Fix race condition in 39530 test
Remove master page nesting when doing direct nav for UI tests

@jassmith jassmith merged commit 122f0e3 into master Nov 16, 2016

1 of 4 checks passed

Android-UITests Finished TeamCity Build Xamarin.Forms :: Debug :: Cycle 9 :: UI Tests :: OSX Test Cloud Package - Run Android 6.0.1 : Snapshot dependency …
Details
OSX-Debug-C8 Finished TeamCity Build Xamarin.Forms :: Debug :: Cycle 9 :: OSX Debug : Exit code 1
Details
iOS9-UITests Finished TeamCity Build Xamarin.Forms :: Debug :: Cycle 9 :: UI Tests :: OSX Test Cloud Package - Run iOS Unified iOS9 : Snapshot dependen…
Details
Windows-Debug-C8 Finished TeamCity Build Xamarin.Forms :: Debug :: Cycle 9 :: Windows Debug : Tests passed: 3567, ignored: 10
Details
@adrianknight89

This comment has been minimized.

Show comment
Hide comment
@adrianknight89

adrianknight89 Nov 26, 2016

Contributor

This might fix the NRE mentioned in https://bugzilla.xamarin.com/show_bug.cgi?id=47637.

Contributor

adrianknight89 commented Nov 26, 2016

This might fix the NRE mentioned in https://bugzilla.xamarin.com/show_bug.cgi?id=47637.

@hartez

This comment has been minimized.

Show comment
Hide comment
@hartez

hartez Nov 26, 2016

Member

@adrianknight89 Where are you seeing a fix for 47637 in this PR?

Member

hartez commented Nov 26, 2016

@adrianknight89 Where are you seeing a fix for 47637 in this PR?

@adrianknight89

This comment has been minimized.

Show comment
Hide comment
@adrianknight89

adrianknight89 Nov 26, 2016

Contributor

In the ViewDidDisappear method of master detail renderers for both Phone and Tablet, PageController now has ? to check against null. There is no repro in that bug description, but you could see the stack trace.

Contributor

adrianknight89 commented Nov 26, 2016

In the ViewDidDisappear method of master detail renderers for both Phone and Tablet, PageController now has ? to check against null. There is no repro in that bug description, but you could see the stack trace.

@hartez

This comment has been minimized.

Show comment
Hide comment
@hartez

hartez Nov 28, 2016

Member

Oh, cool, nice catch. I'd forgotten about that. I'll add 47637 to my list so I can write a reliable UI test for that specific situation and bugzilla issue. (I only ran into it because of a race condition in another UI test.)

Member

hartez commented Nov 28, 2016

Oh, cool, nice catch. I'd forgotten about that. I'll add 47637 to my list so I can write a reliable UI test for that specific situation and bugzilla issue. (I only ran into it because of a race condition in another UI test.)

@hartez hartez deleted the single-app-instance branch May 16, 2017

@samhouts samhouts added this to the 2.3.4 milestone Jun 27, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment