Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[iOS] Add iOS17 tests again #19941

Merged
merged 3 commits into from Feb 8, 2024
Merged

[iOS] Add iOS17 tests again #19941

merged 3 commits into from Feb 8, 2024

Conversation

rmarinho
Copy link
Member

@rmarinho rmarinho commented Jan 17, 2024

Description of Change

Adds UITests and Devicestests on iOS17

fixes #19134

BretJohnson
BretJohnson previously approved these changes Feb 1, 2024
@@ -184,6 +184,7 @@ protected override void Init()

#if __IOS__
[Test]
[Ignore("This test fails intermittently, especially on iOS 17; ignore until we can fix it")]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should be good now if you want to remove this ignore.

@rmarinho
Copy link
Member Author

rmarinho commented Feb 7, 2024

/rebase

rmarinho and others added 2 commits February 7, 2024 16:55
…20256)

* Add visual test support for multiple devices for a single platform

Fixes: #20113

We need this immediately to be able to test both iOS 17.2 on iPhone 15
(the new default) while still testing iOS 16.2 on iPhone X (the old test device).
But this is general support that should meet our future needs as more devices
are added (at least that's the intention).

The comment below explains the scheme:

 /*
 Determine the environmentName, used as the directory name for visual testing snaphots. Here are the rules/conventions:
 - Names are lower case, no spaces.
 - By default, the name matches the platform (android, ios, windows, or mac).
 - Each platform has a default device (or set of devices) - if the snapshot matches the default no suffix is needed (e.g. just ios).
 - If tests are run on secondary devices that produce different snapshots, the device name is used as suffix (e.g. ios-iphonex).
 - If tests are run on secondary devices with multiple OS versions that produce different snapshots, both device name and os version are
   used as a suffix (e.g. ios-iphonex-16_4). We don't have any cases of this today but may eventually. The device name comes first here,
   before os version, because most visual testing differences come from different sceen size (a device thing), not OS version differences,
   but both can happen.
*/

* Rename ios -> ios-iphonex directory

The ios directory will be replaced with iPhone 15
generated images, as it's the new default.

* Update to include deviceName in error messages

* Remove the xharness suffix and update iOS test device names

* Set DeviceName for Android

* Fix cropping on iPhone 15

* Add iPhone 15 snapshots

* Ignore FlyoutHeaderWithZeroMarginShouldHaveNoY for now
@rmarinho rmarinho merged commit ce80f53 into main Feb 8, 2024
40 of 49 checks passed
@rmarinho rmarinho deleted the add-ios17-again branch February 8, 2024 22:21
@github-actions github-actions bot locked and limited conversation to collaborators Mar 10, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area/testing Unit tests, device tests platform/iOS 🍎
Projects
None yet
Development

Successfully merging this pull request may close these issues.

iOS17 tests are not stable on CI
4 participants