Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch UI tests back to the perf-only runner. #3349

Merged
merged 1 commit into from
Sep 30, 2024
Merged

Conversation

pixlwave
Copy link
Member

@pixlwave pixlwave commented Sep 30, 2024

They're not passing yet, but this at least lets CI report the failures we need to fix.

@pixlwave pixlwave requested a review from a team as a code owner September 30, 2024 09:16
@pixlwave pixlwave requested review from Velin92 and removed request for a team September 30, 2024 09:16
Copy link

github-actions bot commented Sep 30, 2024

Warnings
⚠️ Some of the commits are missing ticket numbers. Please consider squashing all commits that don't have a tracking number.

Generated by 🚫 Danger Swift against 429f7c2

Copy link

sonarcloud bot commented Sep 30, 2024

@pixlwave pixlwave added the pr-build for changes related to build, tools, CI/CD label Sep 30, 2024
Copy link

codecov bot commented Sep 30, 2024

❌ 2 Tests Failed:

Tests completed Failed Passed Skipped
837 2 835 0
View the full list of 2 ❄️ flaky tests
PreviewTests test_identityConfirmationScreen()

Flake rate in main: 16.81% (Passed 94 times, Failed 19 times)

Stack Traces | 2.1s run time
failed - Snapshot does not match reference.

@−
"file:.../__Snapshots__/PreviewTests/test_identityConfirmationScreen-iPhone-16-pseudo.1.png"
@+
"file:.../tmp/PreviewTests/test_identityConfirmationScreen-iPhone-16-pseudo.1.png"

To configure output for a custom diff tool, use 'withSnapshotTesting'. For example:

    withSnapshotTesting(diffTool: .ksdiff) {
      // ...
    }

The percentage of pixels that match 0.94415283 is less than required 1.0
The lowest perceptual color precision 0.39937502 is less than required 0.98 (.../PreviewTests/Sources/PreviewTests.swift:84)
PreviewTests test_joinRoomScreen()

Flake rate in main: 20.75% (Passed 84 times, Failed 22 times)

Stack Traces | 3s run time
failed - Snapshot "Unknown" does not match reference.

@−
"file:.../__Snapshots__/PreviewTests/test_joinRoomScreen-iPhone-16-en-GB.Unknown.png"
@+
"file:.../tmp/PreviewTests/test_joinRoomScreen-iPhone-16-en-GB.Unknown.png"

To configure output for a custom diff tool, use 'withSnapshotTesting'. For example:

    withSnapshotTesting(diffTool: .ksdiff) {
      // ...
    }

The percentage of pixels that match 0.9661255 is less than required 1.0
The lowest perceptual color precision 0.39999998 is less than required 0.98 (.../PreviewTests/Sources/PreviewTests.swift:84)

To view individual test run time comparison to the main branch, go to the Test Analytics Dashboard

@pixlwave pixlwave merged commit 5d35a74 into develop Sep 30, 2024
8 of 9 checks passed
@pixlwave pixlwave deleted the doug/ui-tests branch September 30, 2024 10:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr-build for changes related to build, tools, CI/CD
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants