Skip to content

End to End Tests

End to End Tests #7878

Triggered via workflow run March 6, 2024 14:35
@RiotRobotRiotRobot
completed 942fabc
Status Success
Total duration 18m 41s
Artifacts 9

end-to-end-tests.yaml

on: workflow_run
Prepare
4s
Prepare
Matrix: tests
Report results
1m 3s
Report results
Fit to window
Zoom out
Zoom in

Annotations

9 errors, 7 warnings, and 1 notice
Report results
Pull Request not found!
Report results
The template is not valid. matrix-org/pr-details-action/v1.3/action.yml (Line: 15, Col: 12): Error reading JToken from JsonReader. Path '', line 0, position 0.,matrix-org/pr-details-action/v1.3/action.yml (Line: 18, Col: 12): Error reading JToken from JsonReader. Path '', line 0, position 0.,matrix-org/pr-details-action/v1.3/action.yml (Line: 21, Col: 12): Error reading JToken from JsonReader. Path '', line 0, position 0.
[Legacy Crypto] › audio-player/audio-player.spec.ts:152:9 › Audio player › should be correctly rendered - high contrast theme: matrix-react-sdk/playwright/e2e/audio-player/audio-player.spec.ts#L103
1) [Legacy Crypto] › audio-player/audio-player.spec.ts:152:9 › Audio player › should be correctly rendered - high contrast theme Error: Screenshot comparison failed: 154 pixels (ratio 0.01 of all image pixels) are different. Expected: /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/snapshots/audio-player/audio-player.spec.ts/Selected-EventTile-of-audio-player--high-contrast--irc-layout-linux.png Received: /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/test-results/audio-player-audio-player-Audio-player-should-be-correctly-rendered---high-contrast-theme-Legacy-Crypto/Selected-EventTile-of-audio-player--high-contrast--irc-layout-actual.png Diff: /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/test-results/audio-player-audio-player-Audio-player-should-be-correctly-rendered---high-contrast-theme-Legacy-Crypto/Selected-EventTile-of-audio-player--high-contrast--irc-layout-diff.png Call log: - page._expectScreenshot with timeout 5000ms - verifying given screenshot expectation - waiting for locator('.mx_EventTile_last') - locator resolved to <li tabindex="-1" aria-live="off" data-self="true" d…>…</li> - taking element screenshot - disabled all CSS animations - waiting for fonts to load... - fonts loaded - attempting scroll into view action - waiting for element to be stable - 154 pixels (ratio 0.01 of all image pixels) are different. - waiting 100ms before taking screenshot - waiting for locator('.mx_EventTile_last') - locator resolved to <li tabindex="-1" aria-live="off" data-self="true" d…>…</li> - taking element screenshot - disabled all CSS animations - waiting for fonts to load... - fonts loaded - attempting scroll into view action - waiting for element to be stable - captured a stable screenshot - 154 pixels (ratio 0.01 of all image pixels) are different. 101 | 102 | // Take a snapshot of mx_EventTile_last on IRC layout > 103 | await expect(page.locator(".mx_EventTile_last")).toMatchScreenshot( | ^ 104 | `${detail.replaceAll(" ", "-")}-irc-layout.png`, 105 | screenshotOptions, 106 | ); at takeSnapshots (/home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/e2e/audio-player/audio-player.spec.ts:103:58) at /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/e2e/audio-player/audio-player.spec.ts:165:9
[Legacy Crypto] › audio-player/audio-player.spec.ts:239:9 › Audio player › should support creating a reply chain with multiple audio files: matrix-react-sdk/playwright/e2e/audio-player/audio-player.spec.ts#L1
2) [Legacy Crypto] › audio-player/audio-player.spec.ts:239:9 › Audio player › should support creating a reply chain with multiple audio files Test timeout of 30000ms exceeded.
[Legacy Crypto] › audio-player/audio-player.spec.ts:239:9 › Audio player › should support creating a reply chain with multiple audio files: matrix-react-sdk/playwright/e2e/audio-player/audio-player.spec.ts#L250
2) [Legacy Crypto] › audio-player/audio-player.spec.ts:239:9 › Audio player › should support creating a reply chain with multiple audio files Error: locator.click: Test timeout of 30000ms exceeded. Call log: - waiting for locator('.mx_EventTile_last').getByRole('button', { name: 'Reply', exact: true }) 248 | const clickButtonReply = async () => { 249 | await tile.hover(); > 250 | await tile.getByRole("button", { name: "Reply", exact: true }).click(); | ^ 251 | }; 252 | 253 | await uploadFile(page, "playwright/sample-files/upload-first.ogg"); at clickButtonReply (/home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/e2e/audio-player/audio-player.spec.ts:250:76) at /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/e2e/audio-player/audio-player.spec.ts:266:9
[Legacy Crypto] › presence/presence.spec.ts:26:13 › Presence tests › bob unreachable › renders unreachable presence state correctly: matrix-react-sdk/playwright/e2e/presence/presence.spec.ts#L62
3) [Legacy Crypto] › presence/presence.spec.ts:26:13 › Presence tests › bob unreachable › renders unreachable presence state correctly Error: Timed out 5000ms waiting for expect(locator).toContainText(expected) Locator: locator('.mx_EntityTile_unreachable') Expected string: "Bob" Received string: "" Call log: - expect.toContainText with timeout 5000ms - waiting for locator('.mx_EntityTile_unreachable') 60 | await page.getByRole("button", { name: "Room info" }).click(); 61 | await page.locator(".mx_RightPanel").getByText("People").click(); > 62 | await expect(page.locator(".mx_EntityTile_unreachable")).toContainText("Bob"); | ^ 63 | await expect(page.locator(".mx_EntityTile_unreachable")).toContainText("User's server unreachable"); 64 | }); 65 | }); at /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/e2e/presence/presence.spec.ts:62:70
[Legacy Crypto] › room/room.spec.ts:64:9 › Room Directory › should memorize the timeline position when switch Room A -> Room B -> Room A: matrix-react-sdk/playwright/e2e/room/room.spec.ts#L100
4) [Legacy Crypto] › room/room.spec.ts:64:9 › Room Directory › should memorize the timeline position when switch Room A -> Room B -> Room A Error: Timed out 5000ms waiting for expect(locator).toBeInViewport() Locator: locator('.mx_MTextBody .mx_EventTile_body').getByText('test0') Expected: in viewport Received: outside viewport Call log: - expect.toBeInViewport with timeout 5000ms - waiting for locator('.mx_MTextBody .mx_EventTile_body').getByText('test0') 98 | // The timeline should display the first message 99 | // The previous position before switching to Room B should be remembered > 100 | await expect(page.locator(".mx_MTextBody .mx_EventTile_body").getByText("test0")).toBeInViewport(); | ^ 101 | }); 102 | }); 103 | at /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/e2e/room/room.spec.ts:100:91
[Legacy Crypto] › user-onboarding/user-onboarding-new.spec.ts:43:9 › User Onboarding (new user) › app download dialog: matrix-react-sdk/playwright/e2e/user-onboarding/user-onboarding-new.spec.ts#L48
5) [Legacy Crypto] › user-onboarding/user-onboarding-new.spec.ts:43:9 › User Onboarding (new user) › app download dialog Error: Screenshot comparison failed: 8 pixels (ratio 0.01 of all image pixels) are different. Expected: /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/snapshots/user-onboarding/user-onboarding-new.spec.ts/User-Onboarding-new-user-app-download-dialog-1-linux.png Received: /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/test-results/user-onboarding-user-onboarding-new-User-Onboarding-new-user-app-download-dialog-Legacy-Crypto/User-Onboarding-new-user-app-download-dialog-1-actual.png Diff: /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/test-results/user-onboarding-user-onboarding-new-User-Onboarding-new-user-app-download-dialog-Legacy-Crypto/User-Onboarding-new-user-app-download-dialog-1-diff.png Call log: - page._expectScreenshot with timeout 5000ms - verifying given screenshot expectation - waiting for locator('.mx_Dialog') - locator resolved to <div class="mx_Dialog">…</div> - taking element screenshot - disabled all CSS animations - waiting for fonts to load... - fonts loaded - attempting scroll into view action - waiting for element to be stable - 8 pixels (ratio 0.01 of all image pixels) are different. - waiting 100ms before taking screenshot - waiting for locator('.mx_Dialog') - locator resolved to <div class="mx_Dialog">…</div> - taking element screenshot - disabled all CSS animations - waiting for fonts to load... - fonts loaded - attempting scroll into view action - waiting for element to be stable - captured a stable screenshot - 8 pixels (ratio 0.01 of all image pixels) are different. 46 | page.getByRole("dialog").getByRole("heading", { level: 1, name: "Download Element" }), 47 | ).toBeVisible(); > 48 | await expect(page.locator(".mx_Dialog")).toMatchScreenshot(); | ^ 49 | }); 50 | 51 | test("using find friends action should increase progress", async ({ page, homeserver }) => { at /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/e2e/user-onboarding/user-onboarding-new.spec.ts:48:50
[Rust Crypto] › audio-player/audio-player.spec.ts:146:9 › Audio player › should be correctly rendered - light theme with monospace font: matrix-react-sdk/playwright/e2e/audio-player/audio-player.spec.ts#L81
6) [Rust Crypto] › audio-player/audio-player.spec.ts:146:9 › Audio player › should be correctly rendered - light theme with monospace font Error: expect(received).toBeGreaterThan(expected) Expected: > 0 Received: 0 79 | 80 | // Check the status of the seek bar > 81 | expect(await page.locator(".mx_AudioPlayer_seek input[type='range']").count()).toBeGreaterThan(0); | ^ 82 | 83 | // Enable IRC layout 84 | await app.settings.setValue("layout", null, SettingLevel.DEVICE, Layout.IRC); at takeSnapshots (/home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/e2e/audio-player/audio-player.spec.ts:81:88) at /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/e2e/audio-player/audio-player.spec.ts:149:9
🎭 Playwright Run Summary
6 flaky [Legacy Crypto] › audio-player/audio-player.spec.ts:152:9 › Audio player › should be correctly rendered - high contrast theme [Legacy Crypto] › audio-player/audio-player.spec.ts:239:9 › Audio player › should support creating a reply chain with multiple audio files [Legacy Crypto] › presence/presence.spec.ts:26:13 › Presence tests › bob unreachable › renders unreachable presence state correctly [Legacy Crypto] › room/room.spec.ts:64:9 › Room Directory › should memorize the timeline position when switch Room A -> Room B -> Room A [Legacy Crypto] › user-onboarding/user-onboarding-new.spec.ts:43:9 › User Onboarding (new user) › app download dialog [Rust Crypto] › audio-player/audio-player.spec.ts:146:9 › Audio player › should be correctly rendered - light theme with monospace font 121 skipped 655 passed (16.3m)
Prepare
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: Sibz/github-status-action@071b5370da85afbb16637d6eed8524a06bc2053e. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Report results
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: Sibz/github-status-action@071b5370da85afbb16637d6eed8524a06bc2053e, bobheadxi/deployments@88ce5600046c82542f8246ac287d0a53c461bca3, actions/github-script@v6, nwtgck/actions-netlify@7a92f00dde8c92a5a9e8385ec2919775f7647352. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Slow Test: [Legacy Crypto] › read-receipts/redactions.spec.ts#L1
[Legacy Crypto] › read-receipts/redactions.spec.ts took 6.0m
Slow Test: [Rust Crypto] › read-receipts/redactions.spec.ts#L1
[Rust Crypto] › read-receipts/redactions.spec.ts took 5.9m
Slow Test: [Rust Crypto] › read-receipts/new-messages.spec.ts#L1
[Rust Crypto] › read-receipts/new-messages.spec.ts took 3.8m
Slow Test: [Rust Crypto] › read-receipts/high-level.spec.ts#L1
[Rust Crypto] › read-receipts/high-level.spec.ts took 3.6m
Slow Test: [Legacy Crypto] › read-receipts/new-messages.spec.ts#L1
[Legacy Crypto] › read-receipts/new-messages.spec.ts took 3.4m

Artifacts

Produced during runtime
Name Size
all-blob-reports-1 Expired
9.82 MB
all-blob-reports-2 Expired
3.02 MB
all-blob-reports-3 Expired
6.25 MB
all-blob-reports-4 Expired
6.61 MB
all-blob-reports-5 Expired
1.19 MB
all-blob-reports-6 Expired
588 KB
all-blob-reports-7 Expired
949 KB
all-blob-reports-8 Expired
1.1 MB
html-report--attempt-1 Expired
24.5 MB