Skip to content

End to End Tests

End to End Tests #7399

Triggered via workflow run February 14, 2024 13:44
@floriandurosflorianduros
completed 35ad92b
Status Failure
Total duration 19m 17s
Artifacts 9

end-to-end-tests.yaml

on: workflow_run
Prepare
2s
Prepare
Matrix: tests
Report results
38s
Report results
Fit to window
Zoom out
Zoom in

Annotations

12 errors, 15 warnings, and 1 notice
Run Tests 3/8
The process '/usr/bin/xvfb-run' failed with exit code 1
Run Tests 7/8
The process '/usr/bin/xvfb-run' failed with exit code 1
[Legacy Crypto] › settings/appearance-user-settings-tab.spec.ts:25:9 › Appearance user settings tab › should be rendered properly: matrix-react-sdk/playwright/e2e/settings/appearance-user-settings-tab.spec.ts#L36
1) [Legacy Crypto] › settings/appearance-user-settings-tab.spec.ts:25:9 › Appearance user settings tab › should be rendered properly Error: Screenshot comparison failed: 7702 pixels (ratio 0.02 of all image pixels) are different. Expected: /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/snapshots/settings/appearance-user-settings-tab.spec.ts/appearance-tab-linux.png Received: /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/test-results/settings-appearance-user-settings-tab-Appearance-user-settings-tab-should-be-rendered-properly-Legacy-Crypto/appearance-tab-actual.png Diff: /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/test-results/settings-appearance-user-settings-tab-Appearance-user-settings-tab-should-be-rendered-properly-Legacy-Crypto/appearance-tab-diff.png Call log: - page._expectScreenshot with timeout 5000ms - verifying given screenshot expectation - waiting for locator('.mx_Dialog').filter({ has: locator('.mx_UserSettingsDialog') }) - locator resolved to <div class="mx_Dialog">…</div> - taking element screenshot - disabled all CSS animations - waiting for fonts to load... - fonts loaded - waiting for element to be visible and stable - element is visible and stable - 7702 pixels (ratio 0.02 of all image pixels) are different. - waiting 100ms before taking screenshot - waiting for locator('.mx_Dialog').filter({ has: locator('.mx_UserSettingsDialog') }) - locator resolved to <div class="mx_Dialog">…</div> - taking element screenshot - disabled all CSS animations - waiting for fonts to load... - fonts loaded - waiting for element to be visible and stable - element is visible and stable - captured a stable screenshot - 7702 pixels (ratio 0.02 of all image pixels) are different. 34 | await expect(tab.getByRole("button", { name: "Hide advanced" })).toBeVisible(); 35 | > 36 | await expect(tab).toMatchScreenshot("appearance-tab.png"); | ^ 37 | }); 38 | 39 | test("should support switching layouts", async ({ page, user, app }) => { at /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/e2e/settings/appearance-user-settings-tab.spec.ts:36:27
[Legacy Crypto] › settings/appearance-user-settings-tab.spec.ts:25:9 › Appearance user settings tab › should be rendered properly: matrix-react-sdk/playwright/e2e/settings/appearance-user-settings-tab.spec.ts#L36
1) [Legacy Crypto] › settings/appearance-user-settings-tab.spec.ts:25:9 › Appearance user settings tab › should be rendered properly Retry #1 ─────────────────────────────────────────────────────────────────────────────────────── Error: Screenshot comparison failed: 7702 pixels (ratio 0.02 of all image pixels) are different. Expected: /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/snapshots/settings/appearance-user-settings-tab.spec.ts/appearance-tab-linux.png Received: /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/test-results/settings-appearance-user-settings-tab-Appearance-user-settings-tab-should-be-rendered-properly-Legacy-Crypto-retry1/appearance-tab-actual.png Diff: /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/test-results/settings-appearance-user-settings-tab-Appearance-user-settings-tab-should-be-rendered-properly-Legacy-Crypto-retry1/appearance-tab-diff.png Call log: - page._expectScreenshot with timeout 5000ms - verifying given screenshot expectation - waiting for locator('.mx_Dialog').filter({ has: locator('.mx_UserSettingsDialog') }) - locator resolved to <div class="mx_Dialog">…</div> - taking element screenshot - disabled all CSS animations - waiting for fonts to load... - fonts loaded - waiting for element to be visible and stable - element is visible and stable - 7702 pixels (ratio 0.02 of all image pixels) are different. - waiting 100ms before taking screenshot - waiting for locator('.mx_Dialog').filter({ has: locator('.mx_UserSettingsDialog') }) - locator resolved to <div class="mx_Dialog">…</div> - taking element screenshot - disabled all CSS animations - waiting for fonts to load... - fonts loaded - waiting for element to be visible and stable - element is visible and stable - captured a stable screenshot - 7702 pixels (ratio 0.02 of all image pixels) are different. 34 | await expect(tab.getByRole("button", { name: "Hide advanced" })).toBeVisible(); 35 | > 36 | await expect(tab).toMatchScreenshot("appearance-tab.png"); | ^ 37 | }); 38 | 39 | test("should support switching layouts", async ({ page, user, app }) => { at /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/e2e/settings/appearance-user-settings-tab.spec.ts:36:27
[Legacy Crypto] › settings/appearance-user-settings-tab.spec.ts:25:9 › Appearance user settings tab › should be rendered properly: matrix-react-sdk/playwright/e2e/settings/appearance-user-settings-tab.spec.ts#L36
1) [Legacy Crypto] › settings/appearance-user-settings-tab.spec.ts:25:9 › Appearance user settings tab › should be rendered properly Retry #2 ─────────────────────────────────────────────────────────────────────────────────────── Error: Screenshot comparison failed: 7702 pixels (ratio 0.02 of all image pixels) are different. Expected: /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/snapshots/settings/appearance-user-settings-tab.spec.ts/appearance-tab-linux.png Received: /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/test-results/settings-appearance-user-settings-tab-Appearance-user-settings-tab-should-be-rendered-properly-Legacy-Crypto-retry2/appearance-tab-actual.png Diff: /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/test-results/settings-appearance-user-settings-tab-Appearance-user-settings-tab-should-be-rendered-properly-Legacy-Crypto-retry2/appearance-tab-diff.png Call log: - page._expectScreenshot with timeout 5000ms - verifying given screenshot expectation - waiting for locator('.mx_Dialog').filter({ has: locator('.mx_UserSettingsDialog') }) - locator resolved to <div class="mx_Dialog">…</div> - taking element screenshot - disabled all CSS animations - waiting for fonts to load... - fonts loaded - waiting for element to be visible and stable - element is visible and stable - 7702 pixels (ratio 0.02 of all image pixels) are different. - waiting 100ms before taking screenshot - waiting for locator('.mx_Dialog').filter({ has: locator('.mx_UserSettingsDialog') }) - locator resolved to <div class="mx_Dialog">…</div> - taking element screenshot - disabled all CSS animations - waiting for fonts to load... - fonts loaded - waiting for element to be visible and stable - element is visible and stable - captured a stable screenshot - 7702 pixels (ratio 0.02 of all image pixels) are different. 34 | await expect(tab.getByRole("button", { name: "Hide advanced" })).toBeVisible(); 35 | > 36 | await expect(tab).toMatchScreenshot("appearance-tab.png"); | ^ 37 | }); 38 | 39 | test("should support switching layouts", async ({ page, user, app }) => { at /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/e2e/settings/appearance-user-settings-tab.spec.ts:36:27
[Legacy Crypto] › settings/appearance-user-settings-tab.spec.ts:76:9 › Appearance user settings tab › should support changing font size by clicking the font slider: matrix-react-sdk/playwright/e2e/settings/appearance-user-settings-tab.spec.ts#L90
2) [Legacy Crypto] › settings/appearance-user-settings-tab.spec.ts:76:9 › Appearance user settings tab › should support changing font size by clicking the font slider Error: Timed out 5000ms waiting for expect(locator).toBeVisible() Locator: getByTestId('mx_AppearanceUserSettingsTab').locator('.mx_FontScalingPanel_fontSlider').locator('input[value=\'11\']') Expected: visible Received: hidden Call log: - expect.toBeVisible with timeout 5000ms - waiting for getByTestId('mx_AppearanceUserSettingsTab').locator('.mx_FontScalingPanel_fontSlider').locator('input[value=\'11\']') 88 | const MIN_FONT_SIZE = 11; 89 | // Assert that the smallest font size is selected > 90 | await expect(fontSliderSection.locator(`input[value='${MIN_FONT_SIZE}']`)).toBeVisible(); | ^ 91 | await expect( 92 | fontSliderSection.locator("output .mx_Slider_selection_label", { hasText: String(MIN_FONT_SIZE) }), 93 | ).toBeVisible(); at /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/e2e/settings/appearance-user-settings-tab.spec.ts:90:84
[Legacy Crypto] › settings/appearance-user-settings-tab.spec.ts:76:9 › Appearance user settings tab › should support changing font size by clicking the font slider: matrix-react-sdk/playwright/e2e/settings/appearance-user-settings-tab.spec.ts#L90
2) [Legacy Crypto] › settings/appearance-user-settings-tab.spec.ts:76:9 › Appearance user settings tab › should support changing font size by clicking the font slider Retry #1 ─────────────────────────────────────────────────────────────────────────────────────── Error: Timed out 5000ms waiting for expect(locator).toBeVisible() Locator: getByTestId('mx_AppearanceUserSettingsTab').locator('.mx_FontScalingPanel_fontSlider').locator('input[value=\'11\']') Expected: visible Received: hidden Call log: - expect.toBeVisible with timeout 5000ms - waiting for getByTestId('mx_AppearanceUserSettingsTab').locator('.mx_FontScalingPanel_fontSlider').locator('input[value=\'11\']') 88 | const MIN_FONT_SIZE = 11; 89 | // Assert that the smallest font size is selected > 90 | await expect(fontSliderSection.locator(`input[value='${MIN_FONT_SIZE}']`)).toBeVisible(); | ^ 91 | await expect( 92 | fontSliderSection.locator("output .mx_Slider_selection_label", { hasText: String(MIN_FONT_SIZE) }), 93 | ).toBeVisible(); at /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/e2e/settings/appearance-user-settings-tab.spec.ts:90:84
[Legacy Crypto] › settings/appearance-user-settings-tab.spec.ts:76:9 › Appearance user settings tab › should support changing font size by clicking the font slider: matrix-react-sdk/playwright/e2e/settings/appearance-user-settings-tab.spec.ts#L90
2) [Legacy Crypto] › settings/appearance-user-settings-tab.spec.ts:76:9 › Appearance user settings tab › should support changing font size by clicking the font slider Retry #2 ─────────────────────────────────────────────────────────────────────────────────────── Error: Timed out 5000ms waiting for expect(locator).toBeVisible() Locator: getByTestId('mx_AppearanceUserSettingsTab').locator('.mx_FontScalingPanel_fontSlider').locator('input[value=\'11\']') Expected: visible Received: hidden Call log: - expect.toBeVisible with timeout 5000ms - waiting for getByTestId('mx_AppearanceUserSettingsTab').locator('.mx_FontScalingPanel_fontSlider').locator('input[value=\'11\']') 88 | const MIN_FONT_SIZE = 11; 89 | // Assert that the smallest font size is selected > 90 | await expect(fontSliderSection.locator(`input[value='${MIN_FONT_SIZE}']`)).toBeVisible(); | ^ 91 | await expect( 92 | fontSliderSection.locator("output .mx_Slider_selection_label", { hasText: String(MIN_FONT_SIZE) }), 93 | ).toBeVisible(); at /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/e2e/settings/appearance-user-settings-tab.spec.ts:90:84
[Legacy Crypto] › settings/appearance-user-settings-tab.spec.ts:110:9 › Appearance user settings tab › should disable font size slider when custom font size is used: matrix-react-sdk/playwright/e2e/settings/appearance-user-settings-tab.spec.ts#L1
3) [Legacy Crypto] › settings/appearance-user-settings-tab.spec.ts:110:9 › Appearance user settings tab › should disable font size slider when custom font size is used Test timeout of 30000ms exceeded.
[Legacy Crypto] › settings/appearance-user-settings-tab.spec.ts:110:9 › Appearance user settings tab › should disable font size slider when custom font size is used: matrix-react-sdk/playwright/e2e/settings/appearance-user-settings-tab.spec.ts#L114
3) [Legacy Crypto] › settings/appearance-user-settings-tab.spec.ts:110:9 › Appearance user settings tab › should disable font size slider when custom font size is used Error: locator.click: Test timeout of 30000ms exceeded. Call log: - waiting for getByTestId('mx_FontScalingPanel').locator('label').filter({ hasText: 'Use custom size' }) 112 | 113 | const panel = page.getByTestId("mx_FontScalingPanel"); > 114 | await panel.locator("label", { hasText: "Use custom size" }).click(); | ^ 115 | 116 | // Assert that the font slider is disabled 117 | await expect(panel.locator(".mx_FontScalingPanel_fontSlider input[disabled]")).toBeVisible(); at /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/e2e/settings/appearance-user-settings-tab.spec.ts:114:70
[Legacy Crypto] › settings/appearance-user-settings-tab.spec.ts:110:9 › Appearance user settings tab › should disable font size slider when custom font size is used: matrix-react-sdk/playwright/e2e/settings/appearance-user-settings-tab.spec.ts#L1
3) [Legacy Crypto] › settings/appearance-user-settings-tab.spec.ts:110:9 › Appearance user settings tab › should disable font size slider when custom font size is used Retry #1 ─────────────────────────────────────────────────────────────────────────────────────── Test timeout of 30000ms exceeded.
[Legacy Crypto] › settings/appearance-user-settings-tab.spec.ts:110:9 › Appearance user settings tab › should disable font size slider when custom font size is used: matrix-react-sdk/playwright/e2e/settings/appearance-user-settings-tab.spec.ts#L114
3) [Legacy Crypto] › settings/appearance-user-settings-tab.spec.ts:110:9 › Appearance user settings tab › should disable font size slider when custom font size is used Retry #1 ─────────────────────────────────────────────────────────────────────────────────────── Error: locator.click: Test timeout of 30000ms exceeded. Call log: - waiting for getByTestId('mx_FontScalingPanel').locator('label').filter({ hasText: 'Use custom size' }) 112 | 113 | const panel = page.getByTestId("mx_FontScalingPanel"); > 114 | await panel.locator("label", { hasText: "Use custom size" }).click(); | ^ 115 | 116 | // Assert that the font slider is disabled 117 | await expect(panel.locator(".mx_FontScalingPanel_fontSlider input[disabled]")).toBeVisible(); at /home/runner/work/matrix-react-sdk/matrix-react-sdk/matrix-react-sdk/playwright/e2e/settings/appearance-user-settings-tab.spec.ts:114:70
🎭 Playwright Run Summary
6 failed [Legacy Crypto] › settings/appearance-user-settings-tab.spec.ts:25:9 › Appearance user settings tab › should be rendered properly [Legacy Crypto] › settings/appearance-user-settings-tab.spec.ts:76:9 › Appearance user settings tab › should support changing font size by clicking the font slider [Legacy Crypto] › settings/appearance-user-settings-tab.spec.ts:110:9 › Appearance user settings tab › should disable font size slider when custom font size is used [Rust Crypto] › settings/appearance-user-settings-tab.spec.ts:25:9 › Appearance user settings tab › should be rendered properly [Rust Crypto] › settings/appearance-user-settings-tab.spec.ts:76:9 › Appearance user settings tab › should support changing font size by clicking the font slider [Rust Crypto] › settings/appearance-user-settings-tab.spec.ts:110:9 › Appearance user settings tab › should disable font size slider when custom font size is used 6 flaky [Legacy Crypto] › composer/composer.spec.ts:283:17 › Composer › Rich text editor › when Control+Enter is required to send › only sends when you press Control+Enter [Legacy Crypto] › room/room.spec.ts:64:9 › Room Directory › should memorize the timeline position when switch Room A -> Room B -> Room A [Rust Crypto] › audio-player/audio-player.spec.ts:152:9 › Audio player › should be correctly rendered - high contrast theme [Rust Crypto] › audio-player/audio-player.spec.ts:168:9 › Audio player › should be correctly rendered - dark theme [Rust Crypto] › crypto/verification.spec.ts:245:9 › User verification › can receive a verification request when there is no existing DM [Rust Crypto] › read-receipts/redactions.spec.ts:691:17 › Read receipts › redactions › in threads › A thread with a redacted unread is still read after restart 123 skipped 643 passed (17.6m)
Prepare
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: Sibz/github-status-action@071b5370da85afbb16637d6eed8524a06bc2053e. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Run Tests 2/8
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: coactions/setup-xvfb@b6b4fcfb9f5a895edadc3bc76318fae0ac17c8b3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Run Tests 6/8
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: coactions/setup-xvfb@b6b4fcfb9f5a895edadc3bc76318fae0ac17c8b3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Run Tests 8/8
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: coactions/setup-xvfb@b6b4fcfb9f5a895edadc3bc76318fae0ac17c8b3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Run Tests 4/8
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: coactions/setup-xvfb@b6b4fcfb9f5a895edadc3bc76318fae0ac17c8b3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Run Tests 1/8
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: coactions/setup-xvfb@b6b4fcfb9f5a895edadc3bc76318fae0ac17c8b3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Run Tests 5/8
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: coactions/setup-xvfb@b6b4fcfb9f5a895edadc3bc76318fae0ac17c8b3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Run Tests 3/8
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: coactions/setup-xvfb@b6b4fcfb9f5a895edadc3bc76318fae0ac17c8b3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Run Tests 7/8
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: coactions/setup-xvfb@b6b4fcfb9f5a895edadc3bc76318fae0ac17c8b3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Report results
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: Sibz/github-status-action@071b5370da85afbb16637d6eed8524a06bc2053e, bobheadxi/deployments@88ce5600046c82542f8246ac287d0a53c461bca3, actions/github-script@v6, nwtgck/actions-netlify@7a92f00dde8c92a5a9e8385ec2919775f7647352. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Slow Test: [Rust Crypto] › read-receipts/redactions.spec.ts#L1
[Rust Crypto] › read-receipts/redactions.spec.ts took 6.4m
Slow Test: [Legacy Crypto] › read-receipts/redactions.spec.ts#L1
[Legacy Crypto] › read-receipts/redactions.spec.ts took 5.4m
Slow Test: [Rust Crypto] › read-receipts/new-messages.spec.ts#L1
[Rust Crypto] › read-receipts/new-messages.spec.ts took 3.8m
Slow Test: [Rust Crypto] › read-receipts/high-level.spec.ts#L1
[Rust Crypto] › read-receipts/high-level.spec.ts took 3.6m
Slow Test: [Legacy Crypto] › read-receipts/new-messages.spec.ts#L1
[Legacy Crypto] › read-receipts/new-messages.spec.ts took 3.6m

Artifacts

Produced during runtime
Name Size
all-blob-reports-1 Expired
3.59 MB
all-blob-reports-2 Expired
611 KB
all-blob-reports-3 Expired
16.3 MB
all-blob-reports-4 Expired
8.5 MB
all-blob-reports-5 Expired
6.94 MB
all-blob-reports-6 Expired
591 KB
all-blob-reports-7 Expired
15.9 MB
all-blob-reports-8 Expired
1.07 MB
html-report--attempt-1 Expired
47.9 MB