Skip to content

chore: introduce chromium-tip-of-tree-headless-shell#33964

Merged
mxschmitt merged 2 commits intomicrosoft:mainfrom
mxschmitt:introduce-chromium-tip-of-tree-headless-shell
Dec 12, 2024
Merged

chore: introduce chromium-tip-of-tree-headless-shell#33964
mxschmitt merged 2 commits intomicrosoft:mainfrom
mxschmitt:introduce-chromium-tip-of-tree-headless-shell

Conversation

@mxschmitt
Copy link
Contributor

@mxschmitt mxschmitt commented Dec 12, 2024

Motivation: This helps us to find out about upstream headless_shell regressions before they appear in the Beta channel.


I was considering doing the same pattern like we do for chromium (auto use headless-shell) but this doesn't work here since chromium-tip-of-tree is already a channel. Lets keep it how it is and do the manual opt-in.

@mxschmitt mxschmitt added the CQ1 label Dec 12, 2024
@mxschmitt mxschmitt force-pushed the introduce-chromium-tip-of-tree-headless-shell branch from 4ff4cdd to 6ceac06 Compare December 12, 2024 01:23
@mxschmitt mxschmitt added CQ1 and removed CQ1 labels Dec 12, 2024
@github-actions

This comment has been minimized.

headlessShellBrowser.revision = revision;
headlessShellBrowser.browserVersion = browserVersion;
} else if (browserName === 'chromium-tip-of-tree') {
const tipOfTreeBrowser = await browsersJSON.browsers.find(b => b.name === 'chromium-tip-of-tree-headless-shell');
Copy link
Member

@yury-s yury-s Dec 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why can't we just use version of chromium-tip-of-tree, we don't expect these two to ever diverge, do we?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll follow-up.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@mxschmitt mxschmitt added CQ1 and removed CQ1 labels Dec 12, 2024
@github-actions

This comment has been minimized.

@mxschmitt mxschmitt merged commit 8d57b75 into microsoft:main Dec 12, 2024
@mxschmitt mxschmitt deleted the introduce-chromium-tip-of-tree-headless-shell branch December 12, 2024 02:11
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions
Copy link
Contributor

Test results for "tests 1"

10 failed
❌ [installation tests] › npx-global.spec.ts:34:5 › npx playwright install global @package-installations-macos-latest
❌ [installation tests] › playwright-cdn.spec.ts:43:7 › playwright cdn failover should work (https://playwright.azureedge.net/dbazure/download/playwright) @package-installations-macos-latest
❌ [installation tests] › playwright-cdn.spec.ts:43:7 › playwright cdn failover should work (https://playwright.download.prss.microsoft.com/dbazure/download/playwright) @package-installations-macos-latest
❌ [installation tests] › playwright-cdn.spec.ts:43:7 › playwright cdn failover should work (https://playwright.azureedge.net) @package-installations-macos-latest
❌ [installation tests] › playwright-cdn.spec.ts:43:7 › playwright cdn failover should work (https://playwright-akamai.azureedge.net) @package-installations-macos-latest
❌ [installation tests] › playwright-cdn.spec.ts:43:7 › playwright cdn failover should work (https://playwright-verizon.azureedge.net) @package-installations-macos-latest
❌ [installation tests] › playwright-cli-install-should-work.spec.ts:21:5 › install command should work @package-installations-macos-latest
❌ [installation tests] › playwright-packages-install-behavior.spec.ts:66:5 › playwright should work @package-installations-macos-latest
❌ [installation tests] › playwright-packages-install-behavior.spec.ts:88:5 › playwright should work with chromium --only-shell @package-installations-macos-latest
❌ [installation tests] › playwright-packages-install-behavior.spec.ts:97:5 › @playwright/test should work @package-installations-macos-latest

7 flaky ⚠️ [chromium-page] › page/page-event-request.spec.ts:138:3 › should report navigation requests and responses handled by service worker with routing @chromium-ubuntu-22.04-node22
⚠️ [installation tests] › playwright-electron-should-work.spec.ts:21:5 › electron should work @package-installations-macos-latest
⚠️ [installation tests] › playwright-test-package-managers.spec.ts:54:5 › npm: uninstalling ct removes playwright bin @package-installations-macos-latest
⚠️ [chromium-library] › library/inspector/cli-codegen-2.spec.ts:25:7 › cli codegen › should contain open page @ubuntu-20.04-chromium-tip-of-tree
⚠️ [playwright-test] › ui-mode-test-ct.spec.ts:252:5 › should watch component when editing util @ubuntu-latest-node18-1
⚠️ [playwright-test] › ui-mode-test-ct.spec.ts:59:5 › should run component tests after editing test @ubuntu-latest-node20-1
⚠️ [playwright-test] › ui-mode-test-watch.spec.ts:145:5 › should watch all @windows-latest-node18-1

37129 passed, 646 skipped
✔️✔️✔️

Merge workflow run.

@github-actions
Copy link
Contributor

Test results for "tests others"

1 flaky ⚠️ [electron-api] › electron/electron-app.spec.ts:55:5 › should fire close event when the app quits itself @electron-macos-latest

21293 passed, 509 skipped
✔️✔️✔️

Merge workflow run.

@github-actions
Copy link
Contributor

Test results for "tests 2"

1 fatal errors, not part of any test
10 failed
❌ [chromium-library] › library/signals.spec.ts:25:5 › should close the browser when the node process closes @chrome-beta-macos-latest
❌ [chromium-library] › library/inspector/cli-codegen-2.spec.ts:456:7 › cli codegen › should --save-trace @msedge-beta-macos-latest
❌ [chromium-library] › library/inspector/cli-codegen-2.spec.ts:466:7 › cli codegen › should save assets via SIGINT @msedge-beta-macos-latest
❌ [chromium-library] › library/inspector/cli-codegen-2.spec.ts:456:7 › cli codegen › should --save-trace @msedge-dev-macos-latest
❌ [chromium-library] › library/inspector/cli-codegen-2.spec.ts:456:7 › cli codegen › should --save-trace @msedge-macos-latest
❌ [chromium-library] › library/defaultbrowsercontext-1.spec.ts:42:3 › context.addCookies() should work @msedge-ubuntu-20.04
❌ [webkit-library] › library/trace-viewer.spec.ts:1576:1 › should show a popover @webkit-macos-13-large
❌ [webkit-library] › library/trace-viewer.spec.ts:1576:1 › should show a popover @webkit-macos-13-xlarge
❌ [webkit-library] › library/capabilities.spec.ts:94:3 › should play audio @smoke @webkit-macos-15-large
❌ [webkit-library] › library/tracing.spec.ts:432:14 › should produce screencast frames scale @webkit-macos-15-xlarge

93 flaky ⚠️ [chromium-library] › library/beforeunload.spec.ts:20:3 › should close browser with beforeunload page @channel-chromium-macos-latest
⚠️ [chromium-library] › library/browser.spec.ts:54:5 › should dispatch page.on(close) upon browser.close and reject evaluate @channel-chromium-macos-latest
⚠️ [chromium-library] › library/fetch-proxy.spec.ts:21:3 › context request should pick up proxy credentials @channel-chromium-macos-latest
⚠️ [chromium-library] › library/video.spec.ts:381:5 › screencast › should capture navigation @channel-chromium-macos-latest
⚠️ [chromium-library] › library/browser.spec.ts:54:5 › should dispatch page.on(close) upon browser.close and reject evaluate @chrome-beta-macos-latest
⚠️ [chromium-library] › library/browsercontext-base-url.spec.ts:37:3 › should construct a new URL when a baseURL in browserType.launchPersistentContext is passed to page.goto @chrome-beta-macos-latest
⚠️ [chromium-library] › library/browsercontext-proxy.spec.ts:27:3 › should work when passing the proxy only on the context level @chrome-beta-macos-latest
⚠️ [chromium-library] › library/browsertype-connect.spec.ts:838:7 › run-server › socks proxy › should proxy local.playwright requests @chrome-beta-macos-latest
⚠️ [chromium-library] › library/inspector/cli-codegen-2.spec.ts:456:7 › cli codegen › should --save-trace @chrome-beta-macos-latest
⚠️ [chromium-library] › library/inspector/cli-codegen-csharp.spec.ts:171:5 › should work with --save-har @chrome-beta-macos-latest
⚠️ [chromium-library] › library/inspector/cli-codegen-test.spec.ts:88:5 › should not generate recordHAR with --save-har @chrome-beta-macos-latest
⚠️ [chromium-library] › library/inspector/pause.spec.ts:23:3 › should resume when closing inspector @chrome-beta-macos-latest
⚠️ [chromium-library] › library/logger.spec.ts:19:3 › should log @smoke @chrome-beta-macos-latest
⚠️ [chromium-library] › library/browsercontext-base-url.spec.ts:37:3 › should construct a new URL when a baseURL in browserType.launchPersistentContext is passed to page.goto @chrome-macos-latest
⚠️ [chromium-library] › library/browsercontext-proxy.spec.ts:27:3 › should work when passing the proxy only on the context level @chrome-macos-latest
⚠️ [chromium-library] › library/browsertype-connect.spec.ts:853:7 › run-server › socks proxy › should lead to the error page for forwarded requests when the connection is refused @chrome-macos-latest
⚠️ [chromium-library] › library/fetch-proxy.spec.ts:21:3 › context request should pick up proxy credentials @chrome-macos-latest
⚠️ [chromium-library] › library/inspector/cli-codegen-csharp.spec.ts:171:5 › should work with --save-har @chrome-macos-latest
⚠️ [chromium-library] › library/video.spec.ts:189:5 › screencast › should capture static page @chrome-macos-latest
⚠️ [chromium-library] › library/beforeunload.spec.ts:20:3 › should close browser with beforeunload page @chrome-windows-latest
⚠️ [chromium-library] › library/video.spec.ts:581:5 › screencast › should capture static page in persistent context @smoke @chrome-windows-latest
⚠️ [chromium-library] › library/inspector/pause.spec.ts:522:5 › pause › should record from debugger @chromium-headed-ubuntu-24.04
⚠️ [chromium-library] › library/trace-viewer.spec.ts:1474:1 › should not leak recorders @chromium-headed-ubuntu-24.04
⚠️ [chromium-page] › page/page-mouse.spec.ts:244:3 › should tween mouse movement @chromium-headed-ubuntu-24.04
⚠️ [chromium-library] › library/inspector/cli-codegen-3.spec.ts:61:7 › cli codegen › should click locator.nth @chromium-headed-windows-latest
⚠️ [chromium-library] › library/popup.spec.ts:264:3 › should not throw when click closes popup @chromium-tip-of-tree-headless-shell-ubuntu-20.04
⚠️ [chromium-library] › library/browsercontext-reuse.spec.ts:206:1 › should reset mouse position @chromium-tip-of-tree-ubuntu-20.04--headed
⚠️ [chromium-library] › library/inspector/cli-codegen-aria.spec.ts:77:7 › should update aria snapshot highlight @chromium-tip-of-tree-ubuntu-20.04--headed
⚠️ [chromium-library] › library/inspector/cli-codegen-1.spec.ts:24:7 › cli codegen › should click @chromium-tip-of-tree-windows-latest
⚠️ [chromium-library] › library/trace-viewer.spec.ts:1474:1 › should not leak recorders @chromium-tip-of-tree-windows-latest
⚠️ [chromium-library] › library/video.spec.ts:381:5 › screencast › should capture navigation @chromium-tip-of-tree-windows-latest
⚠️ [chromium-library] › library/inspector/cli-codegen-3.spec.ts:171:7 › cli codegen › should generate frame locators (4) @chromium-tip-of-tree-windows-latest--headed
⚠️ [chromium-library] › library/video.spec.ts:381:5 › screencast › should capture navigation @chromium-tip-of-tree-windows-latest--headed
⚠️ [firefox-library] › library/capabilities.spec.ts:252:3 › requestFullscreen @firefox-headed-macos-14-xlarge
⚠️ [firefox-library] › library/video.spec.ts:381:5 › screencast › should capture navigation @firefox-headed-macos-14-xlarge
⚠️ [firefox-library] › library/browsercontext-basic.spec.ts:34:3 › should be able to click across browser contexts @firefox-headed-ubuntu-24.04
⚠️ [firefox-page] › page/page-set-input-files.spec.ts:39:3 › should upload a folder @firefox-headed-ubuntu-24.04
⚠️ [firefox-library] › library/browsercontext-locale.spec.ts:118:3 › should be isolated between contexts @firefox-headed-windows-latest
⚠️ [firefox-library] › library/headful.spec.ts:213:3 › Page.bringToFront should work @firefox-headed-windows-latest
⚠️ [firefox-library] › library/hit-target.spec.ts:398:3 › should click in iframe with padding @firefox-headed-windows-latest
⚠️ [firefox-library] › library/inspector/cli-codegen-1.spec.ts:828:7 › cli codegen › should record slider @firefox-headed-windows-latest
⚠️ [firefox-library] › library/video.spec.ts:381:5 › screencast › should capture navigation @firefox-headed-windows-latest
⚠️ [firefox-page] › page/page-click-scroll.spec.ts💯3 › should scroll into view element in iframe @firefox-headed-windows-latest
⚠️ [firefox-library] › library/client-certificates.spec.ts:705:3 › browser › support http2 if the browser only supports http1.1 @firefox-ubuntu-20.04
⚠️ [chromium-library] › library/browser.spec.ts:54:5 › should dispatch page.on(close) upon browser.close and reject evaluate @msedge-beta-macos-latest
⚠️ [chromium-library] › library/browsercontext-base-url.spec.ts:37:3 › should construct a new URL when a baseURL in browserType.launchPersistentContext is passed to page.goto @msedge-beta-macos-latest
⚠️ [chromium-library] › library/browsercontext-reuse.spec.ts:30:1 › should re-add binding after reset @msedge-beta-macos-latest
⚠️ [chromium-library] › library/chromium/oopif.spec.ts:235:3 › should report google.com frame with headed @msedge-beta-macos-latest
⚠️ [chromium-library] › library/browsercontext-reuse.spec.ts:30:1 › should re-add binding after reset @msedge-beta-ubuntu-20.04
⚠️ [chromium-library] › library/browsercontext-reuse.spec.ts:30:1 › should re-add binding after reset @msedge-beta-windows-latest
⚠️ [chromium-library] › library/inspector/cli-codegen-aria.spec.ts:77:7 › should update aria snapshot highlight @msedge-beta-windows-latest
⚠️ [chromium-library] › library/beforeunload.spec.ts:20:3 › should close browser with beforeunload page @msedge-dev-macos-latest
⚠️ [chromium-library] › library/browsercontext-base-url.spec.ts:37:3 › should construct a new URL when a baseURL in browserType.launchPersistentContext is passed to page.goto @msedge-dev-macos-latest
⚠️ [chromium-library] › library/browsercontext-proxy.spec.ts:27:3 › should work when passing the proxy only on the context level @msedge-dev-macos-latest
⚠️ [chromium-library] › library/browsercontext-reuse.spec.ts:30:1 › should re-add binding after reset @msedge-dev-macos-latest
⚠️ [chromium-library] › library/browsertype-launch-server.spec.ts:43:5 › launch server › should work with wsPath @msedge-dev-macos-latest
⚠️ [chromium-library] › library/chromium/connect-over-cdp.spec.ts:89:5 › should connectOverCDP and manage downloads in default context @msedge-dev-macos-latest
⚠️ [chromium-library] › library/video.spec.ts:381:5 › screencast › should capture navigation @msedge-dev-macos-latest
⚠️ [chromium-library] › library/browsercontext-reuse.spec.ts:30:1 › should re-add binding after reset @msedge-dev-ubuntu-20.04
⚠️ [chromium-library] › library/browsercontext-reuse.spec.ts:30:1 › should re-add binding after reset @msedge-dev-windows-latest
⚠️ [chromium-library] › library/inspector/cli-codegen-3.spec.ts:171:7 › cli codegen › should generate frame locators (4) @msedge-dev-windows-latest
⚠️ [chromium-library] › library/browsercontext-base-url.spec.ts:37:3 › should construct a new URL when a baseURL in browserType.launchPersistentContext is passed to page.goto @msedge-macos-latest
⚠️ [chromium-library] › library/browsercontext-fetch.spec.ts:1229:3 › should work with connectOverCDP @msedge-macos-latest
⚠️ [chromium-library] › library/browsercontext-proxy.spec.ts:27:3 › should work when passing the proxy only on the context level @msedge-macos-latest
⚠️ [chromium-library] › library/chromium/chromium.spec.ts:151:15 › should close service worker together with the context @msedge-macos-latest
⚠️ [chromium-library] › library/inspector/cli-codegen-2.spec.ts:466:7 › cli codegen › should save assets via SIGINT @msedge-macos-latest
⚠️ [chromium-library] › library/inspector/cli-codegen-csharp.spec.ts:171:5 › should work with --save-har @msedge-macos-latest
⚠️ [chromium-library] › library/logger.spec.ts:19:3 › should log @smoke @msedge-macos-latest
⚠️ [chromium-library] › library/proxy.spec.ts:93:11 › should proxy local network requests › with other bypasses › localhost @msedge-macos-latest
⚠️ [chromium-library] › library/browsercontext-reuse.spec.ts:30:1 › should re-add binding after reset @msedge-ubuntu-20.04
⚠️ [chromium-library] › library/video.spec.ts:381:5 › screencast › should capture navigation @msedge-ubuntu-20.04
⚠️ [chromium-library] › library/browsercontext-reuse.spec.ts:30:1 › should re-add binding after reset @msedge-windows-latest
⚠️ [webkit-library] › library/screenshot.spec.ts:206:14 › element screenshot › element screenshot should work with a mobile viewport @tracing-webkit
⚠️ [webkit-library] › library/inspector/cli-codegen-1.spec.ts:611:7 › cli codegen › should select @webkit-headed-macos-14-xlarge
⚠️ [webkit-library] › library/inspector/cli-codegen-1.spec.ts:365:7 › cli codegen › should fill [contentEditable] @webkit-headed-ubuntu-20.04
⚠️ [webkit-library] › library/inspector/cli-codegen-2.spec.ts:249:7 › cli codegen › should handle dialogs @webkit-headed-ubuntu-20.04
⚠️ [webkit-library] › library/trace-viewer.spec.ts:911:1 › should show action source @webkit-headed-ubuntu-20.04
⚠️ [webkit-library] › library/inspector/cli-codegen-pick-locator.spec.ts:36:7 › should update locator highlight @webkit-headed-ubuntu-22.04
⚠️ [webkit-library] › library/inspector/cli-codegen-2.spec.ts:483:7 › cli codegen › should fill tricky characters @webkit-headed-ubuntu-24.04
⚠️ [webkit-library] › library/inspector/cli-codegen-1.spec.ts:930:7 › cli codegen › should clear when recording is disabled @webkit-headed-windows-latest
⚠️ [webkit-library] › library/inspector/cli-codegen-2.spec.ts:249:7 › cli codegen › should handle dialogs @webkit-headed-windows-latest
⚠️ [webkit-library] › library/inspector/cli-codegen-2.spec.ts:313:7 › cli codegen › should record open in a new tab with url @webkit-headed-windows-latest
⚠️ [webkit-library] › library/inspector/cli-codegen-3.spec.ts:26:7 › cli codegen › should click locator.first @webkit-headed-windows-latest
⚠️ [webkit-page] › page/page-request-fallback.spec.ts:239:5 › post data › should amend binary post data @webkit-macos-13-large
⚠️ [webkit-library] › library/tracing.spec.ts:432:14 › should produce screencast frames fit @webkit-macos-13-xlarge
⚠️ [webkit-library] › library/tracing.spec.ts:432:14 › should produce screencast frames fit @webkit-macos-14-xlarge
⚠️ [webkit-library] › library/tracing.spec.ts:432:14 › should produce screencast frames crop @webkit-macos-14-xlarge
⚠️ [webkit-page] › page/wheel.spec.ts:101:3 › should dispatch wheel events after popup was opened @smoke @webkit-macos-15-large
⚠️ [webkit-library] › library/tracing.spec.ts:432:14 › should produce screencast frames fit @webkit-macos-15-xlarge
⚠️ [webkit-library] › library/tracing.spec.ts:432:14 › should produce screencast frames crop @webkit-macos-15-xlarge
⚠️ [webkit-library] › library/browsercontext-viewport-mobile.spec.ts:97:5 › mobile viewport › should fire orientationchange event @webkit-ubuntu-20.04
⚠️ [webkit-library] › library/screenshot.spec.ts:66:14 › page screenshot › should work with a mobile viewport and fullPage @webkit-ubuntu-20.04
⚠️ [webkit-library] › library/browsercontext-reuse.spec.ts:50:1 › should reset serviceworker @webkit-windows-latest

262979 passed, 10099 skipped
✔️✔️✔️

Merge workflow run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants