Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: Install Ubuntu dependencies for fake audio output #31070

Closed
wants to merge 1 commit into from

Conversation

mrobinson
Copy link
Member

Before running tests, try to install some dependencies in order to allow
fake audio output on the GitHub runner. This is a speculative fix for
flaky webaudio tests.

Fix found here: actions/runner-images#1114


  • ./mach build -d does not report any errors
  • ./mach test-tidy does not report any errors
  • These changes fix #___ (GitHub issue number if applicable)
  • These changes do not require tests because this is a CI change.

Before running tests, try to install some dependencies in order to allow
fake audio output on the GitHub runner. This is a speculative fix for
flaky webaudio tests.

Fix found here: actions/runner-images#1114
@sagudev sagudev added the T-full Do a full try run label Jan 12, 2024
@github-actions github-actions bot removed the T-full Do a full try run label Jan 12, 2024
Copy link

🔨 Triggering try run (#7501087211) with platforms=linux,macos,windows and layout=undefined

Copy link

Test results for linux-wpt-layout-2013 from try job (#7501087211):

Flaky unexpected result (14)
  • TIMEOUT [expected OK] /_webgl/conformance/glsl/misc/shader-with-non-reserved-words.html (#16216)
    • NOTRUN [expected PASS] subtest: Overall test
  • TIMEOUT [expected OK] /_webgl/conformance/uniforms/out-of-bounds-uniform-array-access.html (#26225)
    • NOTRUN [expected PASS] subtest: Overall test
  • OK /css/cssom-view/MediaQueryList-addListener-removeListener.html (#24569)
    • PASS [expected FAIL] subtest: listeners are called correct number of times
  • TIMEOUT /fetch/metadata/generated/css-images.sub.tentative.html (#29047)
    • TIMEOUT [expected PASS] subtest: background-image sec-fetch-site - HTTPS downgrade (header not sent) Test timed out
    • FAIL [expected PASS] subtest: border-image sec-fetch-site - HTTPS downgrade (header not sent) assert_unreached: Reached unreachable code
  • TIMEOUT [expected OK] /html/browsers/browsing-the-web/history-traversal/srcdoc/consecutive-srcdoc.html (#29084)
    • TIMEOUT [expected FAIL] subtest: changing srcdoc to about:srcdoc#yo then another srcdoc does two push navigations and we can navigate back Test timed out
  • OK /html/browsers/browsing-the-web/navigating-across-documents/empty-iframe-load-event.html (#29066)
    • PASS [expected FAIL] subtest: Check execution order on load handler
    • PASS [expected FAIL] subtest: Check execution order from nested timeout
  • OK /html/browsers/browsing-the-web/navigating-across-documents/initial-empty-document/load-pageshow-events-iframe-contentWindow.html (#28681)
    • PASS [expected FAIL] subtest: load & pageshow events do not fire on contentWindow of <iframe> element created with src='about:blank'
  • FAIL [expected CRASH] /html/canvas/element/manual/text/canvas.2d.disconnected.html (#30063)
  • TIMEOUT [expected OK] /html/semantics/embedded-content/the-iframe-element/iframe_sandbox_popups_escaping-3.html (#24057)
    • TIMEOUT [expected FAIL] subtest: Check that popups from a sandboxed iframe escape the sandbox if
      allow-popups-to-escape-sandbox is used Test timed out
  • TIMEOUT [expected OK] /html/semantics/embedded-content/the-iframe-element/iframe_sandbox_popups_nonescaping-1.html (#24066)
    • NOTRUN [expected FAIL] subtest: Check that popups from a sandboxed iframe do not escape the sandbox
  • OK [expected ERROR] /html/semantics/scripting-1/the-script-element/defer-script/async-script.html?reload (#29054)
  • TIMEOUT /html/webappapis/scripting/events/compile-event-handler-settings-objects.html (#24246)
    • PASS [expected FAIL] subtest: The entry settings object while executing the compiled callback via Web IDL's invoke must be that of the node document
  • TIMEOUT [expected OK] /referrer-policy/css-integration/svg/internal-stylesheet.html
    • TIMEOUT [expected FAIL] subtest: Styling SVG from internal styles marker-start Test timed out
    • NOTRUN [expected FAIL] subtest: Styling SVG from internal styles marker-mid
    • NOTRUN [expected FAIL] subtest: Styling SVG from internal styles marker-end
    • NOTRUN [expected FAIL] subtest: Styling SVG from internal styles mask
    • NOTRUN [expected FAIL] subtest: Styling SVG from internal styles mask-image
  • TIMEOUT [expected OK] /webstorage/localstorage-about-blank-3P-iframe-opens-3P-window.partitioned.tentative.html (#29053)
    • TIMEOUT [expected PASS] subtest: StorageKey: test 3P about:blank window opened from a 3P iframe Test timed out
Stable unexpected results that are known to be intermittent (21)
  • TIMEOUT [expected OK] /_webgl/conformance/glsl/misc/shader-uniform-packing-restrictions.html (#28103)
    • NOTRUN [expected PASS] subtest: Overall test
  • PASS [expected TIMEOUT] /css/css-fonts/font-size-adjust-reload.html (#30678)
  • OK /css/cssom-view/MediaQueryList-addListener-handleEvent.html (#24571)
    • PASS [expected FAIL] subtest: looks up handleEvent method on every event dispatch
  • OK [expected TIMEOUT] /dom/events/webkit-transition-end-event.html (#26497)
    • FAIL [expected TIMEOUT] subtest: webkitTransitionEnd event listener should trigger for an animation assert_true: received webkitTransitionEnd event expected true got false
    • PASS [expected NOTRUN] subtest: webkitTransitionEnd event listener should not trigger if an unprefixed listener also exists
    • PASS [expected NOTRUN] subtest: webkitTransitionEnd event listener should not trigger if an unprefixed event handler also exists
    • FAIL [expected NOTRUN] subtest: event types for prefixed and unprefixed transitionend event listeners should be named appropriately assert_equals: expected (string) "webkitTransitionEnd" but got (undefined) undefined
    • PASS [expected NOTRUN] subtest: webkitTransitionEnd event listener is case sensitive
  • OK /html/browsers/browsing-the-web/navigating-across-documents/initial-empty-document/load-pageshow-events-window-open.html (#28691)
    • FAIL [expected PASS] subtest: load event does not fire on window.open('about:blank') assert_unreached: load should not be fired Reached unreachable code
  • OK /html/browsers/browsing-the-web/navigating-across-documents/javascript-url-return-value-handling-dynamic.html (#28066)
    • PASS [expected FAIL] subtest: 0041 set in href="" targeting a frame and clicked
    • PASS [expected FAIL] subtest: 0080 00FF set in href="" targeting a frame and clicked
    • PASS [expected FAIL] subtest: 0080 00FF 0100 set in href="" targeting a frame and clicked
    • PASS [expected FAIL] subtest: D83D DE0D set in href="" targeting a frame and clicked
  • OK /html/browsers/history/the-history-interface/traverse_the_history_4.html (#21383)
    • FAIL [expected PASS] subtest: Multiple history traversals, last would be aborted assert_array_equals: Pages opened during history navigation expected property 1 to be 5 but got 3 (expected array [6, 5] got [6, 3])
  • OK /html/browsers/history/the-history-interface/traverse_the_history_5.html (#21383)
    • FAIL [expected PASS] subtest: Multiple history traversals, last would be aborted assert_array_equals: Pages opened during history navigation expected property 1 to be 5 but got 3 (expected array [6, 5] got [6, 3])
  • OK /html/browsers/the-window-object/open-close/creating_browsing_context_test_01.html (#29046)
  • CRASH [expected TIMEOUT] /html/semantics/embedded-content/the-iframe-element/iframe_sandbox_popups_escaping-1.html (#22647)
  • TIMEOUT [expected OK] /html/semantics/embedded-content/the-iframe-element/iframe_sandbox_popups_escaping-2.html (#22667)
    • TIMEOUT [expected FAIL] subtest: Check that popups from a sandboxed iframe escape the sandbox if
      allow-popups-to-escape-sandbox is used Test timed out
  • OK [expected TIMEOUT] /html/semantics/forms/form-submission-0/reparent-form-during-planned-navigation-task.html (#29724)
    • PASS [expected TIMEOUT] subtest: reparent-form-during-planned-navigation-task
  • OK /html/semantics/forms/form-submission-0/text-plain.window.html (#28687)
    • PASS [expected FAIL] subtest: text/plain: 0x00 in name (normal form)
    • FAIL [expected PASS] subtest: text/plain: single quote in name (normal form) assert_equals: expected "a'b=c\r\n" but got ""
  • OK /html/semantics/forms/form-submission-0/urlencoded2.window.html (#28687)
    • FAIL [expected PASS] subtest: application/x-www-form-urlencoded: Basic File test (formdata event) assert_equals: expected "basic=file-test.txt" but got ""
    • PASS [expected FAIL] subtest: application/x-www-form-urlencoded: \r\n in value (formdata event)
  • OK /resource-timing/status-codes-create-entry.html (#28675)
    • PASS [expected FAIL] subtest: Make sure all status codes are reported
  • TIMEOUT /resource-timing/test_resource_timing.html (#25720)
    • FAIL [expected PASS] subtest: PerformanceEntry has correct name, initiatorType, startTime, and duration (img) assert_equals: expected 12377344 but got 12377267
  • TIMEOUT /resource-timing/test_resource_timing.https.html (#25216)
    • FAIL [expected PASS] subtest: PerformanceEntry has correct name, initiatorType, startTime, and duration (img) assert_equals: expected 9856768 but got 9857042
  • OK /webaudio/the-audio-api/the-audiobuffersourcenode-interface/sub-sample-buffer-stitching.html (#22849)
    • FAIL [expected PASS] subtest: X Stitched sine-wave buffers at sample rate 43800 does not equal [0,0.06264832615852356,0.12505052983760834,0.18696144223213196,0.24813786149024963,0.308339387178421,0.36732959747314453,0.4248766601085663,0.480754554271698,0.5347436666488647,0.5866320133209229,0.6362156271934509,0.6832997798919678,0.7276994585990906,0.7692402601242065,0.8077589869499207...] with an element-wise tolerance of {"absoluteThreshold":0.0038986,"relativeThreshold":0}.
      Index Actual Expected AbsError RelError Test threshold
      [15073] 1.2480809571599472e-12 6.4605611562728882e-1 6.4605611562604071e-1 9.9999999999806810e-1 3.8985999999999999e-3
      [15074] 2.5936898589134216e-1 5.9696805477142334e-1 3.3759906888008118e-1 5.6552283858697683e-1 3.8985999999999999e-3
      Max AbsError of 6.4605611562604071e-1 at index of 15073.
      Max RelError of 9.9999999999806810e-1 at index of 15073.
      assert_true: expected true got false
    • FAIL [expected PASS] subtest: X SNR (46.132795778280126 dB) is not greater than or equal to 65.737. Got 46.132795778280126. assert_true: expected true got false
  • OK [expected TIMEOUT] /webmessaging/with-ports/018.html (#24485)
    • PASS [expected TIMEOUT] subtest: origin of the script that invoked the method, javascript:
  • TIMEOUT [expected OK] /webmessaging/without-ports/017.html (#24486)
    • TIMEOUT [expected PASS] subtest: origin of the script that invoked the method, about:blank Test timed out
  • OK [expected ERROR] /workers/baseurl/alpha/import-in-moduleworker.html (#21315)

Copy link

Test results for linux-wpt-layout-2020 from try job (#7501087211):

Flaky unexpected result (15)
  • CRASH [expected PASS] /_webgl/conformance/glsl/bugs/long-expressions-should-not-crash.html (#19221)
  • TIMEOUT [expected OK] /_webgl/conformance/glsl/misc/shader-with-non-reserved-words.html (#16216)
    • NOTRUN [expected PASS] subtest: Overall test
  • TIMEOUT [expected OK] /_webgl/conformance/uniforms/out-of-bounds-uniform-array-access.html (#26225)
    • NOTRUN [expected PASS] subtest: Overall test
  • TIMEOUT [expected PASS] /css/css-fonts/font-size-adjust-reload.html (#30678)
  • TIMEOUT /fetch/metadata/generated/element-img-environment-change.https.sub.html (#30111)
    • FAIL [expected TIMEOUT] subtest: sec-fetch-site - Cross-site, no attributes promise_test: Unhandled rejection with value: object "Error: Failed to query for recorded headers."
    • TIMEOUT [expected NOTRUN] subtest: sec-fetch-site - Same site, no attributes Test timed out
  • OK /html/browsers/browsing-the-web/navigating-across-documents/javascript-url-return-value-handling-dynamic.html (#28066)
    • FAIL [expected PASS] subtest: 0041 set in href="" targeting a frame and clicked assert_equals: expected "A" but got ""
    • FAIL [expected PASS] subtest: 0080 00FF set in href="" targeting a frame and clicked assert_equals: expected "�ÿ" but got ""
    • FAIL [expected PASS] subtest: 0080 00FF 0100 set in href="" targeting a frame and clicked assert_equals: expected "�ÿĀ" but got ""
  • OK /html/browsers/browsing-the-web/navigating-across-documents/navigation-unload-cross-origin.sub.window.html (#29056)
    • PASS [expected FAIL] subtest: Cross-origin navigation started from unload handler must be ignored
  • OK /html/browsers/history/the-history-interface/traverse_the_history_4.html (#21383)
    • PASS [expected FAIL] subtest: Multiple history traversals, last would be aborted
  • TIMEOUT [expected OK] /html/semantics/embedded-content/the-iframe-element/iframe_sandbox_popups_nonescaping-2.html (#22154)
    • NOTRUN [expected FAIL] subtest: Check that popups from a sandboxed iframe do not escape the sandbox
  • TIMEOUT [expected OK] /html/semantics/forms/form-submission-0/reparent-form-during-planned-navigation-task.html (#29724)
    • TIMEOUT [expected PASS] subtest: reparent-form-during-planned-navigation-task Test timed out
  • OK /html/semantics/forms/form-submission-0/urlencoded2.window.html (#28687)
    • PASS [expected FAIL] subtest: application/x-www-form-urlencoded: backslash in filename (formdata event)
  • OK /html/semantics/forms/historical.html (#28568)
    • PASS [expected FAIL] subtest: <input name=isindex> should not be supported
  • OK [expected ERROR] /html/semantics/scripting-1/the-script-element/defer-script/async-script.html?reload (#29054)
  • TIMEOUT [expected OK] /html/webappapis/scripting/processing-model-2/integration-with-the-javascript-job-queue/promise-job-entry.html (#25805)
    • TIMEOUT [expected FAIL] subtest: Fulfillment handler on pending-then-fulfilled promise Test timed out
    • TIMEOUT [expected FAIL] subtest: Rejection handler on pending-then-rejected promise Test timed out
  • OK [expected ERROR] /workers/baseurl/alpha/import-in-moduleworker.html (#21315)
Stable unexpected results that are known to be intermittent (21)
  • TIMEOUT [expected OK] /_webgl/conformance/glsl/misc/shader-uniform-packing-restrictions.html (#28103)
    • NOTRUN [expected PASS] subtest: Overall test
  • OK /css/css-fonts/variations/at-font-face-font-matching.html (#20684)
    • FAIL [expected PASS] subtest: Matching font-weight: '430' should prefer '420 440' over '450 460' assert_equals: Unexpected font on test element expected 487 but got 532
    • FAIL [expected PASS] subtest: Matching font-weight: '430' should prefer '350 399' over '340 398' assert_equals: Unexpected font on test element expected 487 but got 532
    • PASS [expected FAIL] subtest: Matching font-weight: '500' should prefer '500' over '450 460'
    • PASS [expected FAIL] subtest: Matching font-weight: '501' should prefer '502 510' over '503 520'
    • PASS [expected FAIL] subtest: Matching font-weight: '501' should prefer '503 520' over '500'
    • PASS [expected FAIL] subtest: Matching font-stretch: '100%' should prefer '100%' over '110% 120%'
    • FAIL [expected PASS] subtest: Matching font-stretch: '90%' should prefer '60% 70%' over '110% 140%' assert_equals: Unexpected font on test element expected 487 but got 532
    • FAIL [expected PASS] subtest: Matching font-style: 'italic' should prefer 'normal' over 'oblique 0deg' assert_equals: Unexpected font on test element expected 487 but got 532
    • PASS [expected FAIL] subtest: Matching font-style: 'oblique 20deg' should prefer 'oblique 20deg' over 'oblique 30deg 60deg'
    • FAIL [expected PASS] subtest: Matching font-style: 'oblique 20deg' should prefer 'oblique 40deg 50deg' over 'oblique 10deg' assert_equals: Unexpected font on test element expected 487 but got 532
    • And 11 more unexpected results...
  • FAIL [expected PASS] /css/css-transforms/transform-input-018.html (#22725)
  • OK [expected TIMEOUT] /dom/events/webkit-transition-end-event.html (#26497)
    • FAIL [expected TIMEOUT] subtest: webkitTransitionEnd event listener should trigger for an animation assert_true: received webkitTransitionEnd event expected true got false
    • PASS [expected NOTRUN] subtest: webkitTransitionEnd event listener should not trigger if an unprefixed listener also exists
    • PASS [expected NOTRUN] subtest: webkitTransitionEnd event listener should not trigger if an unprefixed event handler also exists
    • FAIL [expected NOTRUN] subtest: event types for prefixed and unprefixed transitionend event listeners should be named appropriately assert_equals: expected (string) "webkitTransitionEnd" but got (undefined) undefined
    • PASS [expected NOTRUN] subtest: webkitTransitionEnd event listener is case sensitive
  • TIMEOUT /fetch/metadata/generated/css-images.sub.tentative.html (#29047)
    • PASS [expected TIMEOUT] subtest: background-image sec-fetch-site - HTTPS downgrade (header not sent)
  • OK /html/browsers/browsing-the-web/navigating-across-documents/initial-empty-document/load-pageshow-events-iframe-contentWindow.html (#28681)
    • FAIL [expected PASS] subtest: load & pageshow events do not fire on contentWindow of <iframe> element created with src='about:blank' assert_unreached: load should not be fired Reached unreachable code
  • OK /html/browsers/browsing-the-web/navigating-across-documents/initial-empty-document/load-pageshow-events-window-open.html (#28691)
    • FAIL [expected PASS] subtest: load event does not fire on window.open('about:blank') assert_unreached: load should not be fired Reached unreachable code
  • TIMEOUT [expected OK] /html/browsers/browsing-the-web/navigating-across-documents/javascript-url-referrer.window.html (#29081)
    • TIMEOUT [expected PASS] subtest: no-referrer referrer policy used to create the starting page Test timed out
  • OK /html/browsers/browsing-the-web/navigating-across-documents/replace-before-load/a-click.html (#28697)
  • OK /html/browsers/history/the-history-interface/traverse_the_history_3.html (#21383)
    • FAIL [expected PASS] subtest: Multiple history traversals, last would be aborted assert_array_equals: Pages opened during history navigation expected property 1 to be 3 but got 2 (expected array [6, 3] got [6, 2])
  • OK /html/browsers/history/the-history-interface/traverse_the_history_5.html (#21383)
    • FAIL [expected PASS] subtest: Multiple history traversals, last would be aborted assert_array_equals: Pages opened during history navigation expected property 1 to be 5 but got 3 (expected array [6, 5] got [6, 3])
  • TIMEOUT [expected OK] /html/interaction/focus/the-autofocus-attribute/supported-elements.html (#24145)
    • TIMEOUT [expected FAIL] subtest: Contenteditable element should support autofocus Test timed out
    • NOTRUN [expected FAIL] subtest: Element with tabindex should support autofocus
    • NOTRUN [expected PASS] subtest: Non-HTMLElement should not support autofocus
    • NOTRUN [expected FAIL] subtest: Host element with delegatesFocus should support autofocus
    • NOTRUN [expected FAIL] subtest: Host element with delegatesFocus including no focusable descendants should be skipped
    • NOTRUN [expected FAIL] subtest: Area element should support autofocus
  • CRASH [expected OK] /html/semantics/embedded-content/the-iframe-element/iframe_sandbox_popups_escaping-1.html (#22647)
  • OK /html/semantics/links/links-created-by-a-and-area-elements/htmlanchorelement_noopener.html (#23205)
    • PASS [expected FAIL] subtest: Check that rel=noopener with target=_self does a normal load
  • OK /html/webappapis/dynamic-markup-insertion/document-write/module-tla-delayed.html (#29137)
    • PASS [expected FAIL] subtest: document.write in an imported module
  • TIMEOUT /resource-timing/nested-context-navigations-iframe.html (#24311)
    • PASS [expected TIMEOUT] subtest: Test that crossorigin iframe navigations are not observable by the parent, even after history navigations by the parent
    • PASS [expected NOTRUN] subtest: Test that cross-site iframe navigations are not observable by the parent, even after history navigations by the parent
    • PASS [expected NOTRUN] subtest: Test that iframe navigations are not observable by the parent
    • PASS [expected NOTRUN] subtest: Test that crossorigin iframe navigations are not observable by the parent
    • PASS [expected NOTRUN] subtest: Test that cross-site iframe navigations are not observable by the parent
    • TIMEOUT [expected NOTRUN] subtest: Test that iframe refreshes are not observable by the parent Test timed out
  • OK /resource-timing/status-codes-create-entry.html (#28675)
    • PASS [expected FAIL] subtest: Make sure all status codes are reported
  • TIMEOUT /resource-timing/test_resource_timing.html (#25720)
    • FAIL [expected PASS] subtest: PerformanceEntry has correct name, initiatorType, startTime, and duration (img) assert_equals: expected 6012160 but got 6012192
  • TIMEOUT /resource-timing/test_resource_timing.https.html (#25216)
    • FAIL [expected PASS] subtest: PerformanceEntry has correct name, initiatorType, startTime, and duration (img) assert_equals: expected 16583936 but got 16584032
  • OK /webaudio/the-audio-api/the-audiobuffersourcenode-interface/sub-sample-buffer-stitching.html (#22849)
    • FAIL [expected PASS] subtest: X Stitched sine-wave buffers at sample rate 43800 does not equal [0,0.06264832615852356,0.12505052983760834,0.18696144223213196,0.24813786149024963,0.308339387178421,0.36732959747314453,0.4248766601085663,0.480754554271698,0.5347436666488647,0.5866320133209229,0.6362156271934509,0.6832997798919678,0.7276994585990906,0.7692402601242065,0.8077589869499207...] with an element-wise tolerance of {"absoluteThreshold":0.0038986,"relativeThreshold":0}.
      Index Actual Expected AbsError RelError Test threshold
      [15073] 2.5229717067366662e+19 6.4605611562728882e-1 2.5229717067366662e+19 3.9051897284293706e+19 3.8985999999999999e-3
      [15074] 2.5936898589134216e-1 5.9696805477142334e-1 3.3759906888008118e-1 5.6552283858697683e-1 3.8985999999999999e-3
      Max AbsError of 2.5229717067366662e+19 at index of 15073.
      Max RelError of 3.9051897284293706e+19 at index of 15073.
      assert_true: expected true got false
    • FAIL [expected PASS] subtest: X SNR (-344.6041616610199 dB) is not greater than or equal to 65.737. Got -344.6041616610199. assert_true: expected true got false
  • OK [expected TIMEOUT] /webmessaging/without-ports/018.html (#24485)
    • PASS [expected TIMEOUT] subtest: origin of the script that invoked the method, javascript:

Copy link

✨ Try run (#7501087211) succeeded.

@mrobinson
Copy link
Member Author

I'm not sure the try run will be very illuminating, because this changes the GitHub Actions configuration. I can trigger a full try though.

@mrobinson
Copy link
Member Author

@sagudev
Copy link
Member

sagudev commented Jan 13, 2024

Something might still not be working: https://github.com/servo/servo/actions/runs/7503685481/job/20431139722#step:6:406

@mrobinson
Copy link
Member Author

Okay. I will abandon this then.

@mrobinson mrobinson closed this Jan 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants