Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test: add WPT user-timing tests #32123

Merged
merged 2 commits into from
Apr 21, 2024
Merged

test: add WPT user-timing tests #32123

merged 2 commits into from
Apr 21, 2024

Conversation

nolanlawson
Copy link
Contributor

@nolanlawson nolanlawson commented Apr 21, 2024 •
edited by mrobinson
Loading

As discussed in #32120 (comment), this adds the WPT user-timing tests to the default list of WPT tests, and also commits the current status of the user-timing tests.

  • ./mach build -d does not report any errors
  • ./mach test-tidy does not report any errors
  • There are tests for these changes

mrobinson
mrobinson requested changes Apr 21, 2024
View reviewed changes
Copy link
Member

@mrobinson mrobinson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! This looks great. One change necessary before landing: you'll also need to copy your results to tests/wpt/meta-legacy-layout because the same tests are run in both layout systems.

@mrobinson mrobinson added the T-linux-wpt-2020 Do a try run of the WPT label Apr 21, 2024
@github-actions github-actions bot removed the T-linux-wpt-2020 Do a try run of the WPT label Apr 21, 2024
@github-actions GitHub Actions
Copy link

🔨 Triggering try run (#8773463628) for Linux WPT

@mrobinson
Copy link
Member

I'll also send this to the CI to see if it agrees with your generated results; sometimes timing / font issues can throw things off.

@nolanlawson
Copy link
Contributor Author

Thanks for the review! Added to meta-legacy-layout.

@github-actions GitHub Actions
Copy link

Test results for linux-wpt-layout-2020 from try job (#8773463628):

Flaky unexpected result (13)
  • OK /css/css-fonts/variations/at-font-face-font-matching.html (#20684)
    • PASS [expected FAIL] subtest: Matching font-weight: '430' should prefer '340 398' over '501 550'
    • PASS [expected FAIL] subtest: Matching font-style: 'italic' should prefer 'normal' over 'oblique 0deg'
    • PASS [expected FAIL] subtest: Matching font-style: 'oblique 21deg' should prefer 'oblique -50deg -20deg' over 'oblique -40deg -30deg'
  • OK /css/css-values/cap-invalidation.html
    • FAIL [expected PASS] subtest: CSS Values and Units Test: cap invalidation

      uncaught exception: Error: assert_not_equals: expect the capital height of Ahem and sans-serif to be different got disallowed value 371.3333333333333

  • OK /html/browsers/browsing-the-web/navigating-across-documents/empty-iframe-load-event.html (#29066)
    • FAIL [expected PASS] subtest: Check execution order from nested timeout

      assert_equals: Expected nested setTimeout to run second expected true but got false

    • FAIL [expected PASS] subtest: Check execution order on load handler

      assert_equals: Expected onload to run first expected false but got true

  • OK /html/browsers/browsing-the-web/read-media/pageload-image-in-popup.html (#23849)
    • PASS [expected FAIL] subtest: The document for a standalone media file should have one child in the body.
  • TIMEOUT [expected OK] /html/browsers/history/the-history-interface/traverse-during-unload.html (#28688)
    • TIMEOUT [expected PASS] subtest: Traversing the history during unload

      Test timed out

  • TIMEOUT [expected OK] /html/browsers/windows/auxiliary-browsing-contexts/opener-setter.html (#22668)
    • TIMEOUT [expected PASS] subtest: Auxiliary browsing context created via window.openand settingwindow.openertonullshould reportwindow.opener null``

      Test timed out

  • CRASH [expected TIMEOUT] /html/semantics/embedded-content/the-iframe-element/iframe_sandbox_popups_nonescaping-2.html (#22154)
  • OK /html/webappapis/dynamic-markup-insertion/document-write/module-delayed.html (#27659)
    • FAIL [expected PASS] subtest: async document.write in a module

      assert_true: onload must be called expected true got false

  • TIMEOUT /html/webappapis/scripting/events/compile-event-handler-settings-objects.html (#24246)
    • PASS [expected FAIL] subtest: The entry settings object while executing the compiled callback via Web IDL's invoke must be that of the node document
  • TIMEOUT [expected OK] /html/webappapis/scripting/processing-model-2/integration-with-the-javascript-job-queue/promise-job-entry.html (#25805)
    • TIMEOUT [expected FAIL] subtest: Fulfillment handler on pending-then-fulfilled promise

      Test timed out

    • TIMEOUT [expected FAIL] subtest: Rejection handler on pending-then-rejected promise

      Test timed out

  • ERROR /resource-timing/content-type-parsing.html (#29131)
    • FAIL [expected TIMEOUT] subtest: mime-type 16 : text/html;charset=�gbk

      assert_equals: expected (string) "text/html" but got (undefined) undefined

    • TIMEOUT [expected NOTRUN] subtest: mime-type 17 : text/html;charset= gbk

      Test timed out

  • OK [expected TIMEOUT] /webmessaging/with-ports/018.html (#24485)
    • PASS [expected TIMEOUT] subtest: origin of the script that invoked the method, javascript:
  • TIMEOUT [expected OK] /workers/same-site-cookies/first-party.none.tentative.https.window.html
    • TIMEOUT [expected FAIL] subtest: Check SharedWorker sameSiteCookies option none for first-party

      Test timed out

Stable unexpected results that are known to be intermittent (13)
  • FAIL [expected PASS] /_mozilla/css/iframe/hide_and_show.html (#15265)
  • TIMEOUT [expected OK] /html/browsers/browsing-the-web/navigating-across-documents/javascript-url-referrer.window.html (#29081)
    • TIMEOUT [expected PASS] subtest: no-referrer referrer policy used to create the starting page

      Test timed out

  • OK /html/browsers/browsing-the-web/navigating-across-documents/navigation-unload-cross-origin.sub.window.html (#29056)
    • FAIL [expected PASS] subtest: Cross-origin navigation started from unload handler must be ignored

      promise_test: Unhandled rejection with value: object "SecurityError: The operation is insecure."

  • OK /html/browsers/browsing-the-web/navigating-across-documents/replace-before-load/a-click.html (#28697)
    • FAIL [expected PASS] subtest: aElement.click() before the load event must NOT replace

      assert_equals: expected "http://web-platform.test:8000/common/blank.html?thereplacement" but got "http://web-platform.test:8000/html/browsers/browsing-the-web/navigating-across-documents/replace-before-load/resources/code-injector.html?pipe=sub(none)&code=%0A%20%20%20%20const%20a%20%3D%20document.createElement(%22a%22)%3B%0A%20%20%20%20a.href%20%3D%20%22%2Fcommon%2Fblank.html%3Fthereplacement%22%3B%0A%20%20%20%20document.currentScript.before(a)%3B%0A%20%20%20%20a.click()%3B%0A%20%20"

  • OK [expected TIMEOUT] /html/infrastructure/urls/base-url/document-base-url-window-initiator-is-not-opener.https.window.html (#30970)
  • OK [expected TIMEOUT] /html/semantics/embedded-content/media-elements/track/track-element/no-cuechange-before-play.html (#31014)
    • FAIL [expected TIMEOUT] subtest: Ensure that the 'cuechange' event is not fired before video playback has begun.

      assert_true: Not expecting event, but got canplaythrough event expected true got false

  • OK [expected TIMEOUT] /html/semantics/embedded-content/the-iframe-element/iframe_sandbox_popups_escaping-1.html (#22647)
    • FAIL [expected TIMEOUT] subtest: Check that popups from a sandboxed iframe escape the sandbox if allow-popups-to-escape-sandbox is used

      assert_equals: It came from a sandboxed iframe expected "null" but got "http://web-platform.test:8000"

  • TIMEOUT [expected OK] /html/semantics/embedded-content/the-iframe-element/iframe_sandbox_popups_nonescaping-1.html (#24066)
    • NOTRUN [expected FAIL] subtest: Check that popups from a sandboxed iframe do not escape the sandbox
  • CRASH [expected TIMEOUT] /html/semantics/embedded-content/the-iframe-element/iframe_sandbox_popups_nonescaping-3.html (#24066)
  • OK /html/semantics/embedded-content/the-img-element/sizes/parse-a-sizes-attribute-width-1000px.html (#21666)
    • FAIL [expected PASS] subtest: <img srcset="/images/green-1x1.png?e38 50w, /images/green-16x16.png?e38 51w" sizes="(min-width:calc(0)) 1px"> ref sizes="1px" (width:1000px)

      assert_equals: expected "http://web-platform.test:8000/images/green-1x1.png" but got "http://web-platform.test:8000/images/green-16x16.png"

  • OK [expected TIMEOUT] /html/semantics/forms/form-submission-0/reparent-form-during-planned-navigation-task.html (#29724)
    • PASS [expected TIMEOUT] subtest: reparent-form-during-planned-navigation-task
  • OK /html/semantics/forms/form-submission-0/urlencoded2.window.html (#28687)
    • PASS [expected FAIL] subtest: application/x-www-form-urlencoded: Basic test (formdata event)
    • FAIL [expected PASS] subtest: application/x-www-form-urlencoded: 0x00 in name (normal form)

      assert_equals: expected "a%00b=c" but got ""

  • OK [expected TIMEOUT] /webmessaging/without-ports/017.html (#24486)
    • PASS [expected TIMEOUT] subtest: origin of the script that invoked the method, about:blank

@github-actions GitHub Actions
Copy link

✨ Try run (#8773463628) succeeded.

@nolanlawson nolanlawson mentioned this pull request Apr 21, 2024
Merged
5 tasks
@mrobinson mrobinson added this pull request to the merge queue Apr 21, 2024
Merged via the queue into servo:main with commit fa92b0f Apr 21, 2024
10 checks passed
@nolanlawson nolanlawson mentioned this pull request Apr 21, 2024
Merged
5 tasks
github-merge-queue bot pushed a commit that referenced this pull request Apr 21, 2024
@nolanlawson
docs: update README instructions for updating WPT (#32124)
025a987 
I noticed in #32123 that there are two issues with the docs for updating
Web Platform Tests:

- `./mach update-wpt --sync` fails with an error (`Are you sure you
don't want a patch?`) because it expects a `--patch` arg.
- The `tests/wpt/meta-legacy-layout/` test results need to be updated
with a separate command.

This updates `tests/wpt/README.md` with more complete instructions.

---
<!-- Thank you for contributing to Servo! Please replace each `[ ]` by
`[X]` when the step is complete, and replace `___` with appropriate
data: -->
- [x] `./mach build -d` does not report any errors
- [x] `./mach test-tidy` does not report any errors
- [ ] These changes fix #___ (GitHub issue number if applicable)

<!-- Either: -->
- [ ] There are tests for these changes OR
- [x] These changes do not require tests because this is a docs-only
change

<!-- Also, please make sure that "Allow edits from maintainers" checkbox
is checked, so that we can help you if you get stuck somewhere along the
way.-->

<!-- Pull requests that do not address these steps are welcome, but they
will require additional verification as part of the review process. -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mrobinson mrobinson mrobinson approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@nolanlawson @mrobinson