Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for testing the WebExtensions API in WPT #219

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

kiaraarose
Copy link

No description provided.

This RFC documents our proposal for adding support for testing the WebExtensions API in
the web platform tests.
Comment on lines +25 to +28
Because these tests won’t leverage `testharness.js` directly, we’ve introduced a new
`testharness.js`, `.extension.js`, that will create the necessary boilerplate to
convert the `browser.test` assertions into the corresponding assertions in the test
harness.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably care to some extent about how easy/difficult it is to write tests without using extension.js.

Rather than putting all the code necessary to map from browser.test to testharness.js in the server-generated wrapper, it would probably be better to put it into a separate file — which then gets included (along with testharness and testdriver) by the server-generated wrapper?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have actually tried to implement the proposal for executorsafari.py in web-platform-tests/wpt#50648 (comment) and I think it would be easier to integrate the tests without adding extension.js and using regular testharness.js tests. I think something like this would work:

// META: script=/resources/testdriver.js?feature=bidi
// META: script=/resources/testdriver-vendor.js
// META: script=/web-extension/resources/webextension-test.js

webextension_test(async () => {
    // Call for illustrative purposes, in practice, webextension_test
    // defined in webextension-test.js can implement it. 
    await test_driver.bidi.webExtension.install();
}, test_desc);

A problem with the current approach is that the infrastructure does not detect the ?feature=bidi inside the server-generated wrapper.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here is a runnable implementation of this web-platform-tests/wpt#51090

@jgraham
Copy link
Contributor

jgraham commented Mar 3, 2025

Can we get an update here to detail the current proposal? At the moment what's in the RFC doesn't seem to match what's actually in the patch.

My understanding is that the proposal is for a new test wrapper indicated by .extension.js in the filename, a new testdriver API, and a function exposed by the wrapper that implements a protocol for transferring test results back from the extension? In particular the latter protocol needs to be specified somewhere (I assume it's what browser.test does, but unless that has a written specification somewhere we need enough detail in the RFC that one can understand what the requirements are without reading Chromium-specific documents).


There are two potential concerns with this implementation:

1. We have no precedent for tests run via a Classic command in some user agents
Copy link
Contributor

@OrKoN OrKoN Mar 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have concerns that we will end up with two methods for dealing with extensions exposed to test authors following different (and separately evolving) specs that are not fully implemented by all vendors. I think it would be preferable to start with adding methods to testdriver.js that match the WebDriver BiDi specification and allow synchronous handling of these methods for non-WebDriver-BiDi implementations (prototype: web-platform-tests/wpt#51090).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants