Allow user agents to use more permissive header validation in extensions

[rdcronin]

What is the issue with the Fetch Standard?

Background

All major browser vendors – including Google Chrome, Microsoft Edge, Mozilla Firefox, and Apple Safari, among others – have aligned on a shared extensions platform. The core functionality is shared in all these browsers, as are many individual APIs.

In general, the extensions platform tries to rely on the web platform for functionality when possible. This avoids introducing multiple APIs to accomplish the same thing. Extensions are meant to be "the web plus more", not an alternative platform to the web.

Extensions can also run on individual websites (e.g., via content scripts) and access cross-origin data from other sites via the fetch API if they have host access to those sites. This is a critical part of how extensions work and enables fundamental extension use cases.

Sometimes, a web API accomplishes almost what extensions need, but does not satisfy the entirety of a use case. In certain cases, we would like to relax the web specification to allow extensions to leverage these APIs in ways beyond what the open web can do, but keeping inline with both the core purpose of the API and the extensions platform.

Despite some prior art¹ where extension behavior has influenced W3C specifications, this is fairly new ground. Therefore, we are proposing a fairly small change which will not cover all cases where extensions diverge from the specification but will allow us to explore how best to approach this in the future.

Problem

The fetch API allows callers to set headers on the given request via the headers property on the RequestInit object. However, this explicitly forbids certain headers from being set; these are called forbidden header names. These headers are instead set by the browser, and often have security considerations.

One such header is the Origin header, which allows servers to restrict which data they allow in a cross-origin context. By design, the browser does not allow websites to modify this header, since that would allow websites to imitate a same-site request even if they were cross-site. However, plenty of tools allow users to set custom headers, and it is accepted that spoofing is often possible in more privileged contexts. This is merely how browsers indicate to a site which origin a request is coming from, and not a way servers should control any kind of sensitive data access without additional checks.

Extensions are tools acting on behalf of the user and need to be able to set this header, but the current options are heavy and have undesirable tradeoffs. For example, extensions implementations in all major browsers, by design, can modify the Origin header by listening for the request in the declarativeNetRequest or webRequest APIs and modifying the header there. Additionally, requests made from an extension origin to an origin to which the extension has access are already treated as same-site requests (and do not have the origin header set by the browser). This is not a security risk, as we limit this to extensions to which the extension has host permission (and could thus run scripts directly upon).

We would like to allow developers to set this header directly through the fetch API without introducing further divergence from the specified behavior. This would prevent them from needing to request access to other APIs which are not well scoped to the task and therefore significantly increase the permissions an extension asks for to meet a common use case.

Proposed Solution

We propose a modification to the validate a header algorithm to make it explicit that the User-Agent may skip step three in certain contexts (addition marked by asterisks):

To validate a header (name, value) for a Headers object headers:

1. If name is not a header name or value is not a header value, then throw a TypeError.
2. If headers’s guard is "immutable", then throw a TypeError.
3. If headers’s guard is "request" and (name, value) is a forbidden request-header, then return false. **The user agent may choose to skip this step and allow the header in certain contexts.**
4. If headers’s guard is "response" and name is a forbidden response-header name, then return false.
5. Return true.

For extensions, this would be a renderer at an extension origin, such as an extension page or service worker (not a content script), where that extension also has host access to the target origin. However, defining exactly when and how browsers should permit this would be beyond the scope of the proposed spec modification, which would only indicate the user agent could allow forbidden header modification in some cases.

Chrome, Firefox and Safari have indicated support of exploring this change in the WebExtensions Community Group: w3c/webextensions#786 (comment)

Additionally, if there is appetite, we could consider further changes to document the existing divergence from the spec to make requests behave as same origin if the extension has host permissions to the request and initiator origin.

Alternatives considered

1. Introduce a new API in the browser namespace, e.g. browser.fetch(): This would increase implementation complexity, even if we try to share logic, and there would be a risk of divergence as our respective API signatures change over time. Eventually, it may no longer be possible to reconcile the two.
2. Modify the behavior in extensions without spec changes: There have been times we've gotten push-back for these divergences from the spec, and explicitly calling out that this is a case that can happen seems like it only has advantages (makes the implementation match the spec, makes it easier for other browsers to align, makes it clear to other folks that there are exceptions when forbidden headers aren't forbidden, etc).

Proposal Authors

Google: @rdcronin, @oliverdunk, @domfarolino

Mozilla: @Rob--W, @zombie, @dotproto

Apple: @xeenon, @davidjohnson91, @kiaraarose

Microsoft: @mukul-p

Footnotes

1. The Event.isTrusted property has little use on the web, but was added for extensions, as noted in the Intent to Implement and Ship. In the Push API, User Agents generally require the [userVisibleOnly](https://developer.mozilla.org/en-US/docs/Web/API/PushSubscriptionOptions/userVisibleOnly) flag to be true but there is precedence for allowing extensions to set it to false. The CSP specification explicitly mentions extensions as a context that should not be impacted. ↩