Implement JS fetch API

[jdm]

This issue is tracking the implementation of the JS fetch API following the project plan. This is assigned to our RGSoC team, so please talk with me before starting to work on any part of it.

[fitzgen]

cc myself

[jdm]

Things to be aware of when starting this project:

• I expect the vast majority of code changes to take place in components/script/dom/ - this is the code that's responsible for implementing the APIs that websites make use of, as well as exposing them to the JS environment
• in particular, each new interface that needs to be implemented (Request, Response, and Headers) will need to follow the steps for creating a new interface
• there are already tests for the missing interfaces, which can be run via ./mach test-wpt tests/wpt/web-platform-tests/fetch/api/request/ (similarly for fetch/api/response/ and fetch/api/headers/) - you can learn more about the testing infrastructure in our in-tree documentation and the testharness documentation
• we will need to modify include.ini to start running the tests automatically on our automated testing machines
• the majority of files in components/script/dom/ correspond to similarly-named interfaces defined in WebIDL files in components/script/dom/webidls/ - when confused about something, look for examples of other interfaces and how they relate to the Rust implementation in the corresponding source files
• our documentation attempts to explain the complexities of exposing Rust code via JavaScript APIs; it is entirely likely that large parts of those explanations will be confusing as you're starting out, but keep in mind that documentation exists and please make suggestions about ways it could be improved :)
• when implementing new interfaces, it is recommended to comment out everything and get a working servo build, then work on uncommenting one attribute or method member of the interface at a time. If at any time some piece of WebIDL syntax causes a python exception to occur while compiling, feel free to comment it out and ask whether it's necessary - there are a number of pieces of the WebIDL specification that Servo does not support properly yet.

[jeenalee]

cc'ing myself

[jeenalee]

NOTE: This summary is outdated. The web platform tests have been updated.

So far, the Headers, Request, and Response APIs have been mostly implemented. Some web platform tests for these APIs fail:

• fetch/api/headers:
  • FAIL: 15 subtests
    • 13 are related to missing OpenEndedDictionary (#13144)
    • 2 are blocked by external (#13004)
  • TIMEOUT: 1 test
    • fetch/api/headers/headers-idl.html
• fetch/api/request:
  • FAIL: 11 subtests
    • 2 are related to missing OpenEndedDictionary (#13144)
    • 1 is directly related to Promise (#4282)
    • 6 are related to missing Body implementation which is blocked by Promise (#4282)
      • 1 subtest: how Body is initialized
      • 2 subtests: missing text()
      • 2 subtests: how Body is used to initialize Request
      • 1 subtest: missing Body methods
    • 1 is related to referrer not being properly parsed, i.e. about:client becomes client. RequestInfo (which is used as an input for constructing Request) will have a referrer Client when about:client is given. This results in Request with a referrer client when it should be about:client. Could this be an error in how input for request is created?
    • 1 is related to referrer and referrer policy not being properly saved in request like above
  • TIMEOUT: 4 tests
    • fetch/api/request/request-init-003.sub.html
    • fetch/api/request/request-clone.sub.html
    • fetch/api/request/request-idl.html
    • fetch/api/request/request-disturbed.html
• fetch/api/response:
  • FAIL: 13 subtests
    • 1 is related to missing OpenEndedDictionary (#13144)
    • 12 are related to missing Body implementation which is blocked by Promise (#4282)
      • 1 subtest: how Body is initialized
      • 2 subtests: missing text()
      • 5 subtests: missing blob()
      • 2 subtests: missing stream/ReadableStream in Body
      • 2 subtests: how null response's body is returned as undefined.
  • TIMEOUT: 1 test
    • fetch/api/request/response-idl.html

[jeenalee]

Updated summary of failing tests

• fetch/api/basic/
  • not summarized yet!
• fetch/api/cors/
  • currently disabled
  • intermittently fails
• fetch/api/credentials/
  • authorization should be managed properly
    • authentication-basic-worker.html
    • authentication-basic.html
  • cookies should be managed properly
    • cookies-worker.html
    • cookies.html
• fetch/api/headers/
  • only headers-idl.html isn't passing
• fetch/api/policies/
  • tests related to csp block
    • csp-blocked.html
    • csp-blocked-worker.html
  • when fetching request's referrer is empty, it should be noted in fetched response's headers
    • referrer-no-referrer-worker.html
    • referrer-no-referrer.html
  • fetching request's origin should be noted in fetched response's headers
    • referrer-origin-when-cross-origin-worker.html
    • referrer-origin-when-cross-origin.html
    • referrer-origin-worker.html
    • referrer-origin.html
  • Request's referrer should be the full url of current document/worker
    • referrer-unsafe-url.html
    • referrer-unsafe-url-worker.html
• fetch/api/redirect/
  • currently disabled
  • intermittently fails
• fetch/api/request/
  • clone() should return a request that is identical to the original request
    • request-clone.sub.html
  • request-cache.html is disabled (#13458)
  • body mixin methods should be implemented
    • request-consume-empty.html
    • request-consume.html
  • if a request is disturbed, various things should/should not happen
    • request-disturbed.html
  • request header should be created with various objects (JS Object, array, headers) and get information from the body if none is given
    • request-headers.html
    • request-error.html
  • referrer should be properly parsed, i.e. currently, about:client becomes client. RequestInfo (which is used as an input for constructing Request) will have a referrer Client when about:client is given. This results in Request with a referrer client when it should be about:client. Could this be an error in how input for request is created?
    • request-init-001.sub.html
  • request's body should be initialized in various ways
    • request-init-002.html
  • request should be initialized correctly
    • request-init-003.sub.html
• fetch/api/response/
  • static method error() should return correct response
    • response-static-error.html
  • Stream/ReadableStream should be supported
    • response-cancel-stream.html
    • response-consume-stream.html
  • after response's body is locked, promise should reject
    • response-stream-disturbed-*.html
  • response's body should be initialized in various ways
    • response-init-002.html
  • default Body should be given in response
    • response-init-001.html
  • body mixin methods should be implemented
    • response-consume-empty.html
    • response-consume.html
  • clone() should return a response that is identical to the original response
    • response-clone.html

[jdm]

I think filing specific issues for anything that isn't working will be more useful at this point.