data URLs: revised specification

[cvrebert]

As of SimonSapin/data-urls@82fe8eb , it no longer defines any algorithms.

[annevk]

I know. Not really sure what to do. Should URL define it inline? Are you interested in defining it?

[cvrebert]

Are you interested in defining it?

No, just noticed the problem when looking up Fetch-ing data: on a lark because of https://dev.opera.com/blog/opera-35/#fetch-api-data-and-blob-url-scheme-support

[cvrebert]

X-Ref: whatwg/url#68

[annevk]

I think we should define it in Fetch and leave the URL Standard to concern itself with a higher layer (namely the initial parse step).

https://simonsapin.github.io/data-urls/ has some good test angles.

https://gist.github.com/annevk/4287452653921b2b7de35e4208b4a985 has a basic setup for a web-platform-tests test suite that can be reused by non-browser implementations (especially if we split out the JSON format into its own resource at some point).

[annevk]

Bugs worth looking at:

• https://bugzilla.mozilla.org/show_bug.cgi?id=548516
• https://bugzilla.mozilla.org/show_bug.cgi?id=1388594
• https://bugzilla.mozilla.org/show_bug.cgi?id=1389251

[domenic]

I think we should define it in Fetch and leave the URL Standard to concern itself with a higher layer (namely the initial parse step).

I think I agree with this, but could you say more about the split?

[annevk]

First you run the URL parser over a data URL string to get a URL record. That URL record is the input to the data URL processor (or whatever we call it) which ends up returning a response (this is the algorithm Fetch will invoke from Scheme Fetch and also define someplace). Once we have that we could maybe also consider a Response.fromDataURL() or some such if we wanted to.

[annevk]

See whatwg/html#2912 for base64 handling.

[annevk]

Differences between browsers:

• Safari only accepts ";base64" at the end as per the RFC. Others also accept the part before the comma containing ";base64;". I'm inclined to go with others even though Safari is better.
• Chrome and Firefox drop unknown parameters. Others don't. (I haven't investigated deeply yet what is known and unknown, but on the surface it seems rather bogus as image/gif;charset=x is apparently fine.) Inclined to go with others.
• Chrome and Firefox lowercase MIME types and known parameter names (not parameter values). Others don't. Inclined to go with others.
• Chrome and Firefox remove U+0020 for non text/* MIME types (though note that application/xml does not count so it's more complicated and that %20 does end up as non-removed U+0020). This doesn't seem to be needed. See also one of the Mozilla bugs above for further analysis by me. Inclined to go with others.
• Edge and Firefox uses "text/plain" rather than "text/plain;charset=US-ASCII" as fallback for invalid MIME types. Safari often echoes the input string, doesn't even prepend a lone ";charset=x" with "text/plain". Inclined to go with Chrome (which uses the default also as the fallback).
• Chrome and Edge don't strip U+000C when base64 decoding. Chrome does for window.atob() so probably silly mismatch. Inclined to go with others.

[domenic]

First you run the URL parser over a data URL string to get a URL record. That URL record is the input to the data URL processor (or whatever we call it) which ends up returning a response (this is the algorithm Fetch will invoke from Scheme Fetch and also define someplace).

I was hoping there'd be some intermediate algorithm that takes a URL record (or perhaps just a path string) and returns you a tuple like (MIME type, byte sequence) (maybe with text encoding too? Or just MIME parameters in general?). This algorithm would then live in the URL Standard.

That feels like something that'd be easier to reuse across environments or as part of larger pipelines. And in general just give a good factoring of "data URL processing" separate from fetching.

WDYT?

[annevk]

That's more or less what I ended up with (the multiple return values I have now should probably be a tuple), but I'm not sure it should be in the URL Standard. That would also lead credence to the URL Standard having to deal with the mailto: URL scheme and other such things, whereas I don't think it should.

[domenic]

Hmmmmmmm. I think I am convinced by the slippery-slope argument. data: feels special intuitively, but reflecting logically for a while, I think it's not. The process of going from URL -> action and resource is fetching, so I guess it makes sense there. Thanks!

[annevk]

I filed https://bugzilla.mozilla.org/show_bug.cgi?id=1392241 as a tracking bug for Firefox. It also contains a summary of the state of this issue and the associated PR.

[annevk]

Additional bugs filed:

• https://bugs.webkit.org/show_bug.cgi?id=182325
• https://bugs.chromium.org/p/chromium/issues/detail?id=807562
• https://developer.microsoft.com/en-us/microsoft-edge/platform/issues/15718410/