Handling of data URLs

[hiroshige-g]

Hi,
I encountered some questions while I'm implementing Fetch API + data scheme in Chromium/Blink.

Question 1: Are XHRs to data URLs intentionally prohibited, or just XHR's spec lacks same-origin data-URL flag setting or so?

(I expect the latter because I thought previously we could use XHRs to data URLs)

Fetch API + data URLs: fetch('data://...') is resolved for all modes because same-origin data-URL flag is set in Request() constructor.
However, all XHRs to 'data://...' are rejected according to the spec [https://xhr.spec.whatwg.org/], because same-origin data-URL flag is not set (and the default is "unset") and mode is CORS or CORS-with-forced-preflight.

Question 2: What is the intention of unsetting same-origin data-URL flag on redirect? Are redirects from HTTP(S) to data URLs intentionally allowed in no-cors mode?

Example: fetch('http://example.com/A') where the response from 'http://example.com/A' returns a 'Location: data://...' header.
In such cases, on redirect, same-origin data-URL flag is unset and thus fetch is rejected, except for when mode is "no-cors".
In "no-cors" mode, the case of <request's mode is "no-cors"> is applied and opaque response is returned.

Is "same-origin data-URL flag" introduced to forbid redirects to data URLs in general?
If so, such redirects should be rejected also in "no-cors" mode.

Related Chromium bugs:
Redirects to data URLs are intentionally forbidden in Chromium
(so perhaps redirects to data URLs in Fetch API will be also rejected):
https://code.google.com/p/chromium/issues/detail?id=64092
https://code.google.com/p/chromium/issues/detail?id=272072
Implementing Fetch API + data URLs:
https://code.google.com/p/chromium/issues/detail?id=521475

[annevk]

Thank you. To answer your questions. 1) XMLHttpRequest should indeed set that flag. 2) All redirects should unset the same-origin data-URL flag. What makes you think it is only unset for certain modes?

[annevk]

How do you want to appear in acknowledgments section by the way? As Hiroshige Hayashizaki (I'm assuming that's you) or also your name in kanji?

[hiroshige-g]

Thanks!

2. All redirects should unset the same-origin data-URL flag. What makes you think it is only unset for certain modes?

Ah, my statement was ambiguous; I meant:

• the same-origin data-URL flag is unset in all redirects for all modes,
• but fetch is resolved in "no-cors" mode (and rejected in other modes),
  because the processing falls through into the fifth bullet of Step 8 of main fetch:
  "=> request's mode is "no-cors""
  after the condition
  "=> request's current url's scheme is "data" and request's same-origin data-URL flag is set"
  was tested and not met.

acknowledgments

"Hiroshige Hayashizaki" is fine. Thanks!

[annevk]

Okay. So if mode is "no-cors" you can end up with an opaque response that is a data URL. The idea behind this was e.g., <img> setting this flag and displaying the data URL either way, but it was wrapped in an opaque response you would not be able to paint it on <canvas> and then export the <canvas> data.

Similar with fetch(). You can read a data URL normally, but if it's the result of a redirect you only have an opaque response which you can perhaps give to something that can display it down the road, but you won't be able to do much else with it.

[annevk]

@hiroshige-g I think I can close this, correct?

[hiroshige-g]

Yes, thanks for fixing the spec for 1) and clarification for 2)!

(However, for 2), I expect redirects to data URLs in no-cors mode will be disabled in Chromium, at least in the initial support for data URLs, because redirects to data URLs are intentionally forbidden in Chromium as I mentioned above.)

[annevk]

@hiroshige-g you mean that <img src=redirect> and <iframe src=redirect> where redirect redirects to a data URL will end up with a network error? If so, we might want to specify that, but when I tested that didn't seem to be the case.

[hiroshige-g]

@annevk
My test case of <img src=redirect> failed (i.e. image was not shown) on Chrome: https://jsfiddle.net/r4q07x2h/2/
The redirect is accepted (a green box is shown) on Firefox 40.0.2,
and rejected on Chrome 46.0.2471.2. (Tested on Windows)

Could you let me know the test case you used? That would be quite helpful for us.

[annevk]

I don't remember, it seems like did not test properly. It also seems <iframe> rejects with a network error. I think we should just specify that, since we did want to align in Gecko at some point.

[annevk]

https://bugzilla.mozilla.org/show_bug.cgi?id=1018872 has additional context for the mess around data URLs.

[annevk]

@hiroshige-g, @bzbarsky objected to changing the handling of data URLs. He would favor that we retain the old handling where you can redirect to a data URL, but the result would be an opaque response. Both cases special case the data URLs in the handling of redirects, but arguably marking them opaque (and therefore failing "cors") is a bit more principled and consistent.

What do you think? Reopening for now.

[bzbarsky]

Why does this require special casing of data: URLs?

It seems to me like the principled thing is that data: can be loaded anywhere you load things, but defaults to CORS-cross-origin unless opted in otherwise by the caller. That should make the opaque response here automatic, no?

And of course a bunch of the platform needs to opt in to data: being considered CORS-same-origin.

[annevk]

When we last discussed this we decided that a redirect should make a data URL response opaque.

[bzbarsky]

Right, that's the "default" data behavior.

[annevk]

No, even in the case of this opt-in flag being set, since the person using this API might not expect it to come from a redirect. (E.g., an open redirect or a hostile cross-origin URL.)

[bzbarsky]

The opt-in flag is on a per-request basis. So it would have to be set by the redirect code to get set here.

[annevk]

Ah, there is the misunderstanding. The request you pass to fetch is kept around. That's why redirects will have to unset the flag.

[bzbarsky]

We might be defining the term "request" differently. I'm talking about whatever object there are two of ("pre-redirect" and "post-redirect") when an HTTP redirect happens. That's the object that the flag should live on.

[bzbarsky]

That is, this flag is not a property of a fetch wholesale but rather a property of a single actual handling of a URL, whatever you want to call that thing.

[annevk]

There's no such thing conceptually. But unsetting the flag during a redirect has the same effect and was what was in the specification before.

[annevk]

@hiroshige-g @mikewest would you be okay with me changing this back in Fetch to match what @bzbarsky wants? Either way it's a special case from the point of view of Fetch' algorithms, but allowing redirects to data URLs would not break http://software.hixie.ch/utilities/cgi/data/data for instance, which seems nice.

[hiroshige-g]

It is fine with me that the spec allows redirects to data URLs in no-cors mode while Chromium reject them per Chromium-specific policy.
Or, do you want Chromium to allow such redirects to be spec-conformant at this point?

[annevk]

I would want Chromium to not have Chromium-specific policies, ideally.

[tyoshino]

We're basically honoring abarth's comment in https://crbug.com/64092. It doesn't prevent us from revisiting it but it requires consulting our security team.

[annevk]

It seems like abarth's concern is primarily making sure you end up with the right origin ("security context"). That is handled pretty well for navigations in the specification so I think we should just allow this and hope Chromium relaxes this constraint at some point. I don't see it helping much to turn this into a network error.

[annevk]

Done, the specification now allows redirects to data URLs again. Chrome will have to figure out how Firefox managed to do that securely, I suppose. Per the specification it should be secure.

[youennf]

Some related tests in web-platform-tests/wpt#3644

[youennf]

As I read the spec now, redirections to data URLs are triggering a network error, as per step 4 of section 5.4, which would align with Chrome behavior. Where was it discussed?

[annevk]

See #309, but note that HTML's navigate has its own redirect logic that for now still allows data URLs. This is only for redirects from other endpoints, such as <img> and fetch(), for which I believe all browsers already agreed.

[youennf]

I don't think WebKit is matching this.
Basically, whenever mode is cors or same-origin, redirections to data URLs will fail (XHR, img/script with cors). Whenever mode is no-cors (img, script for instance), redirections to data URLs will succeed.

Doing a quick check, firefox seems to load redirected data URLs for img/script, treating them as same origin I believe.

[annevk]

Is it just data URLs or are there other URL schemes you also allow redirects to from these contexts? I suppose I can file a new issue on this. Note that treating them as same-origin is potentially leaky (although not if you also treat them as same-origin inside <iframe>).

[youennf]

data URLs is the only case I encountered. I would restrict the issue to that scheme.
Would probably be good to do further checks for completeness.