CSP not working for certain hostnames

[enexusde]

Greetings,

the documentation of the content-security-policy (CSP) might contain an imprecision. Affecting the edibility of a hostname that have a dot as the last character. This will violate the CSP and prevent the browser from requesting from the URL.

The documentation for the host-part (see https://www.w3.org./TR/CSP/#grammardef-host-part ) currently specified by this ABNF:

host-part = "" / [ "." ] 1*host-char ( "." 1host-char )

Unfortunately, sometimes a link to a external web-resource may end having a single dot. Those host-values are violating the host-parts of this ABNF.

Here are some examples for other valid hostnames:

• https://www.w3.org./TR/CSP/#grammardef-host-part
• https://en.wikipedia.org./wiki/SOLID
• https://www.rfc-editor.org./info/rfc7168
• https://dom.spec.whatwg.org./#document

Those websites do allow a dot at the end. Those websites currently are served correctly for a few reasons:

1. They do not use CSP.
2. They do not link external resources.
3. They do only link relative URIs.
4. They only link URLs matching the 'self' directive.

Other websites do NOT allow a dot at the end:

• http://microsoft.com./ (currently show 404)

Some websites do a redirect to the non-dot form:

• https://www.google.com./search?q=hello (will redirect to https://www.google.com/search?q=hello)

There might be no difference for the domain/DNS, since it is the same domain and the same DNS.
It is a difference for the server (throu the referrer/origin/host-request-header) and its specified values in the content-security-policy though.

Please do not keep this specification as-it-is for a long peroid of time! This might etablishing tactical misuses of this ABNF. Serving links only by dot-ending-hostnames (whitelisting throu 'self' directive only) might be misused as some kind of content-protection for CDNs.

I respectfuly request either:

1. A host-part ABNF allowing a dot at the end.

• or -

2. A explaination why a dot is not allowed.

I am aware that this change might affect the implementation of CSP in browsers.

Thank you for your attention!

Kind regards

Peter Rader

Fachinformatiker AE / IT Software Developer
Peter Rader

[mikewest]

Ah, trailing dots.

We probably should fix CSP's grammar to account for this, and change implementations to match.

That said, that patch wouldn't actually address the underlying issue, which is that the DNS layer shouldn't be leaking into the web-facing layer. I'm more or less on board with @ericlaw1979's (perhaps tounge-in-cheek) that we should stop supporting trailing .s altogether. We likely can't do that quickly given enterprise and DNS search domains, but it would be an improvement over our current behavior.

since it is the same domain and the same DNS

This isn't true, at least, not universally. Chromium-based browsers treat example.com and example.com. as distinct origins. Whether this is reasonable or not is up for debate.

[annevk]

They should definitely be distinct origins. You could try some kind of normalization on top of that (perhaps if you're HSTS or have a public suffix or some such), but we shouldn't change the origin (or site) boundary for this. Way too dangerous.

And yeah, we should fix the CSP grammar and implementations and probably keep this issue scoped to that. 😊

[mikewest]

They should definitely be distinct origins. You could try some kind of normalization on top of that (perhaps if you're HSTS or have a public suffix or some such), but we shouldn't change the origin (or site) boundary for this. Way too dangerous.

Agreed. I recall differences between browser behavior for example.com. and example.com cookies in the past, but I haven't looked at any of this for years. It would be ideal if browsers have aligned on this more than I thought!

And yeah, we should fix the CSP grammar and implementations and probably keep this issue scoped to that. 😊

Certainly. I'm just noting that the impact on CSP is a symptom of an underlying problem.

[bathos]

There’s some prior discussion about this in WHATWG URL that may be of interest.

Chromium-based browsers treat example.com and example.com. as distinct origins.

I think that matches currently specified behavior for WHATWG URL + HTML for origin equality. It looks like RFC 6454 (“Web Origin Concept”) considered them distinct, too — but only in the sense that it didn’t consider “https://example.com.” to be an origin in the first place. It only defines the concept when the host is composed of LDH labels after IDNA normalization —

“[...] assumes the uri-host will contain only LDH labels”

— and IDNA in turn mentions em outright:

The complete name convention using a trailing dot described in RFC 1123, which can be explicit as in "www.example.com."
or implicit as in "www.example.com", is not considered in this specification.

It’s funny that everything’s like “so, about those dots: not our problem” lol.

There’s also this note in URL re: (context-specific?) equality semantics:

[enexusde]

Suggestion:

host-part   = "*" / [ "*." ] 1*host-char *( "." 1*host-char ) [ "." ]

[annevk]

Looking at https://w3c.github.io/webappsec-csp/#host-part-match I think we need something more complicated. At least *. should probably not host-part match all hosts ending with a dot. I guess we should have a length check on pattern being at least 3 code points.

[bathos]

At least *. should probably not host-part match all hosts ending with a dot.

That’s a good point, though AFAICT @enexusde’s suggested host-part grammar wouldn’t accept the string *. right? Nor ..

---

Not sure if other folks would also consider it best to reduce “backtracking” (so to speak) in ABNF*, but just in case, I’d toss out the option of splitting it in two + swapping which side of 1*host-char gets a dot.

host-part = "*" [ "." host-rest ] / host-rest
host-rest = 1*host-char *("." 1*host-char) [ "." ]

(This doesn’t accept *. either. Assuming I actually got it right, anyway. Well, I’m sure y’all got the idea even if I messed it up...)

[Edit1: yeah, I messed it up, since that wouldn’t accept e.g. localhost and the current definition does. Eh!]
[Edit 2: okay, that’s probably what I meant.]

* (It’s a trade off I guess — reducing the amount that multiple alternatives can eat the same prefixes of remaining input makes it easier to tell what’s “going on” / detect actual ambiguities, but probably also tends to leave you with more nonterminals that aren’t directly aligned w/ semantically significant units, hm ... [just thinking out loud here, nm])

[annevk]

I think that's right. I forgot CSP contains statements such as

If expression matches the host-source grammar

though whether anyone implements it that way we better test. Generally whenever there's an ABNF grammar there's a lot of implementation inconsistencies in my experience.

[enexusde]

TWIMC AFAICT a octal representation of the IP might find its way into the host-header-field for ancient browsers. ( http://0177.0000.0000.0001/ = http://127.0.0.1/ ) serving different webresources accordingly.

[enexusde]

Any progress on that? I need a solution yesterday/real quick!

[mikewest]

@SaeidEid has expressed interest in poking at this in Chromium (with @antosart's help). I hope they'll hammer out the spec change as well.

(@annevk, @mozfreddyb: This feels more like a bug fix than something I'd ask them to run through Blink's intent process. Would y'all agree with that from your respective perspectives, or would you like a standards position request?)

[annevk]

Tests and a change to CSP should suffice.

[mozfreddyb]

+1 to that. Adding test cases to wpt and a pull request to the spec should suffice. Are you filing implementation bugs?

[mikewest]

Thanks. I think Saeid can file implementation bugs once the prereqs are in.

[enexusde]

@SaeidEid welcome at google.

[antosart]

I was just looking at this, and I fear there will be some extra work to implement a WPT since it seems that the WPT runner does not currently support any hosts with a dot at the end. Supporting it will likely require changes in the WPT CI, as well as in the various vendors.

It is surprising that hosts with a trailing dot are not tested anywhere in WPT. But I guess we have to start somewhere.

[SaeidEid]

I created a bug for this issue: https://bugs.chromium.org/p/chromium/issues/detail?id=1512075

[mikewest]

See web-platform-tests/wpt#16147.

That said, https://wpt.fyi/results/fetch/metadata/trailing-dot.https.sub.any.html?label=experimental&label=master&aligned works under Edge and Safari, so it's likely worth writing a test even if it doesn't run correctly under Chromium/Gecko (or, alternatively, exploring exactly why it passes under WebKit?).