Support for custom headers for handshake

[Misiu]

Hi,

please consider adding ability to add custom headers for handshake.
In RFC6455 there one interesting point:

The request MAY include any other header fields, for example,
cookies [RFC6265] and/or authentication-related header fields
such as the |Authorization| header field [RFC2616], which are
processed according to documents that define them.

I've found an example how to add custom header to handshake: https://blog.heckel.xyz/2014/10/30/http-basic-auth-for-websocket-connections-with-undertow/ but this is for Java and unfortunately isn't possible in HTML5.

When searching over the net I found many places question about this option, for example:
sta/websocket-sharp#22
https://stackoverflow.com/questions/4361173/http-headers-in-websockets-client-api/4361358#4361358
aspnet/SignalR#888

For example in Python this is possible https://stackoverflow.com/questions/15381414/sending-custom-headers-in-websocket-handshake. Other languages also support this. Last place missing is the browser.

Please consider adding this into specification. Having this even as a draft would allow us to consider browser vendors to add support for it.

If this is incorrect place for adding request about specification please forgive me and please point me to right place.

[domenic]

We'd need implementer interest in this before moving forward, similar to whatwg/html#2177.

[Misiu]

Mozilla - https://support.mozilla.org/en-US/questions/1176810
Chromium - https://bugs.chromium.org/p/chromium/issues/detail?id=768328
EDGE - https://wpdev.uservoice.com/forums/257854-microsoft-edge-developer/suggestions/31600090-websocket-support-for-custom-headers-for-handsha

@domenic I'll add feature request for other browsers soon. Hopefully some of them will show some interest.

[tyoshino]

Thank you Misiu for filing the feature request and the bugs for each vendor.

Regarding complexity, for the web platform, this would require non-trivial amount of work at both spec and impl side.

To introduce custom header feature, we need to make it issue a CORS preflight. Though this is not an HTTP API, considering the original reason why we introduced the CORS preflight, we need it for WebSocket + custom headers.

On Chromium, we have separate stacks for WebSocket and XHR/fetch() (for which we have CORS logic). Some non-trivial refactoring and gluing is needed.

[ricea]

I am one of the authors and maintainers of Chrome's WebSocket implementation. I oppose this proposal.

Reasons:

• The WebSocket handshake security model hinges on exposing no more capabilities for request forgery than are already possible using an img tag. It is not possible to add arbitrary headers to the request using an img tag.
• Apart from the limited sub-protocol negotiation, the WebSocket handshake is not intended as a means to exchange metadata about the connection. Non-browser implementations may use it this way, but I would not recommend it. It is better to think of it as a low-level operation like a TCP/IP SYN packet.
• It would be possible to add this to the WebSocket protocol by using a CORS preflight. However, the utility provided simply doesn't justify the additional implementation, standardisation and maintenance cost.

[Misiu]

@tyoshino @ricea thank You guys for Your reply.
My main idea was to add ability to add Authentication header to WebSockets.

If You look at SignalR repo You will notice that many people are looking for a way to pass authentication via WebSocket - aspnet/SignalR#888 (comment)

Right now the only option to do this is to add token as part of query string.
To simplify this only Authorization header is required.

[ricea]

@Misiu In the specific case of WebSockets I think passing an authentication token in the URL is okay. The reason is that, unlike HTTP URLs, wss: URLs are never exposed to the user. They can't bookmark them or copy-and-paste them. This minimises the risk of accidental sharing. In addition, their appearance in other web APIs is minimal [1]. For example, they won't appear in history. This reduces the risk of leakage via JS APIs.

The best thing from a security perspective would probably be to perform authentication after the WebSocket is connected, but I realise it is undesirable from a resource-usage perspective to permit unauthorised connections to be established in the first place.

Given the portability difficulties with using cookies or http auth, a short-lived authentication token in the URL is probably your best option at the moment.

A specific note with respect to the Authorization header is that normally you'd need to be able to handle a 401 response to use it. But in the WebSocket API error responses are never exposed to the page for security reasons. So that wouldn't work very well.

[1] I actually can't think of any other APIs that expose ws: or wss: URLs at all. Certainly they don't appear in resource timing.

[digitalpacman]

I cannot use WebSocketSharp because it's missing the ability to customize authorization headers. I am trying to integrate with a WebSocket api that takes in OAuth2 bearer tokens.

[digitalpacman]

@ricea That's not the main reason why you don't put authentication values in URLs. It's because of server logs. Auth tokens should be treated with the same security concern as username and password combos. You don't put username and password combinations in urls, so neither should auth tokens. If you have a vulnerability and a user downloads your http logs, they then have access to virtually all of your accounts.

[digitalpacman]

@ricea sadly implementers don't get to make the decision of what is required to use an api. I am currently trying to use an api that REQUIRES oauth2 bearer tokens in the authorization header for server-to-server communication. They offer cookies for browser based security, but I do not have that luxury because I have no sessionids for the user. So all you're doing by saying "we should not do this" is saying "you have to look elsewhere to implement with this api". Which would simply just drive people away from using this implementation. You aren't "saving" anyone but not allowing this, just alienating users.

[maleta]

I encountered this issue looking for solution on my problem - how to remove sid parameter from URL that is set and used by websockets. Now I see @ricea suggestion of adding short living token to URL with explanation that "wss: URLs are never exposed to the user."

OWASP Zap report is suggesting sid param removal from URL, also this question on stackoverflow is refering that CloudFlare is also suggesting sid param removal from URL: https://stackoverflow.com/questions/42759556/how-to-remove-socket-io-sid-parameter-from-url

So my question is, @ricea, do you think that this reported issue is completely irrelevant because of fact that ws/wss urls are never exposed to users?

[ricea]

@maleta

So my question is, @ricea, do you think that this reported issue is completely irrelevant because of fact that ws/wss urls are never exposed to users?

Security measures need to be considered in terms of what threats they are intended to protect against and to what extent they are effective in mitigating those threats.

In the case of the Cloudflare vulnerability, placing the sid parameter in a header rather than in the URL would have provided no protection whatsoever.

Quoting https://bugs.chromium.org/p/project-zero/issues/detail?id=1139:

... we observed encryption keys, cookies, passwords, chunks of POST data and even HTTPS requests for other major cloudflare-hosted sites from other users.

This is the trouble with checklist security. If you removed the sid parameter from the URL, you could check that item off the list and feel that you'd improved your security. But in practice there's no improvement at all.

@digitalpacman If someone has access to your server logs, they can do much worse things than just steal credentials. There's usually all kinds of personally-identifiable information in there. Access needs to be tightly restricted just as with any other kind of user data.

[digitalpacman]

@ricea there is only personally-identifiable information in there if you aren't doing good practices. At most there should be an IP address, but you can opt to not record that either. Not putting SECURITY CREDENTIALS in the url is an extremely popular best practice. Infact, governing bodies for security will instantly fail you if you are passing them in the URL.

Please look at the OAuth2 spec for example and explanation of these vulnerabilities. There is a reason it REQUIRES them in the POST body.

[annevk]

@ricea is this perhaps something we can reconsider if https://wicg.github.io/origin-policy/ becomes a thing? That should nullify most of the CORS preflight cost.

The reason I'm somewhat sympathetic to the requests is because I learned that middleboxes basically ruin full duplex HTTP so WebSocket will likely stick around.

[ricea]

@annevk I think origin policy will probably be implemented in the browser in the same place as CORS, so it would still mean re-wiring the handshake to go down the same code path.

The WebSocket handshake started out as purely a mechanism to negotiate a WebSocket connection. Over time we've added HTTP features like cookies and authentication, but we've paid a high price in implementation complexity.

The "right" way to do WebSocket authentication is to do it at the application layer after the handshake completes. No-one asks how to put an oauth token in a TCP/IP SYN packet. But the reality is that we've ended up in a fuzzy middle-ground that is hard to explain.

[annevk]

@ricea given that we do add browser-supplied cookies and authentication, it's quite a reasonable request that we also enable headers, since much WebSocket server-side infrastructure probably depends on such information being in the handshake.

And at least as far as the specification is concerned the handshake shares many aspects with Fetch, to enable mixed content blocking, HSTS, CSP, etc. So from that perspective this is not that much of a stretch. (I wonder if Chrome's approach is shared by other browsers.)

[davidfowl]

@ricea

The WebSocket handshake security model hinges on exposing no more capabilities for request forgery than are already possible using an img tag. It is not possible to add arbitrary headers to the request using an img tag.

Can you elaborate here. I don't understand why the security model hinges on not allowing headers to be set in the client API? I'm not sure I understand this.

[annevk]

@davidfowl part of the same-origin policy is that we don't send "attacker-controlled" headers to cross-origin URLs. That's why if you want to do that you need to use CORS, which uses a CORS preflight to make an explicit check that the cross-origin URL is okay with the headers about to be transmitted.

The objection here, at least from the Chrome team, is that supporting a CORS preflight for WebSocket is too costly.

[davidfowl]

@annevk Thanks for that clarity!

So attacker controlled headers are seen as more dangerous than attacker controlled query string. Would it help if we restricted the types of headers that could be sent?

[annevk]

Yes, you can basically reach any arbitrary URL (whatever the query string or path), but you can only control headers to a very limited extent (and methods too, only HEAD/GET/POST).

The request headers we allow "attackers" to control are listed at https://fetch.spec.whatwg.org/#cors-safelisted-request-header, but note that we plan to lock that down some more in whatwg/fetch#736. For the kinds of use cases that people seem to have I don't think allowing these would be sufficient (or it would lead to hacks where you put authorization data in Accept-Language or some such).

[davidfowl]

@annevk The current design forces people to send what would typically go in the the Authorization header in the query string. Is that any less secure than allowing say the authorization header to bet set on the upgrade request? (even cross origin)

[annevk]

The concern is not with the security of the application making the request, it's with the security of the remote server. The remote server will have to be robust against arbitrary URLs, but it does not have to be robust against arbitrary headers, since it'll assume (due to the long history of the web and early establishment of the same-origin policy) that those cannot come from browsers.

[davidfowl]

I see, understand the push back now. So there needs to be a pre-flight request integrated into the flow and that's expensive in chrome because of the current implementation split between websockets and xhr/fetch. Is that a fair summary?

[annevk]

Excellent summary.

[davidfowl]

Should we get some of the other browser implementors to chime in? I can probably get somebody from the edge team to chime in here about the difficulty there. I'm not sure how to go about getting the other browser vendors interested enough to look at this issue (safari, firefox, anything else?)

[annevk]

Maybe @mcmanus for Firefox and @youennf for Safari? Feedback from Edge would be good.

[mcmanus]

I'm kinda surprised people aren't smuggling this through subprotocol - seems obvious enough.

architecturally speaking, a CORS preflight wouldn't be a big burden for firefox to implement.

otoh, designs that result in a lot of preflights suck so I'm not convinced we want to enable this rather than pushing it into the post websocket-handshake data.

[benaadams]

rather than pushing it into the post websocket-handshake data.

Then its a per web app custom protocol implementation; open to a slow loris type, slow auth issue from an unknown user; depending on how the application developer implements it; vs a well understood header handling implementation by the webserver?

[blakeanator]

I cannot speak to the spec as I am definitely not an expert in that area. But clearly there is demand from developers to have custom headers for WebSockets within browsers. And clearly it's possible to implement as other languages/libraries have done so.

Unless there is some security risk, I don't see why this shouldn't be a thing. Obviously there is a lot more to the story in terms of modifying the standard - but I'm surprised this isn't on the development radar as something that should be added at some point.

[davidfowl]

Is there anything we can do to push this forward?

[Benedicht]

It seems that the demand and the willingness(mostly at least) is in fawor of this change. How we can make it happen? What about an update to the standard?

[Misiu]

@davidfowl I think you should directly contact the Edge team. Both Chrome and Edge are using the same engine (Chromium), maybe the team on the Microsoft side would be able to implement this so that both browsers will be able to support this. Fingers crossed 🤞

[alexandis]

@Misiu In the specific case of WebSockets I think passing an authentication token in the URL is okay. The reason is that, unlike HTTP URLs, wss: URLs are never exposed to the user. They can't bookmark them or copy-and-paste them.

Uhm... what do you mean?

[benaadams]

The headers can be sent with the Upgrade: WebSocket request as at that time the protocol and scheme is same origin e.g. https:// even though later it changes to a different protocol

[Jarred-Sumner]

In Bun v0.5, we implemented headers support in the WebSocket constructor

In TypeScript types form, essentially this:

declare class WebSocket extends EventTarget {
  constructor(
    url: string,
    options?: { headers?: HeadersInit; protocols?: string[] }
  );
  // ...rest of WebSocket
}

This constructor is used when url is a string and options is an object that is not iterable. The headers used to construct the WebSocket is not exposed to the WebSocket instance (unlike url). It's not exposed mostly to minimize differences between the web API version of WebSocket and Bun's implementation.

[josephrocca]

I notice that throughout this thread, many are suggesting that only the Authorization header needs to be exposed.

In my case I need to get through the gateway of a container hosting service to access my particular container/server, and the only way to get through that gateway is to have an Foo-Api-Key header with the correct value on every request. So this can be easily done with normal HTTPS requests, but there's no way to talk to my server over WebSocket.

Note that this "gateway" API key is different to the "application-level" API key (which is usually Authorization I guess). I just wanted to mention this use case in case people aren't considering cases like this where multiple "layers" of authorization may be required, which requires multiple different headers.

[joffrey-bion]

And here I am many years later implementing a multiplatform Kotlin wrapper for websockets and STOMP, and realizing the only platform that doesn't support this is the browser...

The motivation for allowing custom headers seemed pretty satisfactory. For instance, handling HTTP authentication uniformly for all HTTP requests (including WS upgrade requests), or authenticating the request before completing the handshake and getting data over the WS, or passing other kind of gateway-required headers.

I don't quite understand the reasons that were put forward as a justification for not implementing this. When reading the whole thread, I mostly witnessed a semantics game about the handshake being HTTP or not, or nitpicking on exact words being used in the specs, with little actual arguments.
The only objective argument I could gather was that it would require a CORS pre-flight, which would be a lot of work, but if I understand correctly it's already acknowledged that this will be necessary.

So what remains as a blocker for this to happen?

[JokerQyou]

Well, previously the main blocker is the lack of implementer interest, and things have not changed, if I understand it correctly. Maintainers of Chrome's WebSocket implementation argued their fetch/XHR and WebSocket stack are so different that adding custom headers for WebSocket handshake would result in too much maintenance cost.
Personally I think these are two different views: web developers tend to think WebSocket as a kind of upgraded HTTP, so it should naturally support (most if not all) features of HTTP; Developers of the WebSocket implementations, on the other hand, argues that WebSocket is a low-level stack like TCP/IP, so most of the required features of HTTP simply don't belong to this layer.

[jonenst]

So websockets support the Host header, http paths and query parameters but not the Authorization header. I guess the only reason is that the former are more widely used by the general proxy/reverse proxy system of the web. Because the argument that it could all be implemented by custom application protocols is valid for all of them.

[kaqqao]

Here's something fantastic. Kubernetes smuggles authorization tokens inside Sec-WebSocket-Protocol because of course it does given the situation. It's extra funny how both Kubernetes and Chrome WS API are developed by the same but also entirely different Google.

Can we all agree that if any header at all can be specified, then that header will inevitably be abused to emulate the others and that the resulting situation has absolutely no upsides compared to simply supporting the freaking header that the entire internet seems to so desperately need?

[randomstuff]

Maybe we need a RFC to standardize the practice of trying to pass HTTP headers as subprotocols using such a scheme:

ijustwanttopassasimplehttpheader.$(base64 header_name).$(base64 header_value)

[benaadams]

It should pass the headers with the http UPGRADE request; as then if the headers aren't right you can 404 or 401 before even preforming the upgrade