Websockets support

[andrewwebber]

Does Envoy support websockets as I seem to be having some issues connecting from a golang client with a golang Backend server.

[mattklein123]

No, we don't currently support websockets.

[andrewwebber]

This is very useful information as I will stop trying to find a workaround for this. I guess I will have to refactor my push microservice to use HTTP2 somehow

[mattklein123]

Going to reopen just to track the feature request.

[andrewwebber]

Traefik claims to support HTTP2 as well as websockets. What is the limitation here...

traefik/traefik#8

[mattklein123]

The limitation is we don't support the websocket protocol. :)

It needs to get implemented, probably just as a new L4 filter that sits in front of the tcp_proxy filter.

[rshriram]

Based on gitter conversations with @mattklein123

make a new filter, use the http1 codec, handle the “upgrade", then just transition to straight tcp proxy semantics. Could probably just implement as as filter that sits in front of tcp_proxy, or use the existing http1 codec_impl to do the initial handshake then just pass through data to tcp_proxy.

[andrewwebber]

@rshiram do you have an example JSON confirm you could share?

[rshriram]

@andrewwebber this feature is not implemented yet. I was merely logging our conversations in this topic.

[captjt]

Is there a potential timeline or update whether or not Envoy will support websockets?

[mattklein123]

@jtaylor32 no, no update. Need someone to sign up to implement this.

[mattklein123]

@rshriram I looked through the code and IMO this is the easiest/cleanest way to implement the required functionality. Let's discuss here if there are questions. cc @htuch

• We should explicitly specify in the route configuration whether a route is a WS route or not. Meaning, we will not support attempting to detect if it is WS all the way back to the backend. If we receive a WS upgrade over HTTP/1.1, we will check the route table to see if the matched route is WS, and if not, fail the request with 400 or some other code.
• In ConnectionManagerImpl::ActiveStream::decodeHeaders, we will check if 1) connection is HTTP/1.1, WS upgrade is requested, and consult the route table to check if route match is WS route. Note that internally in the http/1.1 codec, http_parser should correctly raise message complete for an upgrade request after the headers, thus we should see end_stream set to true in the above call. (There may be small logic changes required in the codec to make sure this is true but they should be simple.)
• If we detect that we are going to enter WS mode when we get headers, we will not enter filter chain processing, and will instead shift the connection manager into a WS processing mode.
• For this we should make an entirely new interface and impl so that it's easy to test this behavior. Basically the conn_man will create a WsHandlerImpl which implements the WsHandler interface.
• The main entry point for this interface once the conn_man enters WS mode will be inside ConnectionManagerImpl::onData. If in WS mode, we will dispatch data from the connection directly into the WS handler, and skip the HTTP codec.
• The WS handler implementation can send the upgrade headers to the backend via a raw TCP connection (load balanced per normal via the cluster manager), and then just start proxying data a-la TcpProxy. There is probably some ability to share code between WsHandlerImpl and TcpProxy so need to look at that.

These are rough notes but should be good enough to get started. LMK if you have more detailed questions and we can figure it out.

[rshriram]

It's a pass through. If your backend doesn't speak websockets, then Envoy can't do anything can it?

On Wed, Jun 21, 2017 at 5:20 PM politician ***@***.***> wrote: @mattklein123 <https://github.com/mattklein123> Can you clarify whether the plan is to terminate HTTP/WebSockets at envoy or whether you're going to pass the protocol through? I can't tell from the last bullet whether a backend needs to be a full HTTP/WebSockets server or just a basic TCP socket server that supports a handshake containing the upgrade headers. For my use case, the latter is preferable. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#319 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AH0qd3bAlf5roVtMaFS5tf4yunADTkqGks5sGYkmgaJpZM4Lb2a-> .

-- ~shriram

[htuch]

Why can't Envoy terminate the websockets and speak plain TCP to the backend? This seems a reasonable ask.

[rshriram]

I think there is Nomenclature mismatch. @politician simply put, if you have a nodejs websockets app behind an Envoy for example, you can talk to Envoy via websocket and Envoy will talk to your app via websocket. If your nodejs app doesn't support websockets, then Envoy will refuse websockets connection from clients. Similarly, if the client speaks just http (not websockets), and the server speaks only websockets, then Envoy will refuse client connection for that specific route.

On Wed, Jun 21, 2017 at 8:52 PM htuch ***@***.***> wrote: Why can't Envoy terminate the websockets and speak plain TCP to the backend? This seems a reasonable ask. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#319 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AH0qd8T2DwzHcCbu6Eo-COj326xXhT9Dks5sGbrCgaJpZM4Lb2a-> .

-- ~shriram

[htuch]

I think what's being asked here is that you have a nodejs TCP application (to continue the example), which does not speak HTTP or websockets. We have a browser client that speaks websockets. Envoy could act as a bridge from the client to the nodejs TCP application. Is this what you were thinking of @politician?

[mattklein123]

If there is a use case that involves WS <-> TCP without the upgrade headers, that will be trivial to implement via a configuration option, so we can cross that bridge when we come to it.

[mattklein123]

Fixed by #1274. Thanks @rshriram !

[Nevon]

If there is a use case that involves WS <-> TCP without the upgrade headers, that will be trivial to implement via a configuration option, so we can cross that bridge when we come to it.

Years later now, but does anyone know if this is possible now? I have my own hand-rolled application for WS<->TCP communication, but I was hoping that I could replace it with Envoy.

[gopakumarce]

Not sure where to ask a question - the question is to what extent does envoy act as a proxy for websocket ? After the Upgrade handshake is done, does it ever look into the websocket framing itself / does it care about the websocket data being proper - or does it just treat it as a tcp after wards ?

The reason for asking is to know whether I can use an http upgrade sequence and then send non-websocket data over that connection assuming my client and server are in my control - will envoy have any unhappiness with that ?