WebSocket message lost using Traefik as Proxy

[datamesh-oss]

Do you want to request a feature or report a bug?

bug

What did you do?

Use Traefik as proxy for WebSocket server. For simplicity, we use a rather simple server-client model for test. When a client connects to the server, the server will send 5 messages to the client immediately. The server is behind a Traefik proxy.

The simple Server Client code to reproduce the problem can be found here:
https://github.com/datamesh-oss/traefik-ws

What did you expect to see?

Clients will receive all 5 messages.

What did you see instead?

Sometimes the Client receive only 3 or 4 messages. (The client will stuck forever since it expects 5 messages.) Some messages get dropped by Traefik.

This problem does not exist when we use nginx as proxy or no proxy at all.

Also, when we add a short delay before sending each message, the possibility to see the problem happen decreases. (e.g. the problem appears less often if we increase the delay from 1ms to 2ms)

I suspect it's a race condition somewhere in Traefik. Or there might be something wrong with the gorilla/websocket packages (but it's hard to explain why direct link or proxy from nginx has no such problem. Is it possible that we missed some Traefik configurations?).

Output of traefik version: (What version of Traefik are you using?)

Version: v1.5.0-rc3
Codename: cancoillotte
Go version: go1.9.2
Built: 2017-12-20_02:24:44PM
OS/Arch: darwin/amd64

➜  client git:(master) ✗ go run client.go
client recv message:  :----------------------------::12121 ------ 0
client recv message:  :----------------------------::12121 ------ 1
client recv message:  :----------------------------::12121 ------ 2
client recv message:  :----------------------------::12121 ------ 3
client recv message:  :----------------------------::12121 ------ 4

client recv message:  :----------------------------::12121 ------ 0
client recv message:  :----------------------------::12121 ------ 2
client recv message:  :----------------------------::12121 ------ 3
client recv message:  :----------------------------::12121 ------ 4

message 1 is missing here.

What is your environment & configuration (arguments, toml, provider, platform, ...)?

################################################################
# Global configuration
################################################################

# Enable debug mode
#
# Optional
# Default: false
#
debug = true

# Log level
#
# Optional
# Default: "ERROR"
#
logLevel = "DEBUG"

# Entrypoints to be used by frontends that do not specify any entrypoint.
# Each frontend can specify its own entrypoints.
#
# Optional
# Default: ["http"]
#
defaultEntryPoints = ["ws"]
InsecureSkipVerify=true

# Entrypoints definition
#
# Optional
# Default:
[entryPoints]
    [entryPoints.http]
    address = ":80"
    #[entryPoints.ws]
    #address = ":80"


[file]
	[backends]
		[backends.director]
			[backends.director.servers.all]
	    	url = "http://127.0.0.1:12121"

	[frontends]
		[frontends.director]
        backend = "director"
            [frontends.director.routes.all]
            rule = "PathPrefix:/"

If applicable, please paste the log output in debug mode (--debug switch)

DEBU[2017-12-29T15:17:52+08:00] vulcand/oxy/roundrobin/rr: begin ServeHttp on request  Request="{"Method":"GET","URL":{"Scheme":"","Opaque":"","User":null,"Host":"","Path":"/v1/a","RawPath":"","ForceQuery":false,"RawQuery":"","Fragment":""},"Proto":"HTTP/1.1","ProtoMajor":1,"ProtoMinor":1,"Header":{"Connection":["Upgrade"],"Sec-Websocket-Key":["BKO/hOHJofm3PJb3sIjMfg=="],"Sec-Websocket-Version":["13"],"Upgrade":["websocket"],"User-Agent":["Go-http-client/1.1"]},"ContentLength":0,"TransferEncoding":null,"Host":"127.0.0.1:80","Form":null,"PostForm":null,"MultipartForm":null,"Trailer":null,"RemoteAddr":"127.0.0.1:55653","RequestURI":"/v1/a","TLS":null}"
DEBU[2017-12-29T15:17:52+08:00] vulcand/oxy/roundrobin/rr: Forwarding this request to URL  Request="{"Method":"GET","URL":{"Scheme":"","Opaque":"","User":null,"Host":"","Path":"/v1/a","RawPath":"","ForceQuery":false,"RawQuery":"","Fragment":""},"Proto":"HTTP/1.1","ProtoMajor":1,"ProtoMinor":1,"Header":{"Connection":["Upgrade"],"Sec-Websocket-Key":["BKO/hOHJofm3PJb3sIjMfg=="],"Sec-Websocket-Version":["13"],"Upgrade":["websocket"],"User-Agent":["Go-http-client/1.1"]},"ContentLength":0,"TransferEncoding":null,"Host":"127.0.0.1:80","Form":null,"PostForm":null,"MultipartForm":null,"Trailer":null,"RemoteAddr":"127.0.0.1:55653","RequestURI":"/v1/a","TLS":null}" ForwardURL=http://192.168.1.52:12121 

DEBU[2017-12-29T15:17:52+08:00] vulcand/oxy/forward: begin ServeHttp on request  Request="{"Method":"GET","URL":{"Scheme":"http","Opaque":"","User":null,"Host":"192.168.1.52:12121 

","Path":"","RawPath":"","ForceQuery":false,"RawQuery":"","Fragment":""},"Proto":"HTTP/1.1","ProtoMajor":1,"ProtoMinor":1,"Header":{"Connection":["Upgrade"],"Sec-Websocket-Key":["BKO/hOHJofm3PJb3sIjMfg=="],"Sec-Websocket-Version":["13"],"Upgrade":["websocket"],"User-Agent":["Go-http-client/1.1"]},"ContentLength":0,"TransferEncoding":null,"Host":"127.0.0.1:80","Form":null,"PostForm":null,"MultipartForm":null,"Trailer":null,"RemoteAddr":"127.0.0.1:55653","RequestURI":"/v1/a","TLS":null}"
DEBU[2017-12-29T15:17:52+08:00] vulcand/oxy/forward/websocket: begin ServeHttp on request  Request="{"Method":"GET","URL":{"Scheme":"http","Opaque":"","User":null,"Host":"192.168.1.52:12121 

","Path":"","RawPath":"","ForceQuery":false,"RawQuery":"","Fragment":""},"Proto":"HTTP/1.1","ProtoMajor":1,"ProtoMinor":1,"Header":{"Connection":["Upgrade"],"Sec-Websocket-Key":["BKO/hOHJofm3PJb3sIjMfg=="],"Sec-Websocket-Version":["13"],"Upgrade":["websocket"],"User-Agent":["Go-http-client/1.1"]},"ContentLength":0,"TransferEncoding":null,"Host":"127.0.0.1:80","Form":null,"PostForm":null,"MultipartForm":null,"Trailer":null,"RemoteAddr":"127.0.0.1:55653","RequestURI":"/v1/a","TLS":null}"

[traefiker]

Closed by #2640.

[traefiker]

Closed by #2650.