Zookeeper KV backend – Error running traefik: zk: unknown error

[Overbryd]

Do you want to request a feature or report a bug?

Bug

What did you do?

Run traefik with zookeeper backend on Kubernetes.

What did you expect to see?

Traefik configured and running against Zookeeper.

What did you see instead?

[...]
Error running traefik: zk: unknown error

Output of traefik version: (What version of Traefik are you using?)

traefik version

Version:      v1.4.5
Codename:     roquefort
Go version:   go1.9.2
Built:        2017-12-06_10:16:48AM
OS/Arch:      linux/amd64

zookeeper version

Zookeeper version: 3.4.10

What is your environment & configuration (arguments, toml, provider, platform, ...)?

traefik.toml

defaultEntryPoints = ["http", "https"]
logLevel = "ERROR"

[web]
address = ":8080"

[entryPoints]
  [entryPoints.http]
  address = ":80"
  [entryPoints.https]
  address = ":443"
    [entryPoints.https.tls]

[zookeeper]
endpoint = "zk-svc.zoo.svc.cluster.local:2181"

[acme]
email = "redacted@redacted.com"
storage = "traefik/acme/account"
entryPoint = "https"
OnHostRule = true

If applicable, please paste the log output in debug mode (--debug switch)

traefik storeconfig output

traefik storeconfig --debug --configfile=/config-map/traefik.toml

2017/12/08 14:24:47 Storing configuration: {"GraceTimeOut":10000000000,"Debug":true,"CheckNewVersion":true,"AccessLogsFile":"","AccessLog":null,"TraefikLogsFile":"","LogLevel":"ERROR","EntryPoints":{"http":{"Network":"","Address":":80","TLS":null,"Redirect":null,"Auth":null,"WhitelistSourceRange":null,"Compress":false,"ProxyProtocol":null,"ForwardedHeaders":null},"https":{"Network":"","Address":":443","TLS":{"MinVersion":"","CipherSuites":null,"Certificates":null,"ClientCAFiles":null},"Redirect":null,"Auth":null,"WhitelistSourceRange":null,"Compress":false,"ProxyProtocol":null,"ForwardedHeaders":null}},"Cluster":null,"Constraints":[],"ACME":{"Email":"redacted@redacted.com","Domains":null,"Storage":"traefik/acme/account","StorageFile":"","OnDemand":false,"OnHostRule":true,"CAServer":"","EntryPoint":"https","DNSProvider":"","DelayDontCheckDNS":0,"ACMELogging":false,"TLSConfig":null},"DefaultEntryPoints":["http","https"],"ProvidersThrottleDuration":2000000000,"MaxIdleConnsPerHost":200,"IdleTimeout":0,"InsecureSkipVerify":false,"RootCAs":null,"Retry":null,"HealthCheck":{"Interval":30000000000},"RespondingTimeouts":null,"ForwardingTimeouts":null,"Docker":null,"File":null,"Web":{"Address":":8080","CertFile":"","KeyFile":"","ReadOnly":false,"Statistics":null,"Metrics":null,"Path":"","Auth":null,"Debug":false,"CurrentConfigurations":null,"Stats":null,"StatsRecorder":null},"Marathon":null,"Consul":null,"ConsulCatalog":null,"Etcd":null,"Zookeeper":{"Watch":true,"Filename":"","Constraints":[],"Trace":false,"DebugLogGeneratedTemplate":false,"Endpoint":"zk-svc.zoo.svc.cluster.local:2181","Prefix":"/traefik","TLS":null,"Username":"","Password":""},"Boltdb":null,"Kubernetes":null,"Mesos":null,"Eureka":null,"ECS":null,"Rancher":null,"DynamoDB":null}
2017/12/08 14:24:47 Connected to 10.35.240.104:2181
2017/12/08 14:24:47 Authenticated: id=99138071782883356, timeout=30000
2017/12/08 14:24:47 Re-submitting `0` credentials after reconnect
2017/12/08 14:24:47 Error running traefik: zk: unknown error

snip of the zookeeper log output

2017-12-08 14:24:47,624 [myid:3] - INFO  [ProcessThread(sid:3 cport:-1)::PrepRequestProcessor@648] - Got user-level KeeperException when processing sessionid:0x160358d1056001c type:setData cxid:0x4 zxid:0x7000000f1 txntype:-1 reqpath:n/a Error Path://traefik/accesslogsfile Error:KeeperErrorCode = BadArguments for //traefik/accesslogsfile

2017-12-08 14:24:47,624 [myid:3] - INFO  [ProcessThread(sid:3 cport:-1)::PrepRequestProcessor@514] - Invalid path //traefik/accesslogsfile with session 0x160358d1056001c, reason: Invalid path string "//traefik/accesslogsfile" caused by empty node name specified @1

2017-12-08 14:24:47,622 [myid:3] - INFO  [ProcessThread(sid:3 cport:-1)::PrepRequestProcessor@648] - Got user-level KeeperException when processing sessionid:0x160358d1056001c type:create cxid:0x3 zxid:0x7000000f0 txntype:-1 reqpath:n/a Error Path://traefik Error:KeeperErrorCode = BadArguments for //traefik

2017-12-08 14:24:47,622 [myid:3] - INFO  [ProcessThread(sid:3 cport:-1)::PrepRequestProcessor@514] - Invalid path //traefik with session 0x160358d1056001c, reason: Invalid path string "//traefik" caused by empty node name specified @1

2017-12-08 14:24:47,620 [myid:3] - INFO  [ProcessThread(sid:3 cport:-1)::PrepRequestProcessor@648] - Got user-level KeeperException when processing sessionid:0x160358d1056001c type:create cxid:0x2 zxid:0x7000000ef txntype:-1 reqpath:n/a Error Path:/ Error:KeeperErrorCode = NodeExists for /

2017-12-08 14:28:59,996 [myid:1] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1044] - Closed socket connection for client /10.32.8.30:50904 which had sessionid 0x160358d1056001f

EndOfStreamException: Unable to read additional data from client sessionid 0x160358d1056001f, likely client has closed socket
        at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
        at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
        at java.lang.Thread.run(Thread.java:748)

2017-12-08 14:28:59,996 [myid:1] - WARN  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@368] - caught end of stream exception

2017-12-08 14:28:59,663 [myid:1] - INFO  [CommitProcessor:1:ZooKeeperServer@687] - Established session 0x160358d1056001f with negotiated timeout 30000 for client /10.32.8.30:50904

2017-12-08 14:28:59,660 [myid:1] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@942] - Client attempting to establish new session at /10.32.8.30:50904

2017-12-08 14:28:59,659 [myid:1] - WARN  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@896] - Connection request from old client /10.32.8.30:50904; will be dropped if server is in r-o mode

2017-12-08 14:28:59,658 [myid:1] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@192] - Accepted socket connection from /10.32.8.30:50904

[miadabrin]

Getting the same error

2017/12/11 12:57:10 Storing configuration: {"GraceTimeOut":10000000000,"Debug":true,"CheckNewVersion":false,"AccessLogsFile":"","AccessLog":null,"TraefikLogsFile":"","LogLevel":"INFO","EntryPoints":{"http":{"Network":"","Address":":80","TLS":null,"Redirect":{"EntryPoint":"","Regex":"^http://www[.](.*)","Replacement":"http://$1"},"Auth":null,"WhitelistSourceRange":null,"Compress":true,"ProxyProtocol":null,"ForwardedHeaders":null},"https":{"Network":"","Address":":443","TLS":{"MinVersion":"","CipherSuites":null,"Certificates":null,"ClientCAFiles":null},"Redirect":null,"Auth":null,"WhitelistSourceRange":null,"Compress":true,"ProxyProtocol":null,"ForwardedHeaders":null}},"Cluster":null,"Constraints":[],"ACME":{"Email":"somemail@someplace.com","Domains":null,"Storage":"traefik/acme/account","StorageFile":"","OnDemand":true,"OnHostRule":true,"CAServer":"","EntryPoint":"https","DNSProvider":"","DelayDontCheckDNS":0,"ACMELogging":true,"TLSConfig":null},"DefaultEntryPoints":["http","https"],"ProvidersThrottleDuration":2000000000,"MaxIdleConnsPerHost":500,"IdleTimeout":180000000000,"InsecureSkipVerify":false,"RootCAs":null,"Retry":{"Attempts":3},"HealthCheck":{"Interval":30000000000},"RespondingTimeouts":null,"ForwardingTimeouts":null,"Docker":null,"File":null,"Web":null,"Marathon":null,"Consul":null,"ConsulCatalog":null,"Etcd":null,"Zookeeper":{"Watch":true,"Filename":"","Constraints":[],"Trace":false,"DebugLogGeneratedTemplate":false,"Endpoint":"172.16.0.1:2181","Prefix":"/traefik","TLS":null,"Username":"","Password":""},"Boltdb":null,"Kubernetes":{"Watch":true,"Filename":"","Constraints":[],"Trace":false,"DebugLogGeneratedTemplate":false,"Endpoint":"","Token":"","CertAuthFilePath":"","DisablePassHostHeaders":false,"Namespaces":null,"LabelSelector":""},"Mesos":null,"Eureka":null,"ECS":null,"Rancher":null,"DynamoDB":null}
2017/12/11 12:57:10 Authenticated: id=72268363871551503, timeout=30000
2017/12/11 12:57:10 Re-submitting `0` credentials after reconnect
2017/12/11 12:57:10 Error running traefik: zk: unknown error

[eubnara]

I fix the problem similar to this by applying #948

-prefix = "/traefik"
+prefix = "traefik"

If you do not set "prefix", the default value("/traefik") will be set.

[Overbryd]

@eubnara thanks, this solved the issue.

Edit: For the other things that I encountered afterwards, I created a separate issue: #2581

[traefiker]

Closed by #2580.