Traefik not working with rancher

[matq007]

Bug

What did you do?

I've cloned the latest master (1d2d0ce) and found out that traefik is not loading the traefik configuration. From version 1.3 it returns {}. Rancher version v1.6.5

What did you expect to see?

I've expected traefik would detect frontend and backend.

What did you see instead?

Configuration received from provider rancher: {}

Output of traefik version: (What version of Traefik are you using?)

Version: dev
Codename: cheddar
Go version: go1.8.3
Built: I don't remember exactly
OS/Arch: linux/amd64

What is your environment & configuration (arguments, toml, provider, platform, ...)?

################################################################
## Global configuration
#################################################################

logLevel = "DEBUG"

#[entryPoints]
#  [entryPoints.http]
#  address = ":80"

[web]
address = ":8081"

################################################################
# Rancher configuration backend
################################################################
[rancher]

# Default domain used.
# Can be overridden by setting the "traefik.domain" label on an service.
#
# Required
#
domain = "rancher.novicell.dk"

# Enable watch Rancher changes
#
# Optional
# Default: true
#
Watch = true

# Polling interval (in seconds)
#
# Optional
#
RefreshSeconds = 15

# Expose Rancher services by default in traefik
#
# Optional
# Default: true
#

# Filter services with unhealthy states and health states
#
# Optional
# Default: false
#
EnableServiceHealthFilter = false

# Endpoint to use when connecting to Rancher
#
# Required
Endpoint = "http://10.0.55.165:8080/v1"

# AccessKey to use when connecting to Rancher
#
# Required
AccessKey = "0AF1BABCADF09B91CE91"

# SecretKey to use when connecting to Rancher
#
# Required
SecretKey = "d6U2EorP7RVugZG8ragxERKEYJAm7FBtRs4xfLCY"

If applicable, please paste the log output in debug mode (--debug switch)

Version 1.3

[rancher@nc-rancher-ui01 traefik-1.3]$ ./traefik --configFile=traefik.toml
INFO[2017-08-21T09:43:54+02:00] Traefik version v1.3.5 built on 2017-08-01_04:18:59PM
INFO[2017-08-21T09:43:54+02:00] Using TOML configuration file /home/rancher/traefik-1.3/traefik.toml
DEBU[2017-08-21T09:43:54+02:00] Global configuration loaded {"GraceTimeOut":10000000000,"Debug":false,"CheckNewVersion":true,"AccessLogsFile":"","TraefikLogsFile":"","LogLevel":"DEBUG","EntryPoints":{"http":{"Network":"","Address":":80","TLS":null,"Redirect":null,"Auth":null,"Compress":false}},"Cluster":null,"Constraints":[],"ACME":null,"DefaultEntryPoints":["http"],"ProvidersThrottleDuration":2000000000,"MaxIdleConnsPerHost":200,"IdleTimeout":180000000000,"InsecureSkipVerify":false,"Retry":null,"HealthCheck":{"Interval":30000000000},"Docker":null,"File":null,"Web":{"Address":":8081","CertFile":"","KeyFile":"","ReadOnly":false,"Statistics":null,"Metrics":null,"Path":"","Auth":null},"Marathon":null,"Consul":null,"ConsulCatalog":null,"Etcd":null,"Zookeeper":null,"Boltdb":null,"Kubernetes":null,"Mesos":null,"Eureka":null,"ECS":null,"Rancher":{"Watch":true,"Filename":"","Constraints":null,"Endpoint":"http://10.0.55.165:8080/v1","AccessKey":"0AF1BABCADF09B91CE91","SecretKey":"d6U2EorP7RVugZG8ragxERKEYJAm7FBtRs4xfLCY","ExposedByDefault":false,"Domain":"rancher.novicell.dk","RefreshSeconds":15,"EnableServiceHealthFilter":false},"DynamoDB":null}
INFO[2017-08-21T09:43:54+02:00] Preparing server http &{Network: Address::80 TLS:<nil> Redirect:<nil> Auth:<nil> Compress:false}
INFO[2017-08-21T09:43:54+02:00] Starting provider *server.WebProvider {"Address":":8081","CertFile":"","KeyFile":"","ReadOnly":false,"Statistics":null,"Metrics":null,"Path":"","Auth":null}
INFO[2017-08-21T09:43:54+02:00] Starting provider *rancher.Provider {"Watch":true,"Filename":"","Constraints":null,"Endpoint":"http://10.0.55.165:8080/v1","AccessKey":"0AF1BABCADF09B91CE91","SecretKey":"d6U2EorP7RVugZG8ragxERKEYJAm7FBtRs4xfLCY","ExposedByDefault":false,"Domain":"rancher.novicell.dk","RefreshSeconds":15,"EnableServiceHealthFilter":false}
INFO[2017-08-21T09:43:54+02:00] Starting server on :80
ERRO[2017-08-21T09:43:54+02:00] Error creating server: listen tcp :80: bind: permission denied
DEBU[2017-08-21T09:43:54+02:00] first container len: %!i(int=19)
DEBU[2017-08-21T09:43:54+02:00] Filtering service scheduler/scheduler without traefik.port label
DEBU[2017-08-21T09:43:54+02:00] Filtering service healthcheck/healthcheck without traefik.port label
DEBU[2017-08-21T09:43:54+02:00] Filtering service network-services/metadata without traefik.port label
DEBU[2017-08-21T09:43:54+02:00] Filtering service network-services/network-manager without traefik.port label
DEBU[2017-08-21T09:43:54+02:00] Filtering service healthcheck/healthcheck without traefik.port label
DEBU[2017-08-21T09:43:54+02:00] Filtering service network-services/network-manager without traefik.port label
DEBU[2017-08-21T09:43:54+02:00] Filtering service network-services/metadata without traefik.port label
DEBU[2017-08-21T09:43:54+02:00] Filtering service scheduler/scheduler without traefik.port label
DEBU[2017-08-21T09:43:54+02:00] Filtering service test/ghost-2 without traefik.port label
DEBU[2017-08-21T09:43:54+02:00] Validation of load balancer method for backend backend-ghot failed: invalid load-balancing method ''. Using default method wrr.
DEBU[2017-08-21T09:43:54+02:00] Configuration received from provider rancher: {"backends":{"backend-ghot":{"servers":{"server-0":{"url":"http://:2368","weight":0}},"loadBalancer":{"method":"wrr"}}},"frontends":{"frontend-Host-ghost-rancher-novicell-dk":{"entryPoints":["http"],"backend":"backend-ghot","routes":{"route-frontend-Host-ghost-rancher-novicell-dk":{"rule":"Host:ghost.rancher.novicell.dk"}},"passHostHeader":true,"priority":0,"basicAuth":[]}}}
DEBU[2017-08-21T09:43:54+02:00] Last rancher config received more than 2s, OK
DEBU[2017-08-21T09:43:54+02:00] Creating frontend frontend-Host-ghost-rancher-novicell-dk
DEBU[2017-08-21T09:43:54+02:00] Wiring frontend frontend-Host-ghost-rancher-novicell-dk to entryPoint http
DEBU[2017-08-21T09:43:54+02:00] Creating route route-frontend-Host-ghost-rancher-novicell-dk Host:ghost.rancher.novicell.dk
DEBU[2017-08-21T09:43:54+02:00] Creating backend backend-ghot
DEBU[2017-08-21T09:43:54+02:00] Creating load-balancer wrr
DEBU[2017-08-21T09:43:54+02:00] Creating server server-0 at http://:2368 with weight 0
INFO[2017-08-21T09:43:54+02:00] Server configuration reloaded on :80
^CINFO[2017-08-21T09:43:56+02:00] I have to go... interrupt
INFO[2017-08-21T09:43:56+02:00] Stopping server
DEBU[2017-08-21T09:43:56+02:00] Waiting 10s seconds before killing connections on entrypoint http...
DEBU[2017-08-21T09:43:56+02:00] Entrypoint http closed
INFO[2017-08-21T09:43:56+02:00] Server stopped
INFO[2017-08-21T09:43:56+02:00] Shutting down

On master

INFO[2017-08-21T10:04:27+02:00] Starting server on :80
DEBU[2017-08-21T10:04:27+02:00] Configuration received from provider rancher: {}
DEBU[2017-08-21T10:04:27+02:00] Last rancher config received more than 2s, OK
INFO[2017-08-21T10:04:27+02:00] Server configuration reloaded on :80
^CINFO[2017-08-21T10:04:36+02:00] I have to go... interrupt
INFO[2017-08-21T10:04:36+02:00] Stopping server
DEBU[2017-08-21T10:04:36+02:00] Waiting 10s seconds before killing connections on entrypoint http...
DEBU[2017-08-21T10:04:36+02:00] Entrypoint http closed
INFO[2017-08-21T10:04:36+02:00] Server stopped
INFO[2017-08-21T10:04:36+02:00] Shutting down
ERRO[2017-08-21T10:04:36+02:00] Error creating server: http: Server closed

[matq007]

[UPDATE]: So I've made a mistake and haven't submitted the environment key. Nevertheless if you remove the condition

if service.EnvironmentId != environment.Id {
    continue
}

the frotned is loaded correctly, but the backend is not working. More specifically Backends is empty when configureBackends executes.

---

Found out that listRancherEnvironments is returning only 2 environments which are used in parseAPISeourcedRancherData.

if service.EnvironmentId != environment.Id {
    continue
}

# environment.Id
DEBU[2017-08-23T10:13:23+02:00] ENV VALUE: 1a5
DEBU[2017-08-23T10:13:23+02:00] ENV VALUE: 1a12

This condition is always true in my case which results in having no data. When you look at service JSON below, you can't match them. You can match them only by AccountId. I've tried that and now I'm getting the frontend, but no backend is created. Working on solving that also. Looking forward for some feedback and maybe help :).

"id": "1s43",
"type": "service",
"links": {
"self": "…/v1/services/1s43",
"account": "…/v1/services/1s43/account",
"configItemStatuses": "…/v1/services/1s43/configitemstatuses",
"consumedbyservices": "…/v1/services/1s43/consumedbyservices",
"consumedservices": "…/v1/services/1s43/consumedservices",
"instances": "…/v1/services/1s43/instances",
"networkDrivers": "…/v1/services/1s43/networkdrivers",
"serviceExposeMaps": "…/v1/services/1s43/serviceexposemaps",
"serviceLogs": "…/v1/services/1s43/servicelogs",
"storageDrivers": "…/v1/services/1s43/storagedrivers",
"containerStats": "…/v1/services/1s43/containerstats",
"environment": "…/v1/environment/1e20"
},
"actions": {
"upgrade": "…/v1/services/1s43/?action=upgrade",
"restart": "…/v1/services/1s43/?action=restart",
"update": "…/v1/services/1s43/?action=update",
"remove": "…/v1/services/1s43/?action=remove",
"deactivate": "…/v1/services/1s43/?action=deactivate",
"removeservicelink": "…/v1/services/1s43/?action=removeservicelink",
"addservicelink": "…/v1/services/1s43/?action=addservicelink",
"setservicelinks": "…/v1/services/1s43/?action=setservicelinks"
},
"baseType": "service",
"name": "ghost",
"state": "active",
"accountId": "1a12",
"assignServiceIpAddress": false,
"createIndex": 1,
"created": "2017-08-21T09:21:49Z",
"createdTS": 1503307309000,
"currentScale": 1,
"data": {
"fields": {
"startOnCreate": true,
"secondaryLaunchConfigs": [ ],
"launchConfig": {
"instanceTriggeredStop": "stop",
"kind": "container",
"networkMode": "managed",
"privileged": false,
"publishAllPorts": false,
"readOnly": false,
"startOnCreate": true,
"stdinOpen": true,
"tty": true,
"vcpu": 1,
"labels": {
"traefik.enable": "true",
"traefik.port": "2368",
"traefik.backend": "ghot",
"traefik.frontend.rule": "Host:ghost.rancher.novicell.dk",
"io.rancher.container.pull_image": "always"
},
"secrets": [ ],
"dataVolumes": [ ],
"dataVolumesFrom": [ ],
"dns": [ ],
"dnsSearch": [ ],
"capAdd": [ ],
"capDrop": [ ],
"devices": [ ],
"logConfig": {
"driver": "",
"config": { }
},
"dataVolumesFromLaunchConfigs": [ ],
"imageUuid": "docker:ghost",
"ports": [
"2368:2368/tcp"
],
"system": false,
"version": "0"
},
"scale": 1,
"serviceIndexStrategy": "deploymentUnitBased",
"assignServiceIpAddress": false,
"token": "zES29dyWqj38QpdpiX4z8khWPj72DHF2JiAXbaBv",
"executionCount": 1,
"executionPeriodStart": 1503307310120,
"publicEndpoints": [
{
"ipAddress": "10.0.55.164",
"port": 2368,
"serviceId": "43",
"hostId": "4",
"instanceId": "90"
}
],
"transitioning": null,
"transitioningMessage": null,
"currentScale": 1
}
},
"description": null,
"externalId": null,
"fqdn": null,
"healthState": "healthy",
"instanceIds": [
"1i90"
],
"kind": "service",
"launchConfig": {
"type": "launchConfig",
"capAdd": [ ],
"capDrop": [ ],
"dataVolumes": [ ],
"dataVolumesFrom": [ ],
"devices": [ ],
"dns": [ ],
"dnsSearch": [ ],
"imageUuid": "docker:ghost",
"instanceTriggeredStop": "stop",
"kind": "container",
"labels": {
"traefik.enable": "true",
"traefik.port": "2368",
"traefik.backend": "ghot",
"traefik.frontend.rule": "Host:ghost.rancher.novicell.dk",
"io.rancher.container.pull_image": "always"
},
"logConfig": {
"type": "logConfig",
"config": { },
"driver": ""
},
"networkMode": "managed",
"ports": [
"2368:2368/tcp"
],
"privileged": false,
"publishAllPorts": false,
"readOnly": false,
"secrets": [ ],
"startOnCreate": true,
"stdinOpen": true,
"system": false,
"tty": true,
"version": "0",
"dataVolumesFromLaunchConfigs": [ ],
"vcpu": 1
},
"lbConfig": null,
"linkedServices": null,
"metadata": null,
"publicEndpoints": [
{
"type": "publicEndpoint",
"hostId": "1h4",
"instanceId": "1i90",
"ipAddress": "10.0.55.164",
"port": 2368,
"serviceId": "1s43"
}
],
"removeTime": null,
"removed": null,
"retainIp": null,
"scale": 1,
"scalePolicy": null,
"secondaryLaunchConfigs": [ ],
"selectorContainer": null,
"selectorLink": null,
"stackId": "1st20",
"startOnCreate": true,
"system": false,
"transitioning": "no",
"transitioningMessage": null,
"transitioningProgress": null,
"upgrade": null,
"uuid": "37cd48c5-acb4-4e4e-90e7-85d1faf3733e",
"vip": null,
"environmentId": "1e20"
}

[SantoDE]

Hi @matq007 :)

I created a Pull Request with a fix and also an experimental image including that fix: #2053

Would love to get your feedback :)

[matq007]

Hey @SantoDE, thanks for the pull request. The problem is fixed now, I can see now frontend and backend. There is still a problem when the network is set to host, but I'll make a PR for that. I know how to fix that. Thanks 👍

[traefiker]

Closed by #2053.