Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Traefik crashes when calling Rancher's APIs. #1615

Closed
patriziotufarolo opened this issue May 16, 2017 · 8 comments
Closed

Traefik crashes when calling Rancher's APIs. #1615

patriziotufarolo opened this issue May 16, 2017 · 8 comments

Comments

@patriziotufarolo
Copy link

patriziotufarolo commented May 16, 2017

What version of Traefik are you using?

Version:      v1.2.3
Codename:     morbier
Go version:   go1.7.5
Built:        2017-04-13_07:21:10PM
OS/Arch:      linux/amd64

What is your environment & configuration (arguments, toml...)?

{
 "GraceTimeOut": 10,
 "Debug": false,
 "CheckNewVersion": true,
 "AccessLogsFile": "",
 "TraefikLogsFile": "",
 "LogLevel": "ERROR",
 "EntryPoints": {
  "http": {
   "Network": "",
   "Address": ":80",
   "TLS": null,
   "Redirect": null,
   "Auth": null,
   "Compress": false
  }
 },
 "Cluster": null,
 "Constraints": [],
 "ACME": null,
 "DefaultEntryPoints": [
  "http"
 ],
 "ProvidersThrottleDuration": 2000000000,
 "MaxIdleConnsPerHost": 200,
 "InsecureSkipVerify": false,
 "Retry": null,
 "Docker": null,
 "File": null,
 "Web": {
  "Address": ":8080",
  "CertFile": "",
  "KeyFile": "",
  "ReadOnly": false,
  "Statistics": null,
  "Metrics": null,
  "Auth": null
 },
 "Marathon": null,
 "Consul": null,
 "ConsulCatalog": null,
 "Etcd": null,
 "Zookeeper": null,
 "Boltdb": null,
 "Kubernetes": null,
 "Mesos": null,
 "Eureka": null,
 "ECS": null,
 "Rancher": {
  "Watch": true,
  "Filename": "",
  "Constraints": null,
  "Endpoint": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
  "AccessKey": "XXXXXXXXXXXXXXXXXXXXXXXXXX",
  "SecretKey": "XXXXXXXXXXXXXXXXXXXXXXXXXX",
  "ExposedByDefault": true,
  "Domain": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
 },
 "ConfigFile": "/etc/traefik/traefik.toml"
}

What did you do?

I am trying to integrate Traefik with Rancher using the configuration file to proxy containers. As you can see I kept my configuration as simple as possibile, basing on the one provided in the documentation.
This issue is related to #1218
but for everyone of them seems to have solved; of course this is not my case :D

The Rancher version i'm currently using is 1.5.9 (stable).

defaultEntryPoints = ["http"]
[entryPoints]
  [entryPoints.http]
  address = ":80"
[web]
address = ":8080"
[rancher]
domain = "XXX"
ExposedByDefault = true
Endpoint = "http://XXX/v1"
AccessKey = "XXX"
SecretKey = "XXX"

What did you expect to see?

I expect to see traefik working

What did you see instead?

ERRO[2017-05-16T16:54:16Z] Error in Go routine: runtime error: invalid memory address or nil pointer dereference
goroutine 57 [running]:
runtime/debug.Stack(0x4e837f, 0xc42000c2c0, 0x1db66d3)
        /usr/local/go/src/runtime/debug/stack.go:24 +0x79
runtime/debug.PrintStack()
        /usr/local/go/src/runtime/debug/stack.go:16 +0x22
github.com/containous/traefik/safe.defaultRecoverGoroutine(0x1af0560, 0xc42000a040)
        /go/src/github.com/containous/traefik/safe/routine.go:147 +0x7d
github.com/containous/traefik/safe.GoWithRecover.func1.1(0x1eb3a50)
        /go/src/github.com/containous/traefik/safe/routine.go:138 +0x57
panic(0x1af0560, 0xc42000a040)
        /usr/local/go/src/runtime/panic.go:458 +0x243
github.com/containous/traefik/provider.parseRancherData(0xc42034e300, 0x14, 0x20, 0xc42046ea00, 0x2f, 0x40, 0xc42034ee00, 0x14, 0x20, 0x2bc8032d, ...)
        /go/src/github.com/containous/traefik/provider/rancher.go:373 +0x298
github.com/containous/traefik/provider.(*Rancher).Provide.func1.1(0xc4203546c0, 0x42b488)
        /go/src/github.com/containous/traefik/provider/rancher.go:236 +0x217
github.com/containous/traefik/vendor/github.com/cenk/backoff.RetryNotify(0xc4203e2e60, 0x2a9cb00, 0xc4201895e0, 0x1eb39f0, 0xc420490f48, 0x65f19f)
        /go/src/github.com/containous/traefik/vendor/github.com/cenk/backoff/retry.go:32 +0x3f
github.com/containous/traefik/provider.(*Rancher).Provide.func1()
        /go/src/github.com/containous/traefik/provider/rancher.go:280 +0x132
github.com/containous/traefik/safe.GoWithRecover.func1(0x1eb3a50, 0xc4203e2e40)
        /go/src/github.com/containous/traefik/safe/routine.go:141 +0x4d
created by github.com/containous/traefik/safe.GoWithRecover
        /go/src/github.com/containous/traefik/safe/routine.go:142 +0x49

Thank you

@ldez ldez added area/provider/rancher kind/bug/possible a possible bug that needs analysis before it is confirmed or fixed. labels May 16, 2017
@ldez
Copy link
Member

ldez commented May 16, 2017

could you try with 1.3.0-rc2?

@patriziotufarolo
Copy link
Author

Same result with

Version:      v1.3.0-rc2
Codename:     raclette
Go version:   go1.8.1
Built:        2017-05-16_04:19:54PM
OS/Arch:      linux/amd64

@SantoDE
Copy link
Collaborator

SantoDE commented May 16, 2017

Could you verify that you not set the Service Labels to the traefik Service in Rancher? If they are set, try removing them please :)

@patriziotufarolo
Copy link
Author

@SantoDE I am currently running traefik with "docker run" without labels on my own laptop, pointing to my remote rancher setup.
The command I'm issuing is
docker run --rm -p 8080:8080 -p 80:80 -t -i -v $(pwd)/traefik-rancher.toml:/etc/traefik/traefik.toml traefik:1.3.0-rc2
where traefik-rancher.toml is the file provided above

@SantoDE SantoDE closed this as completed May 16, 2017
@SantoDE SantoDE reopened this May 16, 2017
@SantoDE
Copy link
Collaborator

SantoDE commented May 16, 2017

Could you try with the debug log Level and paste the log please? :)

@patriziotufarolo
Copy link
Author

patriziotufarolo commented May 16, 2017

Here is the output of
docker run --rm -p 8080:8080 -p 80:80 -t -i -v (pwd)/traefik-rancher.toml:/etc/traefik/traefik.toml traefik:1.3.0-rc2 --logLevel=debug

INFO[2017-05-16T19:01:07Z] Traefik version v1.3.0-rc2 built on 2017-05-16_04:19:54PM
INFO[2017-05-16T19:01:07Z] Using TOML configuration file /etc/traefik/traefik.toml
DEBU[2017-05-16T19:01:07Z] Global configuration loaded {"GraceTimeOut":10000000000,"Debug":false,"CheckNewVersion":true,"AccessLogsFile":"","TraefikLogsFile":"","LogLevel":"debug","EntryPoints":{"http":{"Network":"","Address":":80","TLS":null,"Redirect":null,"Auth":null,"Compress":false}},"Cluster":null,"Constraints":[],"ACME":null,"DefaultEntryPoints":["http"],"ProvidersThrottleDuration":2000000000,"MaxIdleConnsPerHost":200,"IdleTimeout":180000000000,"InsecureSkipVerify":false,"Retry":null,"HealthCheck":{"Interval":30000000000},"Docker":null,"File":null,"Web":{"Address":":8080","CertFile":"","KeyFile":"","ReadOnly":false,"Statistics":null,"Metrics":null,"Path":"","Auth":null},"Marathon":null,"Consul":null,"ConsulCatalog":null,"Etcd":null,"Zookeeper":null,"Boltdb":null,"Kubernetes":null,"Mesos":null,"Eureka":null,"ECS":null,"Rancher":{"Watch":true,"Filename":"","Constraints":null,"Endpoint":"XXX","AccessKey":"XXX","SecretKey":"XXX","ExposedByDefault":true,"Domain":"XXX","RefreshSeconds":15,"EnableServiceHealthFilter":false},"DynamoDB":null}
INFO[2017-05-16T19:01:07Z] Preparing server http &{Network: Address::80 TLS:<nil> Redirect:<nil> Auth:<nil> Compress:false}
INFO[2017-05-16T19:01:07Z] Starting provider *server.WebProvider {"Address":":8080","CertFile":"","KeyFile":"","ReadOnly":false,"Statistics":null,"Metrics":null,"Path":"","Auth":null}
INFO[2017-05-16T19:01:07Z] Starting provider *rancher.Provider {"Watch":true,"Filename":"","Constraints":null,"Endpoint":"XXX","AccessKey":"XXXX","SecretKey":"XXX","ExposedByDefault":true,"Domain":"XXX","RefreshSeconds":15,"EnableServiceHealthFilter":false}
INFO[2017-05-16T19:01:07Z] Starting server on :80
DEBU[2017-05-16T19:01:09Z] first container len: %!i(int=20)
ERRO[2017-05-16T19:01:09Z] Error in Go routine: runtime error: invalid memory address or nil pointer dereference
goroutine 65 [running]:
runtime/debug.Stack(0x76303f, 0xc4200103c0, 0x1fb48d6)
        /usr/local/go/src/runtime/debug/stack.go:24 +0x79
runtime/debug.PrintStack()
        /usr/local/go/src/runtime/debug/stack.go:16 +0x22
github.com/containous/traefik/safe.defaultRecoverGoroutine(0x1ca5580, 0x2e97010)
        /go/src/github.com/containous/traefik/safe/routine.go:148 +0x7d
github.com/containous/traefik/safe.GoWithRecover.func1.1(0x2028be0)
        /go/src/github.com/containous/traefik/safe/routine.go:139 +0x57
panic(0x1ca5580, 0x2e97010)
        /usr/local/go/src/runtime/panic.go:489 +0x2cf
github.com/containous/traefik/provider/rancher.parseRancherData(0xc4201f6800, 0x14, 0x20, 0xc4205cc000, 0x2f, 0x40, 0xc420742d00, 0x14, 0x20, 0x3a4cccfa, ...)
        /go/src/github.com/containous/traefik/provider/rancher/rancher.go:391 +0x2a1
github.com/containous/traefik/provider/rancher.(*Provider).Provide.func1.1(0xc4202da000, 0xc4202e4000)
        /go/src/github.com/containous/traefik/provider/rancher/rancher.go:253 +0x211
github.com/containous/traefik/vendor/github.com/cenk/backoff.RetryNotify(0xc420242020, 0x2d907c0, 0xc4202e4000, 0x2028bc0, 0x0, 0xc420660f98)
        /go/src/github.com/containous/traefik/vendor/github.com/cenk/backoff/retry.go:37 +0x88
github.com/containous/traefik/provider/rancher.(*Provider).Provide.func1()
        /go/src/github.com/containous/traefik/provider/rancher/rancher.go:297 +0x132
github.com/containous/traefik/safe.GoWithRecover.func1(0x2028be0, 0xc420242000)
        /go/src/github.com/containous/traefik/safe/routine.go:142 +0x4d
created by github.com/containous/traefik/safe.GoWithRecover
        /go/src/github.com/containous/traefik/safe/routine.go:143 +0x49

^CINFO[2017-05-16T19:01:11Z] I have to go... interrupt
INFO[2017-05-16T19:01:11Z] Stopping server
DEBU[2017-05-16T19:01:11Z] Waiting 10s seconds before killing connections on entrypoint http...
DEBU[2017-05-16T19:01:11Z] Entrypoint http closed
INFO[2017-05-16T19:01:11Z] Server stopped
INFO[2017-05-16T19:01:11Z] Shutting down
ERRO[2017-05-16T19:01:11Z] Error creating server: http: Server closed

@emilevauge
Copy link
Member

The panic occurs at line for key, value := range service.LaunchConfig.Labels {, @SantoDE it seems either service.LaunchConfig or service.LaunchConfig.Labels is nil. Any idea if the fix is easy?

@emilevauge emilevauge added kind/bug/confirmed a confirmed bug (reproducible). priority/P1 need to be fixed in next release and removed kind/bug/possible a possible bug that needs analysis before it is confirmed or fixed. labels May 16, 2017
@ldez
Copy link
Member

ldez commented May 24, 2017

Closed by #1654.

@ldez ldez closed this as completed May 24, 2017
@ldez ldez removed the priority/P1 need to be fixed in next release label May 30, 2017
@ldez ldez added this to the 1.3 milestone Nov 23, 2017
@traefik traefik locked and limited conversation to collaborators Sep 1, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

5 participants