Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Custom error pages doesn't work for a Marathon app #4013

Closed
deric opened this issue Oct 10, 2018 · 4 comments
Closed

Custom error pages doesn't work for a Marathon app #4013

deric opened this issue Oct 10, 2018 · 4 comments
Assignees
Labels
area/provider/marathon area/server kind/bug/possible a possible bug that needs analysis before it is confirmed or fixed. priority/P1 need to be fixed in next release status/5-frozen-due-to-age
Milestone

Comments

@deric
Copy link

deric commented Oct 10, 2018

Do you want to request a feature or report a bug?

Bug

What did you do?

Deployed a Marathon app with following configuration:

traefik.enable: true
traefik.frontend.rule: Host:example.com
traefik.protocol: http
traefik.backend.healthcheck.path: /health
traefik.frontend.entryPoints: http
traefik.backend.loadbalancer.method: drr
traefik.frontend.errors.rapi.query: "/"
traefik.frontend.errors.rapi.backend: -api-json-errors
traefik.frontend.errors.rapi.status: 500-599

furthermore I've deployed added another Marathon app for handling errors.

What did you expect to see?

The main app is scaled to 0 instances, thus there no backend available to handle requests. I'd expect Traefik to handle request with the error handling backend: backend-api-json-errors.

What did you see instead?

time="2018-10-10T19:48:20Z" level=error msg="Caught HTTP Status Code 503, returning error page"
time="2018-10-10T19:48:20Z" level=debug msg="Reporting span 6dd11af5a0cce03d:13a5eb1db3b05976:6dd11af5a0cce03d:1"
time="2018-10-10T19:48:20Z" level=error msg="Recovered from panic in http handler: runtime error: invalid memory address or nil pointer dereference"

Output of traefik version: (What version of Traefik are you using?)

Version:      v1.7.1
Codename:     maroilles
Go version:   go1.11
Built:        2018-09-28_10:24:34PM
OS/Arch:      linux/amd64

What is your environment & configuration (arguments, toml, provider, platform, ...)?

{
 "LifeCycle": {
  "RequestAcceptGraceTimeout": 0,
  "GraceTimeOut": 0
 },
 "GraceTimeOut": 0,
 "Debug": false,
 "CheckNewVersion": true,
 "SendAnonymousUsage": false,
 "AccessLogsFile": "",
 "AccessLog": {
  "file": "/var/log/traefik/traefik.log",
  "format": "common"
 },
 "TraefikLogsFile": "",
 "TraefikLog": null,
 "Tracing": {
  "Backend": "jaeger",
  "ServiceName": "traefik",
  "SpanNameLimit": 0,
  "Jaeger": null,
  "Zipkin": null,
  "DataDog": null
 },
 "LogLevel": "DEBUG",
 "EntryPoints": {
  "api": {
   "Address": "xxxx",
   "TLS": null,
   "Redirect": null,
   "Auth": null,
   "WhitelistSourceRange": null,
   "WhiteList": null,
   "Compress": false,
   "ProxyProtocol": null,
   "ForwardedHeaders": null
  },
  "http": {
   "Address": "xxxx",
   "TLS": null,
   "Redirect": null,
   "Auth": null,
   "WhitelistSourceRange": null,
   "WhiteList": null,
   "Compress": true,
   "ProxyProtocol": null,
   "ForwardedHeaders": null
  },
  "ping": {
   "Address": "xxxx",
   "TLS": null,
   "Redirect": null,
   "Auth": null,
   "WhitelistSourceRange": null,
   "WhiteList": null,
   "Compress": false,
   "ProxyProtocol": null,
   "ForwardedHeaders": null
  }
 },
 "Cluster": null,
 "Constraints": [],
 "ACME": null,
 "DefaultEntryPoints": [
  "http"
 ],
 "ProvidersThrottleDuration": 2000000000,
 "MaxIdleConnsPerHost": 200,
 "IdleTimeout": 0,
 "InsecureSkipVerify": false,
 "RootCAs": null,
 "Retry": {
  "Attempts": 0
 },
 "HealthCheck": {
  "Interval": 30000000000
 },
 "RespondingTimeouts": {
  "ReadTimeout": 0,
  "WriteTimeout": 0,
  "IdleTimeout": 180000000000
 },
 "ForwardingTimeouts": null,
 "AllowMinWeightZero": false,
 "Web": null,
 "Docker": null,
 "File": {
  "Watch": true,
  "Filename": "/etc/traefik/rules.toml",
  "Constraints": null,
  "Trace": false,
  "TemplateVersion": 0,
  "DebugLogGeneratedTemplate": false,
  "Directory": "",
  "TraefikFile": ""
 },
 "Marathon": {
  "Watch": false,
  "Filename": "",
  "Constraints": null,
  "Trace": false,
  "TemplateVersion": 0,
  "DebugLogGeneratedTemplate": false,
  "Endpoint": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
  "Domain": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
  "ExposedByDefault": false,
  "GroupsAsSubDomains": false,
  "DCOSToken": "",
  "MarathonLBCompatibility": false,
  "FilterMarathonConstraints": false,
  "TLS": null,
  "DialerTimeout": 60000000000,
  "ResponseHeaderTimeout": 60000000000,
  "TLSHandshakeTimeout": 5000000000,
  "KeepAlive": 10000000000,
  "ForceTaskHostname": false,
  "Basic": null,
  "RespectReadinessChecks": true
 },
 "Consul": null,
 "ConsulCatalog": null,
 "Etcd": null,
 "Zookeeper": null,
 "Boltdb": null,
 "Kubernetes": null,
 "Mesos": null,
 "Eureka": null,
 "ECS": null,
 "Rancher": null,
 "DynamoDB": null,
 "ServiceFabric": null,
 "Rest": null,
 "API": {
  "EntryPoint": "api",
  "Dashboard": true,
  "Debug": false,
  "CurrentConfigurations": null,
  "Statistics": null,
  "DashboardAssets": null
 },
 "Metrics": {
  "Prometheus": {
   "Buckets": [
    0.1,
    0.3,
    1.2,
    5
   ],
   "EntryPoint": "api"
  },
  "Datadog": null,
  "StatsD": null,
  "InfluxDB": null
 },
 "Ping": {
  "EntryPoint": "ping"
 },
 "HostResolver": null,
 "ConfigFile": "/etc/traefik/traefik.toml"
}
@nmengin nmengin added area/provider/marathon priority/P1 need to be fixed in next release kind/bug/possible a possible bug that needs analysis before it is confirmed or fixed. area/server and removed status/0-needs-triage labels Oct 11, 2018
@deric
Copy link
Author

deric commented Oct 18, 2018

Here's a stacktrace of failing request:

time="2018-10-18T09:33:48Z" level=error msg="Caught HTTP Status Code 503, returning error page"
time="2018-10-18T09:33:48Z" level=error msg="Recovered from panic in HTTP handler [127.0.0.1:38036 - /health]: runtime error: invalid memory address or nil pointer dereference"
time="2018-10-18T09:33:48Z" level=error msg="Stack: goroutine 6859 [running]:\ngithub.com/containous/traefik/middlewares.recoverFunc(0x7fc743112078, 0xc0001ac8d8, 0xc0005fbc00)\n\t/go/src/github.com/containous/traefik/middlewares/recover.go:40 +0x1b7\npanic(0x21d58a0, 0x4530f40)\n\t/usr/local/go/src/runtime/panic.go:513 +0x1b9\ngithub.com/containous/traefik/middlewares.(*Retry).ServeHTTP(0xc000cbf860, 0x7fc743112168, 0xc0001ac960, 0xc000135c00)\n\t/go/src/github.com/containous/traefik/middlewares/retry.go:38 +0x5e8\ngithub.com/containous/traefik/middlewares/tracing.(*HTTPHandlerWrapper).ServeHTTP(0xc000cbf890, 0x7fc743112168, 0xc0001ac960, 0xc000135c00)\n\t/go/src/github.com/containous/traefik/middlewares/tracing/wrapper.go:63 +0xd5\ngithub.com/containous/traefik/middlewares/errorpages.(*Handler).ServeHTTP(0xc0003898f0, 0x7fc743112078, 0xc0001ac920, 0xc000135600, 0xc000943a80)\n\t/go/src/github.com/containous/traefik/middlewares/errorpages/error_pages.go:102 +0x867\ngithub.com/containous/traefik/vendor/github.com/urfave/negroni.middleware.ServeHTTP(0x2935200, 0xc0003898f0, 0xc0010e0960, 0x7fc743112078, 0xc0001ac920, 0xc000135600)\n\t/go/src/github.com/containous/traefik/vendor/github.com/urfave/negroni/negroni.go:33 +0x9c\ngithub.com/containous/traefik/vendor/github.com/urfave/negroni.(*Negroni).ServeHTTP(0xc000cbff20, 0x7fc743112118, 0xc000cd1300, 0xc000135600)\n\t/go/src/github.com/containous/traefik/vendor/github.com/urfave/negroni/negroni.go:81 +0xee\ngithub.com/containous/traefik/vendor/github.com/containous/mux.(*Router).ServeHTTP(0xc0007c8300, 0x7fc743112118, 0xc000cd1300, 0xc000135600)\n\t/go/src/github.com/containous/traefik/vendor/github.com/containous/mux/mux.go:133 +0xf1\ngithub.com/containous/traefik/middlewares.(*HandlerSwitcher).ServeHTTP(0xc00000c008, 0x7fc743112118, 0xc000cd1300, 0xc0005fbf00)\n\t/go/src/github.com/containous/traefik/middlewares/handlerSwitcher.go:24 +0x6f\ngithub.com/containous/traefik/vendor/github.com/urfave/negroni.Wrap.func1(0x7fc743112118, 0xc000cd1300, 0xc0005fbf00, 0xc000943a60)\n\t/go/src/github.com/containous/traefik/vendor/github.com/urfave/negroni/negroni.go:41 +0x4d\ngithub.com/containous/traefik/vendor/github.com/urfave/negroni.HandlerFunc.ServeHTTP(0xc000962600, 0x7fc743112118, 0xc000cd1300, 0xc0005fbf00, 0xc000943a60)\n\t/go/src/github.com/containous/traefik/vendor/github.com/urfave/negroni/negroni.go:24 +0x4e\ngithub.com/containous/traefik/vendor/github.com/urfave/negroni.middleware.ServeHTTP(0x2939e00, 0xc000962600, 0xc000962700, 0x7fc743112118, 0xc000cd1300, 0xc0005fbf00)\n\t/go/src/github.com/containous/traefik/vendor/github.com/urfave/negroni/negroni.go:33 +0x9c\ngithub.com/containous/traefik/vendor/github.com/urfave/negroni.middleware.ServeHTTP-fm(0x7fc743112118, 0xc000cd1300, 0xc0005fbf00)\n\t/go/src/github.com/containous/traefik/vendor/github.com/urfave/negroni/negroni.go:33 +0x60\nnet/http.HandlerFunc.ServeHTTP(0xc000943a40, 0x7fc743112118, 0xc000cd1300, 0xc0005fbf00)\n\t/usr/local/go/src/net/http/server.go:1964 +0x44\ngithub.com/containous/traefik/middlewares.(*RequestHost).ServeHTTP(0x45711d0, 0x7fc743112118, 0xc000cd1300, 0xc0005fbe00, 0xc000943a40)\n\t/go/src/github.com/containous/traefik/middlewares/request_host.go:20 +0x1ec\ngithub.com/containous/traefik/vendor/github.com/urfave/negroni.middleware.ServeHTTP(0x2935020, 0x45711d0, 0xc0009626e0, 0x7fc743112118, 0xc000cd1300, 0xc0005fbe00)\n\t/go/src/github.com/containous/traefik/vendor/github.com/urfave/negroni/negroni.go:33 +0x9c\ngithub.com/containous/traefik/vendor/github.com/urfave/negroni.middleware.ServeHTTP-fm(0x7fc743112118, 0xc000cd1300, 0xc0005fbe00)\n\t/go/src/github.com/containous/traefik/vendor/github.com/urfave/negroni/negroni.go:33 +0x60\nnet/http.HandlerFunc.ServeHTTP(0xc000943a20, 0x7fc743112118, 0xc000cd1300, 0xc0005fbe00)\n\t/usr/local/go/src/net/http/server.go:1964 +0x44\ngithub.com/containous/traefik/vendor/github.com/NYTimes/gziphandler.GzipHandlerWithOpts.func1.1(0x7fc743112118, 0xc000cd1300, 0xc0005fbe00)\n\t/go/src/github.com/containous/traefik/vendor/github.com/NYTimes/gziphandler/gzip.go:292 +0x263\nnet/http.HandlerFunc.ServeHTTP(0xc000a91950, 0x7fc743112118, 0xc000cd1300, 0xc0005fbe00)\n\t/usr/local/go/src/net/http/server.go:1964 +0x44\ngithub.com/containous/traefik/middlewares.(*Compress).ServeHTTP(0x45711d0, 0x7fc743112118, 0xc000cd1300, 0xc0005fbe00, 0xc000943a20)\n\t/go/src/github.com/containous/traefik/middlewares/compress.go:21 +0xe1\ngithub.com/containous/traefik/vendor/github.com/urfave/negroni.middleware.ServeHTTP(0x2934f20, 0x45711d0, 0xc0009626c0, 0x7fc743112118, 0xc000cd1300, 0xc0005fbe00)\n\t/go/src/github.com/containous/traefik/vendor/github.com/urfave/negroni/negroni.go:33 +0x9c\ngithub.com/containous/traefik/vendor/github.com/urfave/negroni.middleware.ServeHTTP-fm(0x7fc743112118, 0xc000cd1300, 0xc0005fbe00)\n\t/go/src/github.com/containous/traefik/vendor/github.com/urfave/negroni/negroni.go:33 +0x60\ngithub.com/containous/traefik/vendor/github.com/thoas/stats.(*Stats).ServeHTTP(0xc00095e120, 0x294c000, 0xc0009439e0, 0xc0005fbe00, 0xc000943a00)\n\t/go/src/github.com/containous/traefik/vendor/github.com/thoas/stats/stats.go:64 +0xc2\ngithub.com/containous/traefik/vendor/github.com/urfave/negroni.middleware.ServeHTTP(0x2936f80, 0xc00095e120, 0xc0009626a0, 0x294c000, 0xc0009439e0, 0xc0005fbe00)\n\t/go/src/github.com/containous/traefik/vendor/github.com/urfave/negroni/negroni.go:33 +0x9c\ngithub.com/containous/traefik/vendor/github.com/urfave/negroni.middleware.ServeHTTP-fm(0x294c000, 0xc0009439e0, 0xc0005fbe00)\n\t/go/src/github.com/containous/traefik/vendor/github.com/urfave/negroni/negroni.go:33 +0x60\ngithub.com/containous/traefik/middlewares.(*metricsMiddleware).ServeHTTP(0xc00096e000, 0x294c040, 0xc000943920, 0xc0005fbe00, 0xc000943940)\n\t/go/src/github.com/containous/traefik/middlewares/metrics.go:66 +0x2fc\ngithub.com/containous/traefik/vendor/github.com/urfave/negroni.middleware.ServeHTTP(0x29350e0, 0xc00096e000, 0xc000962680, 0x294c040, 0xc000943920, 0xc0005fbe00)\n\t/go/src/github.com/containous/traefik/vendor/github.com/urfave/negroni/negroni.go:33 +0x9c\ngithub.com/containous/traefik/vendor/github.com/urfave/negroni.middleware.ServeHTTP-fm(0x294c040, 0xc000943920, 0xc0005fbe00)\n\t/go/src/github.com/containous/traefik/vendor/github.com/urfave/negroni/negroni.go:33 +0x60\nnet/http.HandlerFunc.ServeHTTP(0xc000943860, 0x294c040, 0xc000943920, 0xc0005fbe00)\n\t/usr/local/go/src/net/http/server.go:1964 +0x44\ngithub.com/containous/traefik/middlewares/accesslog.(*LogHandler).ServeHTTP(0xc00087d1d0, 0x7fc7431120e8, 0xc0001ac8f8, 0xc0005fbd00, 0xc000943860)\n\t/go/src/github.com/containous/traefik/middlewares/accesslog/logger.go:181 +0xba9\ngithub.com/containous/traefik/vendor/github.com/urfave/negroni.middleware.ServeHTTP(0x2935120, 0xc00087d1d0, 0xc000962660, 0x7fc7431120e8, 0xc0001ac8f8, 0xc0005fbd00)\n\t/go/src/github.com/containous/traefik/vendor/github.com/urfave/negroni/negroni.go:33 +0x9c\ngithub.com/containous/traefik/vendor/github.com/urfave/negroni.middleware.ServeHTTP-fm(0x7fc7431120e8, 0xc0001ac8f8, 0xc0005fbd00)\n\t/go/src/github.com/containous/traefik/vendor/github.com/urfave/negroni/negroni.go:33 +0x60\ngithub.com/containous/traefik/middlewares/tracing.(*entryPointMiddleware).ServeHTTP(0xc0009620c0, 0x7fc743112078, 0xc0001ac8d8, 0xc0005fbc00, 0xc0009437e0)\n\t/go/src/github.com/containous/traefik/middlewares/tracing/entrypoint.go:36 +0x44e\ngithub.com/containous/traefik/vendor/github.com/urfave/negroni.middleware.ServeHTTP(0x29352a0, 0xc0009620c0, 0xc000962640, 0x7fc743112078, 0xc0001ac8d8, 0xc0005fbc00)\n\t/go/src/github.com/containous/traefik/vendor/github.com/urfave/negroni/negroni.go:33 +0x9c\ngithub.com/containous/traefik/vendor/github.com/urfave/negroni.middleware.ServeHTTP-fm(0x7fc743112078, 0xc0001ac8d8, 0xc0005fbc00)\n\t/go/src/github.com/containous/traefik/vendor/github.com/urfave/negroni/negroni.go:33 +0x60\nnet/http.HandlerFunc.ServeHTTP(0xc0009437c0, 0x7fc743112078, 0xc0001ac8d8, 0xc0005fbc00)\n\t/usr/local/go/src/net/http/server.go:1964 +0x44\ngithub.com/containous/traefik/middlewares.NegroniRecoverHandler.func1(0x7fc743112078, 0xc0001ac8d8, 0xc0005fbc00, 0xc0009437c0)\n\t/go/src/github.com/containous/traefik/middlewares/recover.go:24 +0x87\ngithub.com/containous/traefik/vendor/github.com/urfave/negroni.HandlerFunc.ServeHTTP(0x2702070, 0x7fc743112078, 0xc0001ac8d8, 0xc0005fbc00, 0xc0009437c0)\n\t/go/src/github.com/containous/traefik/vendor/github.com/urfave/negroni/negroni.go:24 +0x4e\ngithub.com/containous/traefik/vendor/github.com/urfave/negroni.middleware.ServeHTTP(0x2939e00, 0x2702070, 0xc000962620, 0x7fc743112078, 0xc0001ac8d8, 0xc0005fbc00)\n\t/go/src/github.com/containous/traefik/vendor/github.com/urfave/negroni/negroni.go:33 +0x9c\ngithub.com/containous/traefik/vendor/github.com/urfave/negroni.(*Negroni).ServeHTTP(0xc0003064b0, 0x295bcc0, 0xc00108cee0, 0xc0005fbc00)\n\t/go/src/github.com/containous/traefik/vendor/github.com/urfave/negroni/negroni.go:81 +0xee\ngithub.com/containous/traefik/vendor/github.com/containous/mux.(*Router).ServeHTTP(0xc00095e1e0, 0x295bcc0, 0xc00108cee0, 0xc0005fbc00)\n\t/go/src/github.com/containous/traefik/vendor/github.com/containous/mux/mux.go:133 +0xf1\ngithub.com/containous/traefik/h2c.Server.Serve.func1(0x295bcc0, 0xc00108cee0, 0xc0005fb600)\n\t/go/src/github.com/containous/traefik/h2c/h2c.go:78 +0x409\nnet/http.HandlerFunc.ServeHTTP(0xc000978020, 0x295bcc0, 0xc00108cee0, 0xc0005fb600)\n\t/usr/local/go/src/net/http/server.go:1964 +0x44\nnet/http.serverHandler.ServeHTTP(0xc0009680d0, 0x295bcc0, 0xc00108cee0, 0xc0005fb600)\n\t/usr/local/go/src/net/http/server.go:2741 +0xab\nnet/http.(*conn).serve(0xc00067f360, 0x295e140, 0xc000cd1040)\n\t/usr/local/go/src/net/http/server.go:1847 +0x646\ncreated by net/http.(*Server).Serve\n\t/usr/local/go/src/net/http/server.go:2851 +0x2f5\n"
time="2018-10-18T09:34:40Z" level=info msg="Skipping same configuration for provider marathon"

@ldez ldez self-assigned this Oct 18, 2018
@ldez
Copy link
Member

ldez commented Oct 18, 2018

Could you try this image ldez/traefik:retry

@deric
Copy link
Author

deric commented Oct 18, 2018

With the provided image it seems to be working:

time="2018-10-18T22:09:59Z" level=error msg="Caught HTTP Status Code 503, returning error page"

@traefiker
Copy link
Contributor

Closed by #4075.

@traefiker traefiker added this to the 1.7 milestone Oct 23, 2018
@traefik traefik locked and limited conversation to collaborators Sep 1, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area/provider/marathon area/server kind/bug/possible a possible bug that needs analysis before it is confirmed or fixed. priority/P1 need to be fixed in next release status/5-frozen-due-to-age
Projects
None yet
Development

No branches or pull requests

5 participants