Skip to content

/prometheus

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

/debug/pprof endpoints return 404 when using -web.route-prefix #2183

Closed
mberhault opened this Issue Nov 13, 2016 · 2 comments

Comments

Assignees
No one assigned
Labels
component/ui help wanted kind/bug low hanging fruit priority/P2
Projects
None yet
Milestone
No milestone
4 participants
@mberhault @mark-adams @beorn7 @brian-brazil
@mberhault
Copy link

mberhault commented Nov 13, 2016

What did you do?
Running prometheus with -web.external-url=http://<myhost>:9090/prometheus/, also tried with just --web.route-prefix=/prometheus
Normal endpoints work fine (everything prepended with /prometheus/).
Attempted to hit the /debug/pprof/ endpoints (from net/http/pprof) when trying to provide more information for #1873.

What did you expect to see?
404 for all pprof endpoints. Tried both with and without the prefix:
/debug/pprof
/prometheus/debug/pprof
Without the --web.route-prefix flag, the default enpoints work just fine.

What did you see instead? Under which circumstances?
I would expect working /debug/pprof endpoints since those get automatically registered through the handler.

Environment
I first noticed this in production, running with -web.external-url since we're sitting behind oauth2_proxy. This was easily reproducible locally.

  • System information:
    Darwin 14.3.0 x86_64
    Linux 3.16.0-4-amd64 x86_64

  • Prometheus version:
    prometheus, version 1.2.3 (branch: master, revision: c1eee5b)
    build user: root@d8eb84e17a12
    build date: 20161103-21:46:57
    go version: go1.7.3

  • Alertmanager version:
    N/A

  • Prometheus configuration file:
    N/A

  • Alertmanager configuration file:
    N/A

  • Logs:
    Absolutely nothing shows up in the logs:

$ ~/Downloads/prometheus-1.2.3.darwin-amd64/prometheus -config.file=prometheus-testing.yml --web.route-prefix=/prometheus --log.level=debug
INFO[0000] Starting prometheus (version=1.2.3, branch=master, revision=c1eee5b0da2540b9dfd2f70752015b0fce83b616)  source=main.go:75
INFO[0000] Build context (go=go1.7.3, user=root@d8eb84e17a12, date=20161103-21:46:57)  source=main.go:76
INFO[0000] Loading configuration file prometheus-testing.yml  source=main.go:247
INFO[0000] Loading series map and head chunks...         source=storage.go:354
INFO[0000] 7126 series loaded.                           source=storage.go:359
WARN[0000] No AlertManagers configured, not dispatching any alerts  source=notifier.go:176
INFO[0000] Starting target manager...                    source=targetmanager.go:76
INFO[0000] Listening on :9090                            source=web.go:240
WARN[0006] Received SIGTERM, exiting gracefully...       source=main.go:229
INFO[0006] See you next time!                            source=main.go:236
...

@beorn7 beorn7 added the kind/bug label Nov 14, 2016

@brian-brazil brian-brazil added priority/P2 low hanging fruit component/ui help wanted and removed help wanted labels Jul 14, 2017

@mark-adams

This comment has been minimized.

Sign in to view
Copy link
Contributor

mark-adams commented Aug 10, 2017

I'd love to take a shot at this one.

@mark-adams mark-adams referenced this issue Aug 10, 2017

Merged

Fix pprof endpoints when -web.route-prefix or -web.external-url is used #3054

mark-adams added a commit to mark-adams/prometheus that referenced this issue Aug 19, 2017

@mark-adams
Fix pprof endpoints when -web.route-prefix or -web.external-url is used 
Whenever a route prefix is applied, the router prepends the prefix to
the URL path on the request. For most handlers, this is not an issue
because the request's path is only used for routing and is not actually
needed by the handler itself. However, Prometheus delegates the handling
of the /debug/* endpoints to the http.DefaultServeMux which has it's own
routing logic that depends on the url.Path. As a result, whenever a
prefix is applied, the prefixed URL is passed to the DefaultServeMux
which has no awareness of the prefix and returns a 404.

This change fixes the issue by creating a new serveDebug handler which
routes requests /debug/* requests to appropriate net/http/pprof handler
and removing the net/http/pprof import in cmd/prometheus since it is no
longer necessary.

Fixes prometheus#2183.
9eedb54

@juliusv juliusv closed this in #3054 Aug 22, 2017

juliusv added a commit that referenced this issue Aug 22, 2017

@mark-adams @juliusv
Fix pprof endpoints when -web.route-prefix or -web.external-url is us… 
…ed (#3054)

Whenever a route prefix is applied, the router prepends the prefix to
the URL path on the request. For most handlers, this is not an issue
because the request's path is only used for routing and is not actually
needed by the handler itself. However, Prometheus delegates the handling
of the /debug/* endpoints to the http.DefaultServeMux which has it's own
routing logic that depends on the url.Path. As a result, whenever a
prefix is applied, the prefixed URL is passed to the DefaultServeMux
which has no awareness of the prefix and returns a 404.

This change fixes the issue by creating a new serveDebug handler which
routes requests /debug/* requests to appropriate net/http/pprof handler
and removing the net/http/pprof import in cmd/prometheus since it is no
longer necessary.

Fixes #2183.
Loading status checks…
77c816b
@lock

This comment has been minimized.

Sign in to view
Copy link

lock bot commented Mar 23, 2019

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@lock lock bot locked and limited conversation to collaborators Mar 23, 2019

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
You can’t perform that action at this time.