Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/pprof: add HTTPS support with client certificates #20939

Closed
sorenmat opened this issue Jul 7, 2017 · 11 comments
Closed

cmd/pprof: add HTTPS support with client certificates #20939

sorenmat opened this issue Jul 7, 2017 · 11 comments

Comments

@sorenmat
Copy link

@sorenmat sorenmat commented Jul 7, 2017

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go version go1.8.3 darwin/amd64

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/smo/go"
GORACE=""
GOROOT="/usr/local/Cellar/go/1.8.3/libexec"
GOTOOLDIR="/usr/local/Cellar/go/1.8.3/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/dh/gykpvks552v4h7yfms86x3r80000gn/T/go-build191348200=/tmp/go-build -gno-record-gcc-switches -fno-common"
CXX="clang++"
CGO_ENABLED="1"
PKG_CONFIG="pkg-config"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"

What did you do?

Tried to connect to a HTTPS service in order to pull out pprof data

What did you expect to see?

The pprof data

What did you see instead?

http fetch https://192.168.99.100:32473/debug/pprof/profile?seconds=5: Get https://192.168.99.100:32473/debug/pprof/profile?seconds=5: remote error: tls: bad certificate

I think it would sort of nice to be able to connect via pprof to a server that runs SSL/TLS with client certificates. There
is a work-around this of cause, just put up and proxy that does SSL termination, but that might not always be feasible.
I assume the command would look something like.
go tool pprof -seconds 5 -caert cert.pem -key key.pem -ca.cert ca.pem https://192.168.99.100:32473/debug/pprof/profile

@sorenmat sorenmat changed the title cmd/pprof Should work over HTTPS cmd/pprof Should work over HTTPS with client certificates Jul 7, 2017
@odeke-em odeke-em changed the title cmd/pprof Should work over HTTPS with client certificates cmd/pprof: add HTTPS support with client certificates Jul 7, 2017
@odeke-em odeke-em added the Proposal label Jul 7, 2017
@mvdan
Copy link
Member

@mvdan mvdan commented Jul 7, 2017

Why not download the profile using a program that likely has these features already, like wget or curl, then use the file in go tool pprof instead?

@OneOfOne
Copy link
Contributor

@OneOfOne OneOfOne commented Jul 8, 2017

It is already supported:

$ go tool pprof -seconds 5 https+insecure://192.168.99.100:32473/debug/pprof/profile
@odeke-em
Copy link
Member

@odeke-em odeke-em commented Jul 21, 2017

@OneOfOne do you know if this is documented? If not, could we document this and then we can close the issue, as you've already provided the fix.

@sorenmat
Copy link
Author

@sorenmat sorenmat commented Jul 21, 2017

@OneOfOne not sure that works if you have mutual TLS enabled. But @mvdan had a point, I can download it using curl and then load it with proof.
We can mark this is resolved IMO, just me not thinking 🙄

@OneOfOne
Copy link
Contributor

@OneOfOne OneOfOne commented Jul 21, 2017

It's documented in the go1.8 release notes but I think it should be added to the tool's documentation.

https://tip.golang.org/doc/go1.8

The pprof tool can now profile TLS servers and skip certificate validation by using the “https+insecure” URL scheme.

@odeke-em
Copy link
Member

@odeke-em odeke-em commented Jul 21, 2017

@OneOfOne wanna send a documentation CL? /cc @bradfitz

@bradfitz
Copy link
Contributor

@bradfitz bradfitz commented Jul 21, 2017

Keep in mind that pprof is vendored from an upstream repo, so you can't change some parts of it in Go's repo directly. If you did, it'd just get deleted the next time we imported from the upstream pprof repo.

But I'm not sure which docs you're proposing editing.

https://golang.org/cmd/pprof/ doesn't have much as-is.

@odeke-em
Copy link
Member

@odeke-em odeke-em commented Jul 21, 2017

Ahh I see, thanks for the clarification @bradfitz. I've godoc'd https://github.com/google/pprof and also grep'd for insecure and found that 'https+insecure' is in an internal package for the default fetcher so I no longer believe that documenting this trivial.

@ggaurav10
Copy link

@ggaurav10 ggaurav10 commented Sep 11, 2017

Hi,
I am trying to profile swarm using "go tool pprof"
Below are error messages with different options that I have tried:

# go tool pprof http://127.0.0.1:2376/debug/pprof/trace
Fetching profile over HTTP from http://127.0.0.1:2376/debug/pprof/trace
http://127.0.0.1:2376/debug/pprof/trace: Get http://127.0.0.1:2376/debug/pprof/trace: net/http: HTTP/1.x transport connection broken: malformed HTTP response "\x15\x03\x01\x00\x02\x02"
failed to fetch any profiles

# go tool pprof https+insecure://127.0.0.1:2376/debug/pprof/trace
Fetching profile over HTTP from https+insecure://127.0.0.1:2376/debug/pprof/trace
https+insecure://127.0.0.1:2376/debug/pprof/trace: Get https://127.0.0.1:2376/debug/pprof/trace: remote error: tls: bad certificate
failed to fetch any profiles

Below are the config details:

# go version
go version go1.9 linux/amd64

# /var/vcap/packages/swarm/bin/swarm --version
swarm version 1.2.8 (HEAD)

Swarm is running with debug option enabled, however, the curl ouput at "info" endpoint has the debug set to false:

./bin/swarm --debug=true --log-level=info manage --cluster-driver=swarm --host unix:///var/vcap/sys/run/swarm_manager/swarm_manager.sock --engine-failure-retry 10 --host=0.0.0.0:2376 --strategy=spread --filter=health --filter=port --filter=dependency --filter=affinity --filter=constraint --replication=false --replication-ttl=30s --advertise=xx.xx.xx.xx2376 --heartbeat=20s --api-enable-cors=false --cluster-opt=swarm.overcommit=0.5 --tls=true --tlscacert=/var/vcap/jobs/swarm_manager/config/docker.cacert --tlscert=/var/vcap/jobs/swarm_manager/config/docker.cert --tlskey=/var/vcap/jobs/swarm_manager/config/docker.key --tlsverify=true nodes://yy.yy.yy.yy:4243

#curl -k --key /var/vcap/jobs/swarm_manager/config/docker.key --cert /var/vcap/jobs/swarm_manager/config/docker.cert https://127.0.0.1:2376/info

The output of above cmd has "Debug":false in its output.

Is there anything that I missing here?

@sjug
Copy link

@sjug sjug commented Nov 14, 2017

I've pushed a PR to pprof to add this functionality. google/pprof#261

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Apr 13, 2018

This needs to be fixed upstream, so closing the Go issue.

@golang golang locked and limited conversation to collaborators Apr 13, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
9 participants
You can’t perform that action at this time.