https proxy example please

[suntong]

I'm trying to learn to use goproxy through the example https://github.com/elazarl/goproxy/tree/master/examples/goproxy-httpdump.

I found that it can only handle http requests, not https'. Could you provide an example that can handle https requests please? (As almost all web servers are moving away from http to https now, e.g., github, wikipedia, wordpress, etc).

PS, by "handle https requests" I meant that none of the https requests are dumped by the current goproxy-httpdump program.

Thanks

[suntong]

Suppose tracking https requests is impossible (or too difficult), then my next question is how to output those "Tunnel to" requests from goproxy-httpdump?

I.e., each https request is lead by a normal HTTP request of type "Tunnel to", which has the following headers:

CONNECT csi.gstatic.com:443 HTTP/1.1
Host: csi.gstatic.com:443
Connection: keep-alive
User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.116 Safari/537.36

A SSLv3-compatible ClientHello handshake was found. Fiddler extracted the parameters below.

Version: 3.3 (TLS/1.2)
Random: ...

I need to dump those requests as well from goproxy-httpdump.

Thanks

[sundy-li]

+ 1 

[https://github.com/elazarl/goproxy/blob/master/examples/goproxy-transparent/transparent.go]
This example is not very well...

[elazarl]

What about this example:
https://github.com/elazarl/goproxy/blob/master/examples/goproxy-eavesdropper/main.go

[suntong]

The problem is that I found goproxy can only handle http requests, not https'.
That was the question, and I am afraid that I wasn't able to find the answer from the goproxy-eavesdropper example. Maybe I'm wrong about that.

[fredcy]

I've built a simple HTTPS logging proxy on top of goproxy, at https://github.com/fredcy/proxy

[rahulwa]

I guess if you add this to your main function, it should work. It will do man-in-the-middle attacks against HTTPS.

proxy.OnRequest(goproxy.ReqHostMatches(regexp.MustCompile("^.*:443$"))).
	HandleConnectFunc(func(host string, ctx *goproxy.ProxyCtx) (*goproxy.ConnectAction, string) {
		return goproxy.MitmConnect, host
	})

[fredcy]

That's pretty much what I do in my logging proxy.
https://github.com/fredcy/proxy/blob/73914b909caf97d0f92a0d9b075c3fda33197b0f/main.go#L109-L110

[suntong]

Thanks a lot @fredcy & @rahulwa, I'll look into it this weekend and get back to you...

[suntong]

This, in deed, is doing man-in-the-middle attacks against HTTPS, which is not allowed by default.

The fiddler circumvents it by installing its own Root Certificate into the system (on Windows), @fredcy, how did you solve the problem? Thx.

[fredcy]

I imported the CA certificate embedded in goproxy into the Keystore on my MacOS client and marked it as trusted. With that done my Mac client uses the proxy without any complaints or warnings about cert problems.

The needed certificate is contained in the certs.go file. At the time of writing it's here:

goproxy/certs.go

Lines 24 to 57 in 4327d5f

	var CA_CERT = []byte(`-----BEGIN CERTIFICATE-----
	MIIF9DCCA9ygAwIBAgIJAODqYUwoVjJkMA0GCSqGSIb3DQEBCwUAMIGOMQswCQYD
	VQQGEwJJTDEPMA0GA1UECAwGQ2VudGVyMQwwCgYDVQQHDANMb2QxEDAOBgNVBAoM
	B0dvUHJveHkxEDAOBgNVBAsMB0dvUHJveHkxGjAYBgNVBAMMEWdvcHJveHkuZ2l0
	aHViLmlvMSAwHgYJKoZIhvcNAQkBFhFlbGF6YXJsQGdtYWlsLmNvbTAeFw0xNzA0
	MDUyMDAwMTBaFw0zNzAzMzEyMDAwMTBaMIGOMQswCQYDVQQGEwJJTDEPMA0GA1UE
	CAwGQ2VudGVyMQwwCgYDVQQHDANMb2QxEDAOBgNVBAoMB0dvUHJveHkxEDAOBgNV
	BAsMB0dvUHJveHkxGjAYBgNVBAMMEWdvcHJveHkuZ2l0aHViLmlvMSAwHgYJKoZI
	hvcNAQkBFhFlbGF6YXJsQGdtYWlsLmNvbTCCAiIwDQYJKoZIhvcNAQEBBQADggIP
	ADCCAgoCggIBAJ4Qy+H6hhoY1s0QRcvIhxrjSHaO/RbaFj3rwqcnpOgFq07gRdI9
	3c0TFKQJHpgv6feLRhEvX/YllFYu4J35lM9ZcYY4qlKFuStcX8Jm8fqpgtmAMBzP
	sqtqDi8M9RQGKENzU9IFOnCV7SAeh45scMuI3wz8wrjBcH7zquHkvqUSYZz035t9
	V6WTrHyTEvT4w+lFOVN2bA/6DAIxrjBiF6DhoJqnha0SZtDfv77XpwGG3EhA/qoh
	hiYrDruYK7zJdESQL44LwzMPupVigqalfv+YHfQjbhT951IVurW2NJgRyBE62dLr
	lHYdtT9tCTCrd+KJNMJ+jp9hAjdIu1Br/kifU4F4+4ZLMR9Ueji0GkkPKsYdyMnq
	j0p0PogyvP1l4qmboPImMYtaoFuYmMYlebgC9LN10bL91K4+jLt0I1YntEzrqgJo
	WsJztYDw543NzSy5W+/cq4XRYgtq1b0RWwuUiswezmMoeyHZ8BQJe2xMjAOllASD
	fqa8OK3WABHJpy4zUrnUBiMuPITzD/FuDx4C5IwwlC68gHAZblNqpBZCX0nFCtKj
	YOcI2So5HbQ2OC8QF+zGVuduHUSok4hSy2BBfZ1pfvziqBeetWJwFvapGB44nIHh
	WKNKvqOxLNIy7e+TGRiWOomrAWM18VSR9LZbBxpJK7PLSzWqYJYTRCZHAgMBAAGj
	UzBRMB0GA1UdDgQWBBR4uDD9Y6x7iUoHO+32ioOcw1ICZTAfBgNVHSMEGDAWgBR4
	uDD9Y6x7iUoHO+32ioOcw1ICZTAPBgNVHRMBAf8EBTADAQH/MA0GCSqGSIb3DQEB
	CwUAA4ICAQAaCEupzGGqcdh+L7BzhX7zyd7yzAKUoLxFrxaZY34Xyj3lcx1XoK6F
	AqsH2JM25GixgadzhNt92JP7vzoWeHZtLfstrPS638Y1zZi6toy4E49viYjFk5J0
	C6ZcFC04VYWWx6z0HwJuAS08tZ37JuFXpJGfXJOjZCQyxse0Lg0tuKLMeXDCk2Y3
	Ba0noeuNyHRoWXXPyiUoeApkVCU5gIsyiJSWOjhJ5hpJG06rQNfNYexgKrrraEin
	o0jmEMtJMx5TtD83hSnLCnFGBBq5lkE7jgXME1KsbIE3lJZzRX1mQwUK8CJDYxye
	i6M/dzSvy0SsPvz8fTAlprXRtWWtJQmxgWENp3Dv+0Pmux/l+ilk7KA4sMXGhsfr
	bvTOeWl1/uoFTPYiWR/ww7QEPLq23yDFY04Q7Un0qjIk8ExvaY8lCkXMgc8i7sGY
	VfvOYb0zm67EfAQl3TW8Ky5fl5CcxpVCD360Bzi6hwjYixa3qEeBggOixFQBFWft
	8wrkKTHpOQXjn4sDPtet8imm9UYEtzWrFX6T9MFYkBR0/yye0FIh9+YPiTA6WB86
	NCNwK5Yl6HuvF97CIH5CdgO+5C7KifUtqTOL8pQKbNwy0S3sNYvB+njGvRpR7pKV
	BUnFpB/Atptqr4CUlTXrc5IPLAqAfmwk5IKcwy3EXUbruf9Dwz69YA==
	-----END CERTIFICATE-----`)

I think the ca.pem file is supposed to be exactly that cert but it's out of date. I opened #216 about that.

[elazarl]

You are correct. CA.pem out of date. Mind sending a PR to fix that?

…

On Thu, Apr 13, 2017, 5:15 PM Fred Yankowski ***@***.***> wrote: I imported the CA certificate embedded in goproxy into the Keystore on my MacOS client and marked it as trusted. With that done my Mac client uses the proxy without any complaints or warnings about cert problems. The needed certificate is contained in the certs.go file. At the time of writing it's here: https://github.com/elazarl/goproxy/blob/4327d5f85a6da046b9b4818382a3e51f795a249b/certs.go#L24-L57 I think the ca.pem file is supposed to be exactly that cert but it's out of date. I opened #216 <#216> about that. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#192 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAP4ogwo-heMiAv0svNPntYOOOb9dAPcks5rvi3mgaJpZM4KUFEf> .

[fredcy]

OK. #218

[suntong]

I spent time working on something else, but
looks good. 👍