Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net/http/httptrace: expose transport-added request headers #19761

Closed
cobls opened this issue Mar 29, 2017 · 11 comments
Closed

net/http/httptrace: expose transport-added request headers #19761

cobls opened this issue Mar 29, 2017 · 11 comments

Comments

@cobls
Copy link

@cobls cobls commented Mar 29, 2017

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

1.8

I'd like to record request headers to visualise in an http polling application. There are headers set on the underlying transport request such as compression or keepalives that I am not able to view via the Response.Request.Headers or in any other way I can see.

@bradfitz
Copy link
Contributor

@bradfitz bradfitz commented Mar 29, 2017

Have you tried https://golang.org/pkg/net/http/httptrace/#ClientTrace?

See https://blog.golang.org/http-tracing

Do you have a concrete suggestion for what's missing in the httptrace package?

@bradfitz
Copy link
Contributor

@bradfitz bradfitz commented Mar 29, 2017

@bradfitz bradfitz changed the title Feature: Expose transport request headers to Request.Headers net/http/httptrace: expose transport-added request headers Mar 29, 2017
@cobls
Copy link
Author

@cobls cobls commented Mar 30, 2017

We could use httptrace if the transport request was sent into WroteRequest similar to how conn is passed into GotConn. However I don't think we should have to look into the trace to find the complete set of request headers.

I'd love it if Response.Request.Header included the extra headers set in transport.

I can create a code snippet if I'm not explaining this well, however my code will likely confuse you more!

@bradfitz
Copy link
Contributor

@bradfitz bradfitz commented Mar 30, 2017

I'd love it if Response.Request.Header included the extra headers set in transport.

That's not an option. We can't mutate that. It would break existing code, and possibly introduce data races. It's a backwards incompatible change.

First, let's make a list of the complete set of headers which can be added implicitly.

Only "Accept-Encoding" and "Connection", or more? And what do you need them for?

@cobls
Copy link
Author

@cobls cobls commented Mar 30, 2017

We have an internal application that polls a great many services over http for service health. When we detect a failure we capture as much information as possible to help diagnose the failure. Since we give users the option to set custom headers in their tests we provide them what we thought was a complete list of headers as part of the failure attributes.

Recently we had an issue using httptracer on an http/2 connection (where we did not realise we were falling through to http/2) and went down the path of diagnosing why we were re-using connections when we had explicitly disabled keepalives. The service receiving these polls dumped out the request including headers which included Accept-Encoding and Connection. If these were exposed as part of our failure we'd have had more insight.

I understand we can't mutate Request, however transport is not sending the same request as the user sets by adding these headers in flight. I'd be OK with using httptracer to expose this if possible though the purist in me thinks the response should include the complete request sent.

@tombergan
Copy link
Contributor

@tombergan tombergan commented Aug 10, 2017

This is a reasonable feature request.

We could use httptrace if the transport request was sent into WroteRequest similar to how conn is passed into GotConn. However I don't think we should have to look into the trace to find the complete set of request headers.

I think httptrace is the best place for this, actually. It's not possible to mutate Request for the reasons that Brad mentioned. It's unfortunate that WroteHeaders() doesn't take an Info argument, and it's too late to change the signature. The possible solutions include:

  1. Add Header to WroteRequestInfo. Might require copying the Header (I'd have to look).

  2. Add WroteHeaderField(key, value). Would be called before WroteHeaders. Does not require copying the Header, but is slightly annoying to use because it's called once for each key/value pair.

  3. Some combination of the above that only exports headers that are actually set or changed by the HTTP library.

@tombergan tombergan added this to the Go1.10 milestone Aug 10, 2017
@gopherbot
Copy link

@gopherbot gopherbot commented Oct 1, 2017

Change https://golang.org/cl/67430 mentions this issue: net/http/httptrace: expose transport-added request headers for http/1.1

@odeke-em
Copy link
Member

@odeke-em odeke-em commented Dec 11, 2017

Gently pinging the issue here: @tombergan any ideas for this issue? -- as I can see that the CL has an unresolved -2 by the original author.

@bradfitz bradfitz modified the milestones: Go1.10, Go1.11 Jan 5, 2018
@bradfitz bradfitz added the NeedsFix label Jan 5, 2018
gopherbot pushed a commit that referenced this issue Jun 27, 2018
Some headers, which are set or modified by the http library,
are not written to the standard http.Request.Header and are
not included as part of http.Response.Request.Header.

Exposing all headers alleviates this problem.

This is not a complete solution to 19761 since it does not have http/2 support.

Updates #19761

Change-Id: Ie8d4f702f4f671666b120b332378644f094e288b
Reviewed-on: https://go-review.googlesource.com/67430
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
@gopherbot
Copy link

@gopherbot gopherbot commented Jun 27, 2018

Change https://golang.org/cl/121236 mentions this issue: net/http/httptrace: add clarification never added to CL 67430

gopherbot pushed a commit that referenced this issue Jun 27, 2018
Updates #19761

Change-Id: Iac3bd4c40002f8e348452b50bff54dee3210d447
Reviewed-on: https://go-review.googlesource.com/121236
Reviewed-by: Ian Lance Taylor <iant@golang.org>
@gopherbot
Copy link

@gopherbot gopherbot commented Jul 9, 2018

Change https://golang.org/cl/122816 mentions this issue: http2: fire httptrace.ClientTrace.WroteHeaderField if set

@bradfitz bradfitz self-assigned this Jul 9, 2018
gopherbot pushed a commit to golang/net that referenced this issue Jul 9, 2018
ClientTrace.WroteHeaderField was added in Go 1.11.

Updates golang/go#19761 (fixes after vendor into std)

Change-Id: I9a7af31b8601b9cd6efdee63d31a6c05102473d2
Reviewed-on: https://go-review.googlesource.com/122816
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Andrew Bonventre <andybons@golang.org>
@gopherbot
Copy link

@gopherbot gopherbot commented Jul 9, 2018

Change https://golang.org/cl/122591 mentions this issue: net/http: update bundled http2

@gopherbot gopherbot closed this in 194bbe8 Jul 9, 2018
@golang golang locked and limited conversation to collaborators Jul 9, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants
You can’t perform that action at this time.