Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: crypto/tls: add (*tls.Conn).HandshakeContext and add context to ClientHelloInfo, CertificateRequestInfo #32406

Open
johanbrandhorst opened this issue Jun 3, 2019 · 29 comments

Comments

@johanbrandhorst
Copy link
Member

@johanbrandhorst johanbrandhorst commented Jun 3, 2019

Proposal

I propose an unexported context.Context field is added to the ClientHelloInfo and CertificateRequestInfo crypto/tls types. We should also add a Context() context.Context method to these types to access this context. Further, we should add a new method, HandshakeContext(context.Context) error to the tls.Conn struct, which will be used to propagate a context down the handshake call stack. The existing Handshake() error would call to the new method with a context.Background() context.

Standard library uses of (*tls.Conn).Handshake() should be moved over to the new method, where appropriate. For example, it is not clear it is appropriate to change the existing Read and Write methods on the *tls.Conn to use the new handshake method, but in (*net/http.Server).serve, it is clear that moving to the new function would enhance the request lifetime control in the function.

Motivation

In recent Go releases, we've been able to use the handy GetCertificate and GetClientCertificate methods of the *tls.Config to dynamically control certificate management in Go apps. This is fantastic, and has lead to things like https://godoc.org/golang.org/x/crypto/acme/autocert and https://github.com/johanbrandhorst/certify which are somewhat unique to the Go ecosystem.

Unfortunately, one glaring omission from the API is a connection context for cancellation and request scoped variable propagation. This means users have to implement custom timeouts or block their TLS connections forever in case of problems. It also means powerful tools like tracing and metrics that make use of the context cannot be used.

Interaction with net/http

net/http.Server provide BaseContext, which is used to set a global context for the duration of (*http.Server).Serve, and ConnContext, which is used on every new connection. The context passed to (*tls.Conn).HandshakeContext would necessarily be a child of these contexts, as the existing Handshake call is made after these contexts are created. See

@agnivade agnivade changed the title crypto/tls: add request context to ClientHelloInfo and CertificateRequestInfo proposal: crypto/tls: add request context to ClientHelloInfo and CertificateRequestInfo Jun 3, 2019
@gopherbot gopherbot added this to the Proposal milestone Jun 3, 2019
@gopherbot gopherbot added the Proposal label Jun 3, 2019
@FiloSottile
Copy link
Member

@FiloSottile FiloSottile commented Jun 4, 2019

It would help if you could elaborate on the various use cases: what you are trying to do in each situation, what doesn't work at the moment, and how a context would help.

I've in the past wanted to surface details of the ClientHelloInfo to net/http Handlers, so I can see the use case, but I'd like to build a generic solution.

@johanbrandhorst
Copy link
Member Author

@johanbrandhorst johanbrandhorst commented Jun 4, 2019

My use case specifically is to allow my library (certify) to cancel its outgoing requests if the incoming connection is closed. Additionally, it would allow detailed tracing to capture the latency cost of dynamically provisioned TLS certificates, something that is currently hidden inside the TLS handshake time in the standard library. Simply having a context associated with the underlying connection that could be used in outgoing net/http requests would be enough.

@bradfitz
Copy link
Contributor

@bradfitz bradfitz commented Jun 4, 2019

@johanbrandhorst, sounds like good reasons. For the same reason we didn't add a Context struct field to net/http.Request and used a method instead, we should instead add a Context method to crypto/tls.ClientHelloInfo.

@johanbrandhorst
Copy link
Member Author

@johanbrandhorst johanbrandhorst commented Jun 4, 2019

OK, I will attempt to implement this.

@johanbrandhorst
Copy link
Member Author

@johanbrandhorst johanbrandhorst commented Jun 4, 2019

Do the tags need updating?

@gopherbot
Copy link

@gopherbot gopherbot commented Jun 6, 2019

Change https://golang.org/cl/181097 mentions this issue: crypto/tls, net/http: add context to tls structs

@johanbrandhorst
Copy link
Member Author

@johanbrandhorst johanbrandhorst commented Sep 4, 2019

Since the tree has recently opened again, bumping this for another look at the initial implementation. Could we tag this with 1.14? I think this proposal was accepted in #32406 (comment).

@mvdan
Copy link
Member

@mvdan mvdan commented Sep 17, 2019

@bradfitz @FiloSottile could you please clarify whether this proposal is accepted? If so, we can milestone it for 1.14 and review the CL.

@bradfitz
Copy link
Contributor

@bradfitz bradfitz commented Sep 17, 2019

Looks like it's stuck in the Crypto Proposal Review queue. @FiloSottile owns that meeting.

@johanbrandhorst
Copy link
Member Author

@johanbrandhorst johanbrandhorst commented Sep 30, 2019

Friendly ping on this. @FiloSottile is there an update on this?

@johanbrandhorst
Copy link
Member Author

@johanbrandhorst johanbrandhorst commented Oct 14, 2019

Friendly ping.

@johanbrandhorst
Copy link
Member Author

@johanbrandhorst johanbrandhorst commented Oct 28, 2019

Friendly ping. @FiloSottile anything I can do to help with the Crypto Proposal Review queue?

@johanbrandhorst
Copy link
Member Author

@johanbrandhorst johanbrandhorst commented Nov 11, 2019

Friendliest of bumps.

@johanbrandhorst
Copy link
Member Author

@johanbrandhorst johanbrandhorst commented Apr 15, 2020

With 1.14 out, is there a plan to review the crypto proposals?

@ianlancetaylor ianlancetaylor added this to Incoming in Proposals Apr 15, 2020
@rsc
Copy link
Contributor

@rsc rsc commented May 20, 2020

@johanbrandhorst, crypto proposal review is proceeding fairly well (see all the crypto proposals in the minutes at https://golang.org/s/proposal-minutes), but there are many.

@rsc
Copy link
Contributor

@rsc rsc commented Jun 3, 2020

@rsc rsc moved this from Incoming to Active in Proposals Jun 3, 2020
@FiloSottile
Copy link
Member

@FiloSottile FiloSottile commented Jun 3, 2020

Hey, sorry for the delay, this one fell off my radar.

Another common problem that would be nice to solve at the same time is propagating info from the callbacks to net/http handlers. Maybe exposing this context also from ConnectionState would work, as it's exposed as Request.TLS? Or should the net/http Request context be a child of the TLS one? (How would that work with Server.BaseContext and Server.ConnContext?)

Another reason to expose it on ConnectionState is that the new VerifyConnection callback gets a ConnectionState as input. All other callbacks should be covered by ClientHelloInfo and CertificateRequestInfo.

Can we get a full API proposal here on the issue, along with details of how it would interact with net/http?

@johanbrandhorst johanbrandhorst changed the title proposal: crypto/tls: add request context to ClientHelloInfo and CertificateRequestInfo proposal: crypto/tls: add request context to ClientHelloInfo, CertificateRequestInfo and ConnectionState Jun 4, 2020
@johanbrandhorst
Copy link
Member Author

@johanbrandhorst johanbrandhorst commented Jun 4, 2020

Thanks for looping back on this @FiloSottile, I've updated the first post with a full API proposal. It does not support passing data back via the context as it is read only. We would need to make the context an exported field to support this, which was argued against in #32406 (comment). I'm open to discuss other solutions, or changing the context to be an exported field.

What are your thoughts?

@FiloSottile
Copy link
Member

@FiloSottile FiloSottile commented Jun 4, 2020

If the TLS context needs to be a child of the ConnContext, how does that happen? I can't see a way for crypto/tls to reach the ConnContext, nor for net/http to set the TLS context.

I unfortunately haven't designed a lot of APIs with Context, so I could use some advice (maybe from @bcmills?) on this proposal.

@johanbrandhorst
Copy link
Member Author

@johanbrandhorst johanbrandhorst commented Jun 4, 2020

I'm not an expert on the net/http server by any stretch, but I thought we could assign to it here:

if tlsConn, ok := c.rwc.(*tls.Conn); ok {
, where we have both the ctx inherited from ConnContext and the *tls.Conn.

@FiloSottile
Copy link
Member

@FiloSottile FiloSottile commented Jun 4, 2020

Assign it how? The proposal does not expose any way to set the Context.

@johanbrandhorst
Copy link
Member Author

@johanbrandhorst johanbrandhorst commented Jun 4, 2020

Ah, good point, we would have to expose something like a WithContext to assign a context to the conn.

I will update the proposal.

@johanbrandhorst johanbrandhorst changed the title proposal: crypto/tls: add request context to ClientHelloInfo, CertificateRequestInfo and ConnectionState proposal: crypto/tls: add request context to Conn, ClientHelloInfo, CertificateRequestInfo and ConnectionState Jun 4, 2020
@johanbrandhorst
Copy link
Member Author

@johanbrandhorst johanbrandhorst commented Jun 4, 2020

I've added the (*crypto/tls.Conn).WithContext method description to the proposal.

@rsc
Copy link
Contributor

@rsc rsc commented Jul 15, 2020

ping @FiloSottile and @katiehockman

@rsc
Copy link
Contributor

@rsc rsc commented Jul 28, 2020

Talked to @FiloSottile.

http.Request.WithContext returns a derived http.Request that now has the context. In the callback you are trying to use, there's no way to return a new tls.Conn, so the tls.Conn.WithContext is a setter - it mutates the receiver instead of returning a derived copy.

So at least in this proposal, WithContext should be named SetContext.

But then the problem is this is the first SetContext we have in the standard library, and it's unclear that's the right path to go down.

/cc @bcmills @Sajmani

@johanbrandhorst
Copy link
Member Author

@johanbrandhorst johanbrandhorst commented Jul 29, 2020

I agree with your analysis and that SetContext would set a potentially dangerous precedent. I will re-examine the problem and see if there's a better way of merging the HTTP context into the tls connection.

@johanbrandhorst
Copy link
Member Author

@johanbrandhorst johanbrandhorst commented Jul 30, 2020

I've updated the proposal to change the use of a SetContext method to instead create a new HandshakeContext method on the tls.Conn. This method would be used to propagate the context down the call stack into the GetClientCertificate and GetCertificate callbacks. PTAL.

@johanbrandhorst johanbrandhorst changed the title proposal: crypto/tls: add request context to Conn, ClientHelloInfo, CertificateRequestInfo and ConnectionState proposal: crypto/tls: add (*tls.Conn).HandshakeContext and add context to ClientHelloInfo, CertificateRequestInfo Aug 1, 2020
@gopherbot
Copy link

@gopherbot gopherbot commented Aug 1, 2020

Change https://golang.org/cl/246338 mentions this issue: DO NOT REVIEW: crypto/tls: add HandshakeContext method to Conn

@johanbrandhorst
Copy link
Member Author

@johanbrandhorst johanbrandhorst commented Aug 1, 2020

I took a stab at what this would look like here: https://go-review.googlesource.com/c/go/+/246338. Forgive me if it's inappropriate to make this sort of experimental change before a proposal has been accepted.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
7 participants
You can’t perform that action at this time.