Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SPDY is deprecated. Switch to HTTP/2. #7452

Open
lavalamp opened this issue Apr 28, 2015 · 37 comments

Comments

@lavalamp
Copy link
Member

@lavalamp lavalamp commented Apr 28, 2015

Filing for tracking purposes.

See #7392 where the spdy package we use doesn't even exist any more. (And btw it has bugs anyway, possibly security bugs-- see the trophies here: https://github.com/dvyukov/go-fuzz)

Blocked on github.com/docker/spdystream switching to HTTP/2.

(Note: HTTP/2 is based on SPDY, with just a few differences-- hopefully this transition will be easy.)

@thockin @ncdc

@ncdc

This comment has been minimized.

Copy link
Member

@ncdc ncdc commented Apr 28, 2015

Waiting on HTTP/2 support for Go that includes low level client/server ability to create and handle streams (https://github.com/bradfitz/http2).

docker/spdystream#53 pulls the upstream spdy code into spdystream.

@lavalamp

This comment has been minimized.

Copy link
Member Author

@lavalamp lavalamp commented Apr 28, 2015

Ah, after docker/spdystream#53 is merged, we can at least make godep happy again. :)

@ncdc

This comment has been minimized.

Copy link
Member

@ncdc ncdc commented Apr 30, 2015

@lavalamp spdystream PR is merged, FYI

@ghost ghost removed the team/master label Aug 20, 2015
@ncdc

This comment has been minimized.

Copy link
Member

@ncdc ncdc commented Dec 1, 2015

@thockin @lavalamp it looks like the Go HTTP/2 issue was just closed (golang/go#6891 (comment)). I glanced quickly through the code, but I don't see a way to hijack the connection and create our own streams, like we do with spdystream. Hopefully we can get this functionality in there at some point...

@ncdc

This comment has been minimized.

Copy link
Member

@ncdc ncdc commented Dec 1, 2015

@ncdc

This comment has been minimized.

Copy link
Member

@ncdc ncdc commented Dec 1, 2015

I created golang/go#13444 as an RFE for end-user stream support

@ddysher

This comment has been minimized.

Copy link
Contributor

@ddysher ddysher commented Feb 23, 2016

What's the plan for the deprecation? We are using nginx in front of apiserver which drops support for SPDY since version 1.9.5. Our workaround is to use an older nginx, but would like to see http/2 support go upstream :)

@ncdc

This comment has been minimized.

Copy link
Member

@ncdc ncdc commented Feb 23, 2016

I prototyped this with a prerelease version of go1.6. It worked somewhat, but there were some rough spots (probably stuff I was doing incorrectly). We can't move to http/2 until go1.6 at the earliest, and even then we'll have to do R&D to see if it's technically feasible. I had been hoping that the http/2 support in go1.6 would allow me to create streams just like we can do in spdystream, but that isn't exposed via the API. For my prototype, I ended up writing a simple multiplexer just like @smarterclayton wrote for the websocket support we have in Kubernetes now.

@ddysher

This comment has been minimized.

Copy link
Contributor

@ddysher ddysher commented Feb 23, 2016

Thanks for the heads up. Looking at the linked comments, there is no intention to add the support in go1.6, so I guess it's probably still months away.

@JensRantil

This comment has been minimized.

Copy link

@JensRantil JensRantil commented Oct 29, 2016

We can't move to http/2 until go1.6 at the earliest

Golang 1.7 is out now. Does that mean we can make progress on this issue now? Anything else stopping us?

@smarterclayton

This comment has been minimized.

Copy link
Contributor

@smarterclayton smarterclayton commented Oct 29, 2016

A usable HTTP/2 framer in Go that allows us hijack control and the ability
to stand up the connection. Last I saw we were still blocked without
explicitly forking the Go implementation.

In the meantime, websockets continue to be an option (with portforward
being the last bit remaining).

On Oct 29, 2016, at 2:22 PM, Jens Rantil notifications@github.com wrote:

We can't move to http/2 until go1.6 at the earliest

Golang 1.7 is out now. Does that mean we can make progress on this issue
now? Anything else stopping us?


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#7452 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABG_p-fe3BrfKe4VjAQyYLOgK7AjvhLkks5q447ngaJpZM4EK7sh
.

@ncdc

This comment has been minimized.

Copy link
Member

@ncdc ncdc commented Oct 31, 2016

We should be able to use HTTP/2 without any changes to the Go implementation. But we would have to write our own code to do multiplexing of multiple data streams over a single request/response body pair, which we'd rather not do if we can avoid it.

@bradfitz

This comment has been minimized.

Copy link

@bradfitz bradfitz commented Nov 27, 2016

@ncdc, if it helps, https://godoc.org/golang.org/x/build/revdial is used by the Go build system to multiplex multiple net.Conns and a net.Listener all over a single net.Conn.

@fraenkel

This comment has been minimized.

Copy link
Contributor

@fraenkel fraenkel commented Nov 28, 2016

@bradfitz Thanks for the pointer, but it doesn't fix the underlying issue. While SPDY is being used, it is not following HTTP request/response semantics. A logical connection is created and then data is sent in either direction as needed. It would be equivalent to the client sending an HTTP request and the server doing a Push whenever.

@ncdc

This comment has been minimized.

Copy link
Member

@ncdc ncdc commented Nov 28, 2016

@fraenkel I don't think there's any technical limitation in the facilities provided by the Go HTTP/2 implementation. As I wrote above, we would just need to write our own muxing/framing code, which we'd rather avoid (or find a library that can do it?).

@fraenkel

This comment has been minimized.

Copy link
Contributor

@fraenkel fraenkel commented Nov 28, 2016

@ncdc It might be easier to just build on top of gRPC which provides bi-directional streaming or copy what they have done.

@smarterclayton

This comment has been minimized.

Copy link
Contributor

@smarterclayton smarterclayton commented Nov 28, 2016

@diegodelemos diegodelemos referenced this issue Dec 13, 2016
@luxas

This comment has been minimized.

Copy link
Member

@luxas luxas commented Jan 17, 2017

Ping, any update on this?

@wlan0

This comment has been minimized.

Copy link
Member

@wlan0 wlan0 commented Jan 20, 2017

We at Rancher Labs are also interested in this change.

@smarterclayton

This comment has been minimized.

Copy link
Contributor

@smarterclayton smarterclayton commented Jun 1, 2017

We can actually just move to web sockets in the near term. The server has supported it for a while and HOL blocking is not an issue

@fejta-bot

This comment has been minimized.

Copy link

@fejta-bot fejta-bot commented Jan 2, 2018

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or @fejta.
/lifecycle stale

@luxas

This comment has been minimized.

Copy link
Member

@luxas luxas commented Jan 2, 2018

/remove-lifecycle stale
/lifecycle frozen

Assuming @smarterclayton is still working on #50428

@josdotso

This comment has been minimized.

Copy link

@josdotso josdotso commented Sep 11, 2018

Is there any status update on this issue?

@mfranczy

This comment has been minimized.

Copy link

@mfranczy mfranczy commented Nov 13, 2018

Please ignore my reference, it has nothing to do with that issue, if I could I would remove it, sorry.

@ensonic

This comment has been minimized.

Copy link

@ensonic ensonic commented Nov 16, 2018

We run a http/2 reverse proxy to expose a on-prem api-server. This issues breaks kubectl exec and thereby also tools like helm.

@dims

This comment has been minimized.

Copy link
Member

@dims dims commented Nov 16, 2018

long-term-issue (note-to-self)

@mikedanese

This comment has been minimized.

Copy link
Member

@mikedanese mikedanese commented Nov 17, 2018

I've started working on streaming over h2 here:

https://github.com/kubernetes/kubernetes/compare/master...mikedanese:h2?expand=1

I hope to make some progress in 1.14

@mikedanese mikedanese self-assigned this Nov 17, 2018
@mikedanese

This comment has been minimized.

Copy link
Member

@mikedanese mikedanese commented Nov 20, 2018

I'd like to see this happen in two parts:

  1. support all streaming codecs over http/2.
  2. deprecate spdy streaming codecs.

I think (1) is probably the biggest win.

@mikedanese

This comment has been minimized.

Copy link
Member

@mikedanese mikedanese commented Nov 21, 2018

I have a rough prototype of (1) working e2e from kubectl to CRI. Migration will be tricky. I haven't thought of a way for clients to discover h2 streaming capabilities since the entire stack (kubectl -> apiserver -> kubelet -> cri) needs to support streaming over h2. With a significantly more complex implementation, the upgrade aware proxy could translate between clients and servers speaking different protocols. If that were the case, we'd likely only need single hop discovery to transition smoothly.

@mikedanese

This comment has been minimized.

Copy link
Member

@mikedanese mikedanese commented Dec 4, 2018

#71411 is an example of the dangers of dropping an http connection to TCP. There's a category of bugs to be avoided but this one was particularly insidious.

@scult

This comment has been minimized.

Copy link

@scult scult commented May 13, 2019

Any update on this?

@mikedanese

This comment has been minimized.

Copy link
Member

@mikedanese mikedanese commented May 13, 2019

We are blocked on support for https://tools.ietf.org/html/rfc8441 in x/net/http2 if we don't want to implement something non-standard again.

@jayunit100

This comment has been minimized.

Copy link
Member

@jayunit100 jayunit100 commented May 26, 2019

Is it possible for the API server to support HTTP2 for watches etc... while still using HTTP1 for everything else? xref kubernetes/client-go#374. Forgive me if this question is a little ignorant, im not up on the latest API stuff :).

@jmillikin-stripe

This comment has been minimized.

Copy link
Contributor

@jmillikin-stripe jmillikin-stripe commented Oct 11, 2019

Has there been any progress on this since May? The use of SPDY for kubectl exec continues to be a pain point because it's no longer supported by common network proxies (nginx, haproxy, envoy, and so on).

Naively, I would expect something like interactive sessions to be implemented on top of gRPC or another thin layer on the normal HTTP/2 framing. Websockets, CONNECT, and other ways to get a raw byte stream have a difficult security model and need much more support in intermediate software compared to plain HTTP/2.

@jfrabaute

This comment has been minimized.

Copy link

@jfrabaute jfrabaute commented Oct 12, 2019

Hi,

I have a setup where proxying kubectl exec is working, using openresty (docker image openresty/openresty:1.15.8.1-2-stretch to be precise).
It should also work with nginx as openresty is based on nginx.

The relevant config in the location block to have kubectl exec proxied correctly is to add:

              proxy_set_header Upgrade $http_upgrade;
              proxy_set_header Connection "Upgrade";

this is the listen line in the server block:

listen 443 http2 ssl

The proxied kube-apiserver is running 1.14.3-gke.11 (so, k8s 1.14 with gke modified version).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.