New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support serving web content from the same port #75

Closed
tv42 opened this Issue Feb 26, 2015 · 70 comments

Comments

Projects
None yet
@tv42
Copy link

tv42 commented Feb 26, 2015

https://github.com/grpc/grpc-common/blob/master/PROTOCOL-HTTP2.md#appendix-a---grpc-for-protobuf says grpc protobuf mapping uses service names as paths. Would it be possible to serve web content from other urls?

I see TLS side does alpn, so that's an easy place to hook up.

Any thoughts about non-TLS? e.g. a service running on localhost. Of course that would mean needing to do a http/1 upgrade negotiation, as then the port could not default to http/2.

Use case 1: host a web application and its api on the same port.

Use case 2: serve a health check response.

Use case 3: serve a "nothing to see here" html page.

Use case 4: serve a /robots.txt file.

@tv42

This comment has been minimized.

Copy link
Author

tv42 commented Feb 28, 2015

I just realized the grpc tls alpn is just HTTP/2 drafts, not a separate protocol. Scratch that plan; the splitting of grpc vs other needs to happen more per-{request|stream|something}.

@stapelberg

This comment has been minimized.

Copy link

stapelberg commented Feb 28, 2015

I’d be interested in this as well.

@acasajus

This comment has been minimized.

Copy link

acasajus commented Mar 9, 2015

Since there's no grpc/grpc-experiments#152 Can there be a failover stream handler? If the grpc server cannot process the stream (the service/method doesn't exist for instance) we could have another handler that devs can define to process it.

@iamqizhao

This comment has been minimized.

Copy link
Contributor

iamqizhao commented Mar 9, 2015

Yup, I think adding some additional handlers for non-grpc content is the way to go just like what we have done for google internal rpc framework. I will raise it to design discussion and get back to you once we have a decision.

@acasajus

This comment has been minimized.

Copy link

acasajus commented Mar 17, 2015

From grpc/grpc-experiments#152: There's a mandatory content type for grpc. So streams that do not have it can be served by the default http handler.

@acasajus

This comment has been minimized.

Copy link

acasajus commented Apr 14, 2015

Any news on this? Should we go ahead and make a PR?

@xiang90

This comment has been minimized.

Copy link
Contributor

xiang90 commented Apr 15, 2015

I am also interested in this one.

/cc @xiang90 @yichengq

@xiang90

This comment has been minimized.

Copy link
Contributor

xiang90 commented Apr 22, 2015

@iamqizhao Have you had the discussion yet? Any decision? Anything I can help to make this happen?

@iamqizhao

This comment has been minimized.

Copy link
Contributor

iamqizhao commented Apr 22, 2015

I think this is good to have. But I do not see a compelling demand to have
it now given a lot of other stuffs I am working on. So it is on my radar
but a low priority item. I will be more than happy to take a PR from you
guys.

On Wed, Apr 22, 2015 at 10:53 AM, Xiang Li notifications@github.com wrote:

@iamqizhao https://github.com/iamqizhao Have you had the discussion
yet? Any decision? Anything I can help to make this happen?


Reply to this email directly or view it on GitHub
#75 (comment).

@xiang90

This comment has been minimized.

Copy link
Contributor

xiang90 commented Apr 22, 2015

@iamqizhao I can work on it.

Motivation:

  1. currently etcd and a few other my side projects have a http1 endpoint as it API.
  2. I want to support GRPC in the next few releases without adding a new port or break compatibility.

So ideally, grpc can forward http1 requests to my old handlers. But can you give me some direction to do that? I can start to explore and hopefully have a pr for it.

@iamqizhao

This comment has been minimized.

Copy link
Contributor

iamqizhao commented Apr 22, 2015

On Wed, Apr 22, 2015 at 11:16 AM, Xiang Li notifications@github.com wrote:

@iamqizhao https://github.com/iamqizhao I can work on it.

Motivation:

  1. currently etcd and a few other my side projects have a http1
    endpoint as it API.
  2. I want to support GRPC in the next few releases without adding a
    new port or break compatibility.

So ideally, grpc can forward http1 requests to my old handlers. But can
you give me some direction to do that? I can start to explore and hopefully
have a pr for it.

grpc-go uses bradfitz/http2 package to write & read http2 frames. I suspect
the framer.ReadFrame will error out if it receives a http1 request (need
look into the code or check with brad). If it is the case, it is not
trivial to implement this feature without some change to http2 package...

You can probably check https://github.com/gengo/grpc-gateway to see if
there is alternative to your problem.


Reply to this email directly or view it on GitHub
#75 (comment).

@xiang90

This comment has been minimized.

Copy link
Contributor

xiang90 commented Apr 22, 2015

@iamqizhao

It seems @bradfitz's http2 server supports both http1 and http2 at the same time. Here is the related code

https://github.com/bradfitz/http2/blob/91f80303028022bc2034c277126b405a2257d990/server.go#L185-L192

Are we able to do the similar thing?

@iamqizhao

This comment has been minimized.

Copy link
Contributor

iamqizhao commented Apr 22, 2015

On Wed, Apr 22, 2015 at 4:04 PM, Xiang Li notifications@github.com wrote:

@iamqizhao https://github.com/iamqizhao

It seems @bradfitz https://github.com/bradfitz's http2 server supports
both http1 and http2 at the same time. Here is the related code

https://github.com/bradfitz/http2/blob/91f80303028022bc2034c277126b405a2257d990/server.go#L185-L192

Yep, the idea is to push the divergence stage (http1 or http2) before there
is any real traffic. It uses the usage NPN/ALPN to decide whether
http.Server creates a http2 ServerConn and dispatches the traffic there.
http1 traffic can never enter http2 package. If we put grpc into this
picture, there are grpc, http2 and http1 handlers. Then the usage of
NPN/ALPN is not enough to make decision which server conn we should setup
(apparently no idea between http2.ServerConn and grpc.ServerTransport; and
actually I am not convinced using NPN/ALPN is sufficient for http1 and
grpc). To summarize, to design a working solution, we have to find
something which can be used to see the upcoming traffic on the connection
is http1/http2/grpc. NPN/ALPN seems not the choice to me.

Are we able to do the similar thing?
Can


Reply to this email directly or view it on GitHub
#75 (comment).

@iamqizhao

This comment has been minimized.

Copy link
Contributor

iamqizhao commented Apr 23, 2015

okay, my proposal is sketched as follows:

we can implement a handshaker. The client connects to a http.Server and
then sends a http1.1 GET with some magic string such as "gRpC-sWiTcH". The
corresponding handler will pass that fd to a grpc server (and response to
the client via handshaker too). After that, all the traffic on that fd is
taken care of by the grpc server. Otherwise, the normal http handler will
be used.

On Wed, Apr 22, 2015 at 4:33 PM, Qi Zhao toqizhao@gmail.com wrote:

On Wed, Apr 22, 2015 at 4:04 PM, Xiang Li notifications@github.com
wrote:

@iamqizhao https://github.com/iamqizhao

It seems @bradfitz https://github.com/bradfitz's http2 server supports
both http1 and http2 at the same time. Here is the related code

https://github.com/bradfitz/http2/blob/91f80303028022bc2034c277126b405a2257d990/server.go#L185-L192

Yep, the idea is to push the divergence stage (http1 or http2) before
there is any real traffic. It uses the usage NPN/ALPN to decide whether
http.Server creates a http2 ServerConn and dispatches the traffic there.
http1 traffic can never enter http2 package. If we put grpc into this
picture, there are grpc, http2 and http1 handlers. Then the usage of
NPN/ALPN is not enough to make decision which server conn we should setup
(apparently no idea between http2.ServerConn and grpc.ServerTransport; and
actually I am not convinced using NPN/ALPN is sufficient for http1 and
grpc). To summarize, to design a working solution, we have to find
something which can be used to see the upcoming traffic on the connection
is http1/http2/grpc. NPN/ALPN seems not the choice to me.

Are we able to do the similar thing?
Can


Reply to this email directly or view it on GitHub
#75 (comment).

@bakins

This comment has been minimized.

Copy link

bakins commented Apr 23, 2015

Why not use HTTP Upgrade?

@iamqizhao

This comment has been minimized.

Copy link
Contributor

iamqizhao commented Apr 23, 2015

On Wed, Apr 22, 2015 at 5:50 PM, Brian Akins notifications@github.com
wrote:

Why not use HTTP Upgrade?

To me, it means a more complex solution -- http upgrade can help us
differentiate http1 and http2. Then for http2, after http2.Framer.ReadFrame
returns a frame, we can use the content-type header to differentiate grpc
and normal http2. This constructs a 2-layer solution -- on connection
level, we use http upgrade and on per-stream level, we use content-type.
This sounds too complex to me -- keep in mind that we also need to
implement a hook for grpc server to handle normal http2 traffic in this
solution.


Reply to this email directly or view it on GitHub
#75 (comment).

@bradfitz

This comment has been minimized.

Copy link
Contributor

bradfitz commented Apr 23, 2015

This discussion should happen in the normal grpc discussion places first, not in grpc-go. Once a language-neutral plan is agreed upon, then grpc-go can implement it.

In the meantime, users outside of grpc-go can implement a net.Listener/net.Conn that sniffs the beginning traffic and routes depending on whether it looks like a TLS handshake (with the ALPN for "h2") or an plaintext HTTP/1 request "GET / HTTP/1.1" etc.

But let's please not do some gprc-go-specific hack here if other languages don't do the same.

@xiang90

This comment has been minimized.

Copy link
Contributor

xiang90 commented Apr 23, 2015

@bradfitz Agreed. I will move the discussion to grpc.

@tv42

This comment has been minimized.

Copy link
Author

tv42 commented Apr 23, 2015

@bradfitz That part doesn't help differentiate web traffic from grpc traffic, when they both happen to run over TLS & even the same ALPN. That really is the interesting part.

Actually, I don't see the grpc protocol spec saying anything about what part of it is so special that it can't just look like POST requests to net/http. Anyone have a short answer to that?

@bradfitz

This comment has been minimized.

Copy link
Contributor

bradfitz commented Apr 23, 2015

It's hard to design something until there's a defined problem.

@acasajus

This comment has been minimized.

Copy link

acasajus commented Apr 23, 2015

@tv42 Doesn't the mandatory "Content-Type: application/grpc" work for differentiating between grpc and plain http requests? Granted that's once the handshake has passed.

@tv42

This comment has been minimized.

Copy link
Author

tv42 commented Apr 23, 2015

@acasajus Sure, it would, but right now grpc seems to sort of reimplement part of net/http / bradfitz/http2, so it's not clear how one would pass a request to the other. It's not like one could have a http.Handler inspect the headers and call one of two alternate http.Handlers.

@bradfitz

This comment has been minimized.

Copy link
Contributor

bradfitz commented Apr 24, 2015

@tv42, yeah, grpc-go was developed in parallel with bradfitz/http2. They share a framer, but they have different server implementations. So it's not exactly clear whether and where it would be possible to make one defer to the other. I'd personally like to see them unified someday, perhaps with grpc-go just being a handler under http2 itself, using the http2 server code. But then, grpc might be faster doing it by hand. I'm not sure.

@philips

This comment has been minimized.

Copy link
Contributor

philips commented May 16, 2015

@xiang90 @bradfitz Where is this discussion happening in the "normal grpc" channels?

@bradfitz

This comment has been minimized.

Copy link
Contributor

bradfitz commented May 17, 2015

I actually don't know. I just assumed there was one, since they have a fancy website (http://www.grpc.io/) and all.

@bradfitz

This comment has been minimized.

Copy link
Contributor

bradfitz commented May 17, 2015

@philips

This comment has been minimized.

Copy link
Contributor

philips commented May 17, 2015

@xiang90

This comment has been minimized.

Copy link
Contributor

xiang90 commented May 18, 2015

@iamqizhao @philips

I tend to agree on this https://groups.google.com/forum/#!topic/grpc-io/JnjCYGPMUms. (grpc-java plan)

Can we do something similar in grpc-go?

Thanks.

@tv42

This comment has been minimized.

Copy link
Author

tv42 commented May 18, 2015

As the originator of this issue, I'd like to say this: I really don't care about running gRPC over HTTP/1.1. If you want that, please make a separate issue.

What I want is to have a single TCP port that can be accessed both with a modern browser and a gRPC client, where both behave sanely. That can mean always TLS and all gRPC over HTTP/2; whether the browser uses HTTP/2 or HTTP/1.1 is not relevant.

@iamqizhao

This comment has been minimized.

Copy link
Contributor

iamqizhao commented May 18, 2015

On Mon, May 18, 2015 at 10:08 AM, Xiang Li notifications@github.com wrote:

@iamqizhao https://github.com/iamqizhao @philips
https://github.com/philips

I tend to agree this
https://groups.google.com/forum/#!topic/grpc-io/JnjCYGPMUms. (grpc-java
plan)

Can we do something similar in grpc-go?

As I mentioned before, this will involve some redesign of http2 package. In
general, I suspect whether it is worth doing a lot of redesign and
rewriting to accommodate http1 traffic and I strongly lean to serve http2
and http1 traffic on different ports.

Thanks.


Reply to this email directly or view it on GitHub
#75 (comment).

bradfitz added a commit to bradfitz/grpc-go that referenced this issue Jan 29, 2016

bradfitz added a commit to bradfitz/grpc-go that referenced this issue Jan 31, 2016

Add a ServeHTTP method to *grpc.Server
This adds new http.Handler-based ServerTransport in the process,
reusing the HTTP/2 server code in x/net/http2 or Go 1.6+.

All end2end tests pass with this new ServerTransport.

Updates grpc#75

bradfitz added a commit to bradfitz/grpc-go that referenced this issue Jan 31, 2016

Add a ServeHTTP method to *grpc.Server
This adds new http.Handler-based ServerTransport in the process,
reusing the HTTP/2 server code in x/net/http2 or Go 1.6+.

All end2end tests pass with this new ServerTransport.

Fixes grpc#75

Also:
Updates grpc#495 (lets user fix it with middleware in front)
Updates grpc#468 (x/net/http2 validates)
Updates grpc#147 (possible with x/net/http2)
Updates grpc#104 (x/net/http2 does this)

bradfitz added a commit to bradfitz/grpc-go that referenced this issue Feb 1, 2016

Add a ServeHTTP method to *grpc.Server
This adds new http.Handler-based ServerTransport in the process,
reusing the HTTP/2 server code in x/net/http2 or Go 1.6+.

All end2end tests pass with this new ServerTransport.

Fixes grpc#75

Also:
Updates grpc#495 (lets user fix it with middleware in front)
Updates grpc#468 (x/net/http2 validates)
Updates grpc#147 (possible with x/net/http2)
Updates grpc#104 (x/net/http2 does this)

bradfitz added a commit to bradfitz/grpc-go that referenced this issue Feb 1, 2016

Add a ServeHTTP method to *grpc.Server
This adds new http.Handler-based ServerTransport in the process,
reusing the HTTP/2 server code in x/net/http2 or Go 1.6+.

All end2end tests pass with this new ServerTransport.

Fixes grpc#75

Also:
Updates grpc#495 (lets user fix it with middleware in front)
Updates grpc#468 (x/net/http2 validates)
Updates grpc#147 (possible with x/net/http2)
Updates grpc#104 (x/net/http2 does this)

bradfitz added a commit to bradfitz/grpc-go that referenced this issue Feb 1, 2016

Add a ServeHTTP method to *grpc.Server
This adds new http.Handler-based ServerTransport in the process,
reusing the HTTP/2 server code in x/net/http2 or Go 1.6+.

All end2end tests pass with this new ServerTransport.

Fixes grpc#75

Also:
Updates grpc#495 (lets user fix it with middleware in front)
Updates grpc#468 (x/net/http2 validates)
Updates grpc#147 (possible with x/net/http2)
Updates grpc#104 (x/net/http2 does this)

bradfitz added a commit to bradfitz/grpc-go that referenced this issue Feb 4, 2016

Add a ServeHTTP method to *grpc.Server
This adds new http.Handler-based ServerTransport in the process,
reusing the HTTP/2 server code in x/net/http2 or Go 1.6+.

All end2end tests pass with this new ServerTransport.

Fixes grpc#75

Also:
Updates grpc#495 (lets user fix it with middleware in front)
Updates grpc#468 (x/net/http2 validates)
Updates grpc#147 (possible with x/net/http2)
Updates grpc#104 (x/net/http2 does this)

bradfitz added a commit to bradfitz/grpc-go that referenced this issue Feb 4, 2016

Add a ServeHTTP method to *grpc.Server
This adds new http.Handler-based ServerTransport in the process,
reusing the HTTP/2 server code in x/net/http2 or Go 1.6+.

All end2end tests pass with this new ServerTransport.

Fixes grpc#75

Also:
Updates grpc#495 (lets user fix it with middleware in front)
Updates grpc#468 (x/net/http2 validates)
Updates grpc#147 (possible with x/net/http2)
Updates grpc#104 (x/net/http2 does this)

bradfitz added a commit to bradfitz/grpc-go that referenced this issue Feb 4, 2016

Add a ServeHTTP method to *grpc.Server
This adds new http.Handler-based ServerTransport in the process,
reusing the HTTP/2 server code in x/net/http2 or Go 1.6+.

All end2end tests pass with this new ServerTransport.

Fixes grpc#75

Also:
Updates grpc#495 (lets user fix it with middleware in front)
Updates grpc#468 (x/net/http2 validates)
Updates grpc#147 (possible with x/net/http2)
Updates grpc#104 (x/net/http2 does this)

bradfitz added a commit to bradfitz/grpc-go that referenced this issue Feb 5, 2016

Add a ServeHTTP method to *grpc.Server
This adds new http.Handler-based ServerTransport in the process,
reusing the HTTP/2 server code in x/net/http2 or Go 1.6+.

All end2end tests pass with this new ServerTransport.

Fixes grpc#75

Also:
Updates grpc#495 (lets user fix it with middleware in front)
Updates grpc#468 (x/net/http2 validates)
Updates grpc#147 (possible with x/net/http2)
Updates grpc#104 (x/net/http2 does this)

bradfitz added a commit to bradfitz/grpc-go that referenced this issue Feb 8, 2016

Add a ServeHTTP method to *grpc.Server
This adds new http.Handler-based ServerTransport in the process,
reusing the HTTP/2 server code in x/net/http2 or Go 1.6+.

All end2end tests pass with this new ServerTransport.

Fixes grpc#75

Also:
Updates grpc#495 (lets user fix it with middleware in front)
Updates grpc#468 (x/net/http2 validates)
Updates grpc#147 (possible with x/net/http2)
Updates grpc#104 (x/net/http2 does this)

bradfitz added a commit to bradfitz/grpc-go that referenced this issue Feb 9, 2016

Add a ServeHTTP method to *grpc.Server
This adds new http.Handler-based ServerTransport in the process,
reusing the HTTP/2 server code in x/net/http2 or Go 1.6+.

All end2end tests pass with this new ServerTransport.

Fixes grpc#75

Also:
Updates grpc#495 (lets user fix it with middleware in front)
Updates grpc#468 (x/net/http2 validates)
Updates grpc#147 (possible with x/net/http2)
Updates grpc#104 (x/net/http2 does this)

bradfitz added a commit to bradfitz/grpc-go that referenced this issue Feb 9, 2016

Add a ServeHTTP method to *grpc.Server
This adds new http.Handler-based ServerTransport in the process,
reusing the HTTP/2 server code in x/net/http2 or Go 1.6+.

All end2end tests pass with this new ServerTransport.

Fixes grpc#75

Also:
Updates grpc#495 (lets user fix it with middleware in front)
Updates grpc#468 (x/net/http2 validates)
Updates grpc#147 (possible with x/net/http2)
Updates grpc#104 (x/net/http2 does this)

bradfitz added a commit to bradfitz/grpc-go that referenced this issue Feb 9, 2016

Add a ServeHTTP method to *grpc.Server
This adds new http.Handler-based ServerTransport in the process,
reusing the HTTP/2 server code in x/net/http2 or Go 1.6+.

All end2end tests pass with this new ServerTransport.

Fixes grpc#75

Also:
Updates grpc#495 (lets user fix it with middleware in front)
Updates grpc#468 (x/net/http2 validates)
Updates grpc#147 (possible with x/net/http2)
Updates grpc#104 (x/net/http2 does this)

bradfitz added a commit to bradfitz/grpc-go that referenced this issue Feb 9, 2016

Add a ServeHTTP method to *grpc.Server
This adds new http.Handler-based ServerTransport in the process,
reusing the HTTP/2 server code in x/net/http2 or Go 1.6+.

All end2end tests pass with this new ServerTransport.

Fixes grpc#75

Also:
Updates grpc#495 (lets user fix it with middleware in front)
Updates grpc#468 (x/net/http2 validates)
Updates grpc#147 (possible with x/net/http2)
Updates grpc#104 (x/net/http2 does this)

bradfitz added a commit to bradfitz/grpc-go that referenced this issue Feb 10, 2016

Add a ServeHTTP method to *grpc.Server
This adds new http.Handler-based ServerTransport in the process,
reusing the HTTP/2 server code in x/net/http2 or Go 1.6+.

All end2end tests pass with this new ServerTransport.

Fixes grpc#75

Also:
Updates grpc#495 (lets user fix it with middleware in front)
Updates grpc#468 (x/net/http2 validates)
Updates grpc#147 (possible with x/net/http2)
Updates grpc#104 (x/net/http2 does this)

bradfitz added a commit to bradfitz/grpc-go that referenced this issue Feb 11, 2016

Add a ServeHTTP method to *grpc.Server
This adds new http.Handler-based ServerTransport in the process,
reusing the HTTP/2 server code in x/net/http2 or Go 1.6+.

All end2end tests pass with this new ServerTransport.

Fixes grpc#75

Also:
Updates grpc#495 (lets user fix it with middleware in front)
Updates grpc#468 (x/net/http2 validates)
Updates grpc#147 (possible with x/net/http2)
Updates grpc#104 (x/net/http2 does this)

bradfitz added a commit to bradfitz/grpc-go that referenced this issue Feb 11, 2016

Add a ServeHTTP method to *grpc.Server
This adds new http.Handler-based ServerTransport in the process,
reusing the HTTP/2 server code in x/net/http2 or Go 1.6+.

All end2end tests pass with this new ServerTransport.

Fixes grpc#75

Also:
Updates grpc#495 (lets user fix it with middleware in front)
Updates grpc#468 (x/net/http2 validates)
Updates grpc#147 (possible with x/net/http2)
Updates grpc#104 (x/net/http2 does this)

bradfitz added a commit to bradfitz/grpc-go that referenced this issue Feb 11, 2016

Add a ServeHTTP method to *grpc.Server
This adds new http.Handler-based ServerTransport in the process,
reusing the HTTP/2 server code in x/net/http2 or Go 1.6+.

All end2end tests pass with this new ServerTransport.

Fixes grpc#75

Also:
Updates grpc#495 (lets user fix it with middleware in front)
Updates grpc#468 (x/net/http2 validates)
Updates grpc#147 (possible with x/net/http2)
Updates grpc#104 (x/net/http2 does this)

bradfitz added a commit to bradfitz/grpc-go that referenced this issue Feb 12, 2016

Add a ServeHTTP method to *grpc.Server
This adds new http.Handler-based ServerTransport in the process,
reusing the HTTP/2 server code in x/net/http2 or Go 1.6+.

All end2end tests pass with this new ServerTransport.

Fixes grpc#75

Also:
Updates grpc#495 (lets user fix it with middleware in front)
Updates grpc#468 (x/net/http2 validates)
Updates grpc#147 (possible with x/net/http2)
Updates grpc#104 (x/net/http2 does this)

@lock lock bot locked as resolved and limited conversation to collaborators Sep 26, 2018

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.