Abstract transport from client logic (for gRPC-Web, in-process, etc)

[johanbrandhorst]

Feature Request

Motivation

tonic generates client APIs for HTTP/2 gRPC clients. However;

• The grpc-web protocol defines how browser clients can interact with gRPC servers. Rust has excellent support for compiling to WebAssembly, to run in the browser.
• Sometimes a client and server are co-located, and can take advantage of in-process communication.

Proposal

I propose allowing the transport to be abstracted from the rest of the application logic so that alternative transports could be implemented.

This may add additional overhead in the transport, but ideally it would not.

Alternatives

Implementing specific clients and servers for these use cases is an alternative, but will contain a lot of duplicate logic.

Related issues

tower-rs/tower-grpc#35
grpc/grpc-go#906

[LucioFranco]

@johanbrandhorst Hi! This is definitely something I would like to support. So the goal of the transport module is actually to be the "HTTP/2" implementation. In fact, this transport is far from required!

I'm assuming that we are mostly talking about clients for the moment maybe running in the browser via wasm. This is totally possible if you are able to write a client that can implement https://docs.rs/tonic/0.1.0-alpha.1/tonic/client/trait.GrpcService.html As far as I can tell this should be possible. The second key area that will need some work for this that we use https://docs.rs/http-body/ to abstract what an http body must do. In theory it should be possible to wrap this and map a "HTTP/2" request into a "HTTP/1.1" by consuming the body and updating the headers.

I believe once the body stuff is figured out the client and server support for grpc-web will be very easy. For inter process, this should be very stragiht forward of just using a channel. https://docs.rs/tower-test may already kinda work as well.

(I am still waking up so hopefully that makes some sense)

[johanbrandhorst]

Hi Lucio.

That sounds great! I have literally no experience with Rust, but I was thinking about trying to write a gRPC-Web client in Rust with tonic while testing against a Go gRPC server wrapped in the Improbable grpc-web proxy. Don't count on this happening anytime soon though, but thanks for the pointers!

As you said, the server side would involve implementing the proxy part. It may be useful to look at the Improbable Go proxy for inspiration: https://github.com/improbable-eng/grpc-web/tree/master/go/grpcweb. I think ideally this would be implemented as a wrapper around the HTTP/2 transport, so that users could simply choose whether to turn on or off grpc-web support.

It sounds like this should be very possible, which is great!

[LucioFranco]

@johanbrandhorst yeah that would be nice. The biggest issue is just having to map http 2 bodies to http 1 bodies. Even though they are the same type I believe the trailing headers might be an issue.

@seanmonstar you might have more thoughts around how we might be able to accomplish this?

[johanbrandhorst]

The grpc-web spec defines how to handle trailers - they go in the body. https://github.com/grpc/grpc/blob/master/doc/PROTOCOL-WEB.md.

[LucioFranco]

Yeah, so dealing with the body stream we will need to consume the whole body before we can send it I believe. So that part will be a bit complex. @johanbrandhorst if you want to tackle this, feel free to come onto discord and I can explain more about the body stuff.

[LucioFranco]

Also, I'd like to build a grpc-web proxy but written in rust :)

[johanbrandhorst]

Yeah, so dealing with the body stream we will need to consume the whole body before we can send it I believe. So that part will be a bit complex. @johanbrandhorst if you want to tackle this, feel free to come onto discord and I can explain more about the body stuff.

I will if I ever find the time, but I think I will start with the client side. Thanks for the great project by the way :).

[LucioFranco]

@pkinsky @johanbrandhorst thank you both! I'm not sure exactly what the best path forward might be but I think the first goal would be to build a HTTP/1.1 transport and see how that interacts with our current body usage etc.

[johanbrandhorst]

I've contacted Paul asynchronously to discuss next steps :).

[johanbrandhorst]

I've created a sandbox repo to play around with prototyping this: https://github.com/johanbrandhorst/rust-grpc-web-wasm-test. It's currently just a Go gRPC server with the gRPC-Web proxy, the idea is that we can manually try to implement a first pass on a Rust WASM client. See the README for more information.

[LucioFranco]

@johanbrandhorst sounds great! Looking forward to see what turns out of it! :)

[ctaggart]

I'd really like a rust wasm client. It looks like Blazor WebAssembly has an implementation now. It is starting to catch on: Using gRPC-Web with Blazor WebAssembly

[john-hern]

I've been looking at this problem for a couple of weeks while reading tonic/tower's code. I have started an implementation on the server side part of this. I was hoping to get a little feedback about the approach.

What I've done is create another svc, called ProxySvc. In the MakeSvc handler, I just check for the proxy config, which will passed through the server, and it if exists, I wrap the Svc structure in the ProxySvc and hand it the config.
Ok(BoxService::new(ProxySvc{ inner: Svc { inner: svc, span, conn_info }, config: proxy_config }))
This lets my code execute first/last in the pipeline, allowing me to transform the HTTP/* requests/responses to HTTP/2 and vice versa, and handle the CORs headers.

Is this a reasonable approach within Tonic? Or is there a better place to hook the pipeline? I looked at the layers and they seemed to only apply to one side or the other of the pipeline (Either, Request/Response). I also considered just writing a pure proxy, then exposing it, but figured this approach would likely be cleaner for my needs.

In terms of progress, I was able to get as far as proxying a unary request this weekend using the sample greeter service using the JS GRPC-Web client. Hopefully, I can knock the rest out in the next couple of weekends and clean up my code for a PR.

[johanbrandhorst]

Great work @john-hern! Would you be interested in adding your Rust proxy implementation to the https://github.com/johanbrandhorst/grpc-web-compatibility-test repo? It tests the existing proxies with the known existing clients.

[john-hern]

Hey @johanbrandhorst, I dont have a pure proxy implementation. Currently, I inject right into Tonic's pipeline. This means, I'd have to pull in Tonic as a sub-module into your project, either from my fork, or carry a patch around while I dev. Either feel painful while I try to dev (Unless you have a better approach?). I considered a pure proxy option, but I wanted to avoid having multiple ports open due to my own use case for GRPC-Web. This approach, allows me to accomplish that goal, but may be counter to Tonic's design goals. I was hoping to get some feedback from @LucioFranco, or another maintainer.

There may be a better way to abstract the code to let it operate within Tonic and independently as proxy using Tower. However, I haven't dug into Tonic/Tower deep enough to find the right way to accomplish that at this time. I'll try to dig into that a bit more once I'm happy with my current code, or if someone else has a recommended better way to abstract the code that will allow it to accomplish both goals.

[johanbrandhorst]

A pure proxy shouldn't be necessary, but you mean you can't simply depend on Tonic as a crate? Alternatively it would be cool just to add the tests to the docker-compose setup, the only requirement for that is a docker image with the proxy installed.

[john-hern]

Correct, I actually had to modify a part of Tonic's code to hook the req/resp pipeline to expose the "proxy" code high enough in the stack. It may be that this isn't exactly required, but it was the best approach I could find in my time digging.

I'd be happy to integrate my stuff for testing, using the repo you've created once it's a bit more complete. I'd like to make sure it has proper abstraction, flush out the CORs feature (IE, find a standard implementation, or make mine a bit more robust since it's kind of hacky), implement feature flags and make sure it can handle server side streaming. Should only take a couple more weekends.

[LucioFranco]

@john-hern that seems like a reasonable approach, but I am not fully following it. I would suggest you open a PR and I can take a peek :)

Overall, I'm not 100% convinced if tonic wants to bundle this or if we want to separate the grpc-web components, but I think this is a good step in the right direction.

[john-hern]

Hey @LucioFranco, I sent you a draft PR for the approach mentioned above. Additionally, I tried a different approach Saturday morning (Committed to this repo: https://github.com/john-hern/tonic_grpc_web). I did integrate Tower's CORs services (After some modifications to port the futures to 0.3) in that repo, that'll i'll carry into either approach (Thanks for that code!). I'll play with it some more. In either approach, it would require some changes to Tonic to make this work. (IE, embed the proxy as a feature, or somehow allow me to enable http1 with hyper, and inject another layer since my wrapper wont get called without the right "service name in the path" from what I can tell in my brief debug session.) Next weekend, I'll see if what changes are required to inject it as a layer, which feels like the right approach. Still working that out though. Always happy to get feedback either way.

John

[LucioFranco]

@john-hern I was originally thinking that the grpc-web stuff should live as its own transport for the time being until we figure out how we want to integrate it. In theory you should be able to write a simple make service and return the generated service and pass that to hyper? That should allow you to do the same stuff without having to modify the current tonic transport.

[zancas]

Great work @john-hern! Would you be interested in adding your Rust proxy implementation to the https://github.com/johanbrandhorst/grpc-web-compatibility-test repo? It tests the existing proxies with the known existing clients.

Hi @johanbrandhorst ! The interopability tests seemed to have moved? Is this the current best link:

https://github.com/grpc/grpc-web/blob/master/doc/interop-test-descriptions.md

?

[johanbrandhorst]

Yes, indeed, I updated my repo recently to point to the official repo interop tests. They don't seem to have added any third party tests though so I'm not entirely sure yet what their process is, but I would encourage users looking to take this up now to go there first. I opened an issue to port the existing tests to the official repo too: grpc/grpc-web#828.

[zancas]

Hiya @LucioFranco ! @AloeareV and I have been poking at this a bit. For our use case we need to compile against wasm32-unknown-unknown. As you no doubt know, there's a dependency on socket2 via the transport feature via hyper. WASM doesn't supply std::net::TcpStream.

As you mention above the best approach to implementing a gRPC-Web gateway would be to implement a use-case specific transport.

To get started it seems like we need an appropriate manifest that doesn't depend on modules that aren't available to wasm32-unknown-unknown.

Would that look something like this:

[target."cfg(target_os = \"wasm32\")".dependencies] ?

My guess is that 'target_os' isn't the correct config flag, but I thought it'd be worth while to ask here before digging in (in perhaps the wrong direction).

[LucioFranco]

@zancas I think you should be able to work with all default features disabled and then implement a wasm specific transport ontop of the core grpc features? Using tower service as the glue layer.

[zancas]

Hmm... I haven't looked closely at this in some time, but it seems like WASM builds expect to be executed inside a single thread managed by the JS executor in the browser. At least I think that's what @AloeareV explained to me, but it's been long enough that I may be munging the details..

I suppose a step forward would be to try to feature-gate out incompatibilities when the target is WASM32?

At any event, I'm not actively hacking on this, and I'm not sure-when-or-if I'll return to it.

[gregdhill]

Also, I'd like to build a grpc-web proxy but written in rust :)

https://github.com/gregdhill/rust-grpc-web (cc: @LucioFranco)

[9876691]

Also, I'd like to build a grpc-web proxy but written in rust :)

https://github.com/gregdhill/rust-grpc-web (cc: @LucioFranco)

I have a modified tonic-build which generates a Web gRPC server (using actix-web) and a client which compiles to webassembly.

Source and examples https://github.com/elliptic-email/rust-grpc-web

In particular the Seed.rs example shows how to call out to a Web gRPC client stub.

The client use reqwest https://github.com/seanmonstar/reqwest to forward calls which is how I'm able to get it to compile to WASM.

At the moment it only supports unary calls. I'm still thinking how to add steaming. Websockets, SSE, XHR, not sure yet.

[dakom]

As you no doubt know, there's a dependency on socket2 via the transport feature via hyper. WASM doesn't supply std::net::TcpStream.

@zancas - just curious, but if this is the blocker to using grpc as-is, then would it be possible to abstract this enough so that a custom TcpStream like thing can be passed in for read/write/close/etc.?

Specifically - I'm wondering if it's possible to deploy this as a Cloudflare Worker, which does have the ability to create outbound raw-ish TCP sockets: https://developers.cloudflare.com/workers/runtime-apis/tcp-sockets/

[dragosrotaru]

@dakom DId you find an answer for this? Im also looking to deploy to Cloudflare Worker!