Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat: add grpc server and client (#3403)
Adds a server running a gRPC endpoint over websockets running on port 5003, a `ipfs-grpc-client` module to access the server and a `ipfs-client` module that uses the gRPC client with HTTP fallback. This is to solve shortcomings and limitations of the existing HTTP API and addresses the concerns raised in the 'Streaming HTTP APIs and errors, y u no work?' session we had at IPFS team week in NYC. ## Key points 1. Enables full duplex communication with a remote node When making an HTTP request in the browser, a [FormData][] object must be created. In order to add all the values to the FormData object, an incoming stream must be consumed in its entirety before the first byte is sent to the server. This means you cannot start processing a response before the request has been sent, so you cannot have full-duplex communication between client and server over HTTP. This seems unlikely to change in the near future. With a websocket transport for gRPC-web, individual messages can be sent backwards and forwards by the client or the server enabling full-duplex communication. This is essential for things like progress events from `ipfs.add` in the short term, and exposing the full stream capabilities of libp2p via remote client in the long term. 2. Enables streaming errors The existing HTTP API sends errors as HTTP trailers. No browser supports HTTP trailers so when a stream encounters an error, from the client's point of view the stream just stops with no possibility of finding out what happened. This can also mask intended behaviour cause users to incorrectly interpret the API. For example if you specify a timeout to a DHT query and that timeout is reached, in the browser the stream ends without an error and you take away the results you've received thinking all is well but on the CLI the same operation results in a non-zero exit code. A websocket transport has no restrictions here, since full-duplex communication is possible, errors can be received at any time. 3. Listens on websockets with no HTTP fallback gRPC-web exists and is a way of exposing a gRPC service over HTTP. Whereas gRPC supports four modes (unary, e.g. one request object and one response object, client streaming, server streaming and bidirectional streaming), gRPC-web only supports [unary and server streaming](https://github.com/grpc/grpc-web#wire-format-mode). This is due to limitations of the web platform mentioned above and doesn't give us anything over our existing HTTP API. The gRPC-web team are evaluating several options for client and bidirectional streaming, all of which require new capabilities to be added to browsers and none of which will be available in a reasonable time frame. Notably they have [no plans to use websockets](https://github.com/grpc/grpc-web/blob/master/doc/streaming-roadmap.md#issues-with-websockets) as a transport, even though it solves the problems we have today. The team from [improbable](https://improbable.io/) maintain a [gRPC-web-websockets bridge](https://github.com/improbable-eng/grpc-web) which the client added by this PR is compatible with. Their bridge also has a go implementation of a [reverse proxy](https://github.com/improbable-eng/grpc-web/tree/master/go/grpcwebproxy) for use with gRPC servers to turn them into gRPC-web servers with an optional websocket transport. My proposal is to embrace the use of websockets to solve our problems right now, then move to whatever streaming primitive the gRPC-web team settle on in the years to come. As implemented there's only websockets here and no HTTP fallback as the existing HTTP API works fine for unary operations so there's little to be gained by blocking this work on reimplementing the whole of the HTTP API in gRPC-web, and the client can pick and choose which API it'll use per-call. By running the websocket server on a different port to the existing HTTP API it gives us room to add gRPC-web fallback for the API if we find that useful. 4. Has protobuf definitions for all requests/responses See the [ipfs-grpc-protocol](https://github.com/ipfs/js-ipfs/tree/feat/add-grpc-server-and-client/packages/ipfs-grpc-protocol) module, which contains definitions for API requests/reponses. They've been ported from the existing API and will need some checking. The [ipfs-grpc-server/README.md](https://github.com/ipfs/js-ipfs/blob/feat/add-grpc-server-and-client/packages/ipfs-grpc-server/README.md) has a rundown of the websocket communication protocol that was ported from [improbable-eng/grpc-web](https://github.com/improbable-eng/grpc-web). 5. Options as metadata When making a request, metadata is sent during the preamble - these take the form of a string identical to HTTP headers as the initial websocket message - I've used this mechanism to send the options for a given invocation. Notably these are not defined as a protocol buffer, just an unspecified list of simple key/value pairs - maybe they should be to ensure compatibility between implementations? This will be trivial in the implementation in the PR as it contains a server implementation too but to do it in go will require patching or forking the improbable gRPC proxy. 6. Errors as metadata Similar to the existing HTTP API, message trailers are used to send errors. Four fields are used to re-construct the error on the client side: | Field | Notes | | ----- | ----- | | grpc-status | 0 for success, 1+ for error | | grpc-message | An error message | | grpc-stack | A stack trace with `\n` delimited lines | | grpc-code | A string code such as `'ERROR_BAD_INPUT'` that may be used for i18n translations to show a message to the user in their own language | Similar to options these fields are unspecified, if a convention is not enough, perhaps they should be specified as a protobuf and the trailer sent as binary? 7. Streams When sending data as part of an `ipfs.add`, we send repeated messages that contain a path, a content buffer and an index. The index is used to differentiate between streams - path cannot be used as it could be empty. Only the first supplied `path` is respected for a given index. On the server separate input streams are created for each file being added. A file stream is considered closed when an unset or empty content buffer is received. Ultimately this will allow us to apply backpressure on a per-file basis and read from different file streams in parallel and asymmetrically based on the available server capacity. 8. Performance Observed performance pegs gRPC-web over websockets as similar to the HTTP Client with pretty much zero optimisation work performed 9. Security Browsers require TLS for all use of websocket connections to localhost. They do not require it for the loopback address, however, which this PR uses, though loopback means the traffic will not leave the local machine. The incoming requests start as HTTP requests so have a referer header and user agent so would follow the same restrictions as the existing HTTP API. Fixes #2519 Fixes #2838 Fixes #2943 Fixes #2854 Fixes #2864 [FormData]: https://developer.mozilla.org/en-US/docs/Web/API/FormData
- Loading branch information
1 parent
73d1436
commit ebc1dfa