Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support GRPC as transport #5764

Open
Totktonada opened this issue Jan 29, 2021 · 1 comment
Open

Support GRPC as transport #5764

Totktonada opened this issue Jan 29, 2021 · 1 comment
Labels
feature A new functionality in design Requires design document Initiative
Milestone

Comments

@Totktonada
Copy link
Member

Totktonada commented Jan 29, 2021

Create a module (built-in? external?) that will provide ability to define an GRPC protocol/endpoint that will be served using user defined Lua function(s). I see the following subtasks:

  1. Parse and compile a GRPC protocol/endpoint description, which is given as a Lua string (at least assume the compilation step in the API to have ability to implement it later).
  2. Parse a GRPC request and validate it against given GRPC protocol/endpoint description. The result of parsing / validation should be either a meaningful error or a request in the Lua format.
  3. Provide ability to encode a GRPC response from a Lua value with validation against given protocol/endpoint definition.
  4. Consider providing a helper for using the module together with the tarantool/httpng module: a way to acquire a handler for the http server for a particular protocol/endpoint that will be served by user defined Lua function(s).

Many details are to be investigated and defined better. That is just my very high level vision.

BTW, we can implement a protobuf schema / binary format support as a separate module. Also maybe it is good to start from just protobuf, because GRPC is based on it (AFAIK).


The original wording describes various tasks around GRPC: expose iproto via GRPC, expose a user defined endpoint, connect to an external service. We decided to shrink the scope of this task and concentrate of the second point. See the original description below (it can be used as base for future issues).

The original description

There are several major goals:

  1. Use GRPC as a transport to access spaces and functions exposed via our native binary protocol (iproto). Popular programming languages usualy have GRPC libraries and so it should simplify development of connectors.
  2. Define an application / a module specific GRPC endpoints.
  3. Connect to external services from tarantool using GRPC.

In my understanding, the second and third points assumes that we should provide ability to complile a protocol definition in runtime, otherwise it will not convenient to use from a Lua application / module.

Only the first point is the definition of done of the issue, however the overall design should foresee reusage of significant code parts.

We plan to reuse our future H2O based HTTP/2 server.

@Mons said that he has a vision how to better implement GRPC support for Lua.

@Totktonada Totktonada added feature A new functionality in design Requires design document labels Jan 29, 2021
@Totktonada Totktonada added this to the 2.9.1 milestone Jan 29, 2021
Korablev77 added a commit that referenced this issue Feb 12, 2021
@kyukhin kyukhin added the tmp label Feb 12, 2021
@Totktonada
Copy link
Member Author

Updated the description:

  • Shrunk the task scope according to a @Mons's input.
  • Wrote some high-level subtasks to give an idea what we want to achieve.

I left the estimation as is, because it anyway looks as the large task, at least with my little knowledge around GRPC.

@kyukhin kyukhin removed the tmp label Apr 22, 2021
@kyukhin kyukhin modified the milestones: 2.9.1, 2.10.1 Apr 22, 2021
@kyukhin kyukhin modified the milestones: 2.10.1, backlog Aug 20, 2021
@kyukhin kyukhin removed prio8 labels Dec 9, 2021
@kyukhin kyukhin modified the milestones: backlog, wishlist Dec 30, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature A new functionality in design Requires design document Initiative
Projects
None yet
Development

No branches or pull requests

4 participants