Network transport specification draft

[dkuleshov]

Hi,

Currently in Che there is a couple of solutions to pass data over network between client (IDE, UD, etc) and server (workspace-agent, exec-agent, workspace-master) or between servers (workspace-agent to workspace-master, etc.). And the problem here is that there is no definition of how and when to use one or another approach. Right now there are no clear instructions on when we should use REST or JsonRPC, how requests should be generated and responses should be treated. It may be difficult for a developer to understand why in some component we use Json RPC while in another - REST, while obviously it should be as transparent as possible.

So here I will try to make some kind of a specification draft for the most common cases of data transmission and remote procedures executions that we practice in Che. Now follows the excerpt from the original document.

Motivation:
To be able to transparently use existed REST APIs for async requests (requests with potentially long response or response with big output, like logs). So it will work as normal REST calls for one services and “hybrid” way (REST + JSON RPC) for others.

Network data transmissions are specified by a couple of factors:

• They may be synchronous or asynchronous
• The may have a single response or multiple responses, that come over time

We use REST protocol for synchronous requests and Json RPC for asynchronous. It is important to mention that in general there is no limitation to use Json RPC for synchronous requests as well but it is encouraged not to do that to have more clear picture: if we’re running REST - it is always synchronous, while Json RPC - is always asynchronous. It is encouraged but not strictly required, there obviously may be exceptions based on real life use-case requirements (e.g. LSP requires Json RPC).

For some cases there is a need in multiple responses that may come within arbitrary periods of time (e.g. for project import we need to initialize a procedure but also to track it’s progress), so here we use a tricky combination: there is an initialization request, acceptance response and responses containing business data. The initialization request and acceptance response may be of REST or Json RPC protocol, while multiple over time responses are always Json RPC request/notification (mostly notifications) because of obvious limitations of REST/HTTP stack. To make it more clear let us take a closer look at a workflow for such cases:

• Make a Json RPC/REST request
• Receive asynchronous or synchronous response that basically may mean:
  • Request is valid
  • Prerequisite conditions are met
  • Requested procedure is accepted for execution
• Receive asynchronous Json RPC notifications

So eventually we have a combination of four workflows:

• Synchronous single request and single response (REST).
  • The most simple use-case that should be used where we have simple data flow operations (e.g. post file content, get get content, etc.). Worth mentioning that for some specific cases it is the only valid option, for example many GWT operations in the way we use them are REST oriented (e.g. file upload).
• Asynchronous single request and single response (Json RPC).
  • Should be used where an operation should be run asynchronously, there is a requirement of using Json RPC (e.g. LSP) or it ideologically fits better than REST.
• Synchronous single request and multiple responses over time (REST). This is represented by a combination of REST request/response pair and Json RPC notification.
  • Should be used when we need to receive continuous notifications while acceptance response can be transmitted synchronously (e.g. validation, preprocessing happens fast).
• Asynchronous single request and multiple responses over time (Json RPC).
  • Should be used when we need to receive continuous notifications while acceptance response will be transmitted asynchronously (e.g. validation, preprocessing happens slow or requires additional preconditions to be met).

Last thing I would like to emphasize is that this is not something new from technical perspective, we're already using REST and Json RPC in some similar way however we can't say we have a solid well specified and generalized network data transmission workflow.

[garagatyi]

Nice specification, Dmytro!
As far as I understand the third workflow is a combination of REST request with REST response and Json RPC notification. Am I right? If so, can you make it more clear in the part that describes it? I feel that it may be not clear enough. E.g.:

- Synchronous single request and multiple responses over time (REST + Json RPC).
  - Should be used when we need to receive continuous notifications (Json RPC) while acceptance response can be 
    transmitted synchronously (REST), e.g. validation, preprocessing happens fast.

[dkuleshov]

@garagatyi

As far as I understand the third workflow is a combination of REST request with REST response and Json RPC notification. Am I right?

Absolutely right! Actually it is described in the original document in the part that is not mentioned in this issue. I agree that it is important thing that should be clarified here in the description so I will fix that ASAP.

[che-bot]

Issues go stale after 180 days of inactivity. lifecycle/stale issues rot after an additional 7 days of inactivity and eventually close.

Mark the issue as fresh with /remove-lifecycle stale in a new comment.

If this issue is safe to close now please do so.

Moderators: Add lifecycle/frozen label to avoid stale mode.