Consider backwards-compatible extension for returning of streams

[dazraf]

Firstly, congratulations on the spec! This is a big step forward.

This issue is a proposal to enable a service to return streamed responses.

It's reasonably commonplace for services to expose methods that return a (potentialy infinite) stream of objects. These can be used to stream notifications of transactions, for example.

Can we consider a backwards compatible extension to JSON-RPC to support streamed responses for streaming transports (e.g. http/2, websockets, or overlays like socket.io or sockjs).

An example of such an extension is available here:
https://gitlab.com/bluebank/braid/blob/master/braid-core/README.md#invocation-protocol

Thank you!

[shanejonas]

I really like the idea of streaming this way as opposed to eth_subscribe or swagger callbacks as it lends well to async generators, hooks, and rx js observer style patterns, and changes very little.

This might is a good use case for Specification Extensions. you can use x- to define your own extension:

for example: x-stream: true

 "methods": [
    {
      "name": "list_pets",
      "summary": "List all pets",
      "x-stream": true,
      "tags": [
        {
          "name": "pets"
        }
      ],
      "params": [
        {
          "name": "limit",
          "description": "How many items to return at one time (max 100)",
          "required": false,
          "schema": {
            "type": "integer"
          }
        }
      ]
     // ...snip

Experimenting with the extensions seems like a good way to test out the viability of a proposal like this.

[dazraf]

Thanks! I'm glad the idea resonates with you. I like the idea and design of the protocol extensions aspect of the schema.

One thing I should add is that I'm one of the maintainers of braid. There has been some discussion about moving it's bespoke streaming protocol to be compliant with OpenRPC.

In practical terms, should I create a standalone project that details the extensions and provides a reference implementation? Is the best place to post a link to this for feedback here on this issue?

Btw, there are additions to the protocol that I'm working on for back-pressure.

[BelfordZ]

@dazraf thanks for taking a look at OpenRPC!

As @shanejonas mentioned, specification extensions is a great method through which to extend, but also propose additional functionality to the specification.

A great example of using specification extensions can be seen with OpenAPI + AWS (its fundamentally the same concept). This is a good example of extensions which would never make sense to add directly to open api, but allows for open api documents to specify additional fields necessary for AWS API Gateway to function.

Alternatively, A feature such as examples, servers, any open rpc field really, could have been first implemented and prototyped as a vendor extension, and only once vetted would it be 'promoted' from vendor extension to part of a newer version of the spec.

At the moment, we have yet to dive too deep into vendor extensions (the project is still relatively young). That said, we don't have any recommended format to describe a vendor extension yet. You are a pioneer :).

Would love to hear about your back-pressure stuff. Happy to discuss it with you in a separate issue :).

Also, please do link back what you come up with! I'm very excited to have others contributing!

[dazraf]

Awesome, thank you! I will update this issue with progress.

[BelfordZ]

@dazraf I really like this idea. Seems like it would be a great addition to the JSON-RPC protocol.

Maybe we should adopt the JSON-RPC 2.0 spec, and make a 3.0 spec that includes this idea.

:)

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[BelfordZ]

@dazraf the desire to adopt JSON-RPC 2.0 and create a 2.1 that incorporates your streaming feature is BURNING INSIDE OF ME.

Thoughts?

[dazraf]

Hi Belford,
I really want to start work on this as soon as possible. I've been working on Braid - the project that provides automatic open access to apps and also specially, Corda DLT. On master we've just got to a point that we can generate a rich enough open-api v3 schema. R3 are keen for us to do both open-api and open-rpc. open-api was the slightly higher priority. Now that we've almost reached a release version, I can do more work on open-rpc. First things I thought I can first donate a RFC and a comprehensive sample schema file for the team to review. Following feedback and assuming it's accepted, I can help to build a nominal set of tests around it and work on the implementation. How does that sound?

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.