Allow API client to control the server output

[ghost]

Why

Clients might have different requirements about the data they want from our API Server:

• Some might not want to be fed the history of previous messages when they connect
• Some might want to send/receive transactions in JSON or CBOR

What

• Provide a way for the client to specify what kind of output they want to see when they connect to API Server
• Server configures the connection according to the options.
• Client can control:
  • Configurable transaction format
  • Configurable latching of history

How

We will use websocket path to decide on these two items:

• If a path contains QueryParam history and it's value is 1 then the server will serve complete history
• If a path contains QueryParam tx-output and it's value is cbor then the tx should be displayed as cbor

Example: ?history=0&tx-output=cbor

Related issues:

• Configurable latching of message history on client connection #371
• Output transactions to client in CBOR instead of JSON  #379

[ch1bo]

Another idea by @KtorZ is to use the sub-protocol negotiation in the WebSocket standard.

[matiwinnetou]

+1, this is something that we are observing when building application build on Hydra and this would help.

We are trying to build something like hydra-java-client library and already at the beginning hitting issues described by @abailly-iohk .

Something like request message id and response message id with lack of history and protocol similar to what ogmios is using is something that we would like.

History could be optionally provided but client needs to be interested in this (somehow).

[v0d1ch]

I took a look at related code and these are my findings:

• We can utilize either websocket subprotocols (or path) to make a decision on the api server level what content should we serve
• It is not that easy to pick appropriate json instance we want to serve
• There is a simple monkey-patch solution where we could just overwrite the fields we want in already prepared json
• For replaying of history the easiest would be to introduce a new argument to hydra-node that signals this

So overall I would say the work is pretty easy if we go with the simple solution which is not really ideal but can work.
A proper solution can turn out to be a bit of a rabbit hole because of how things are setup in code (IsTx and friends) and one can spend a lot of time golfing with the types but that might lead to better situation in our code.

Personally I would do a dirty solution to have something working and then improve later.

cc @ch1bo

[ch1bo]

For replaying of history the easiest would be to introduce a new argument to hydra-node that signals this

Not sure if this is an option. I can imagine that some clients of the hydra-node do require the history to work (e.g. hydra-tui) while others don't want it. So IMO it must be per client / connection configurable.

Maybe optional query parameters on the websocket path could work? For example, wss://host:port/?history=0&tx-output=cbor could be interpreted as a websocket which does not send the history and output transactions as CBOR.

[v0d1ch]

We are loading the history before running the server and then we share the TChan for reading/writing https://github.com/input-output-hk/hydra/blob/master/hydra-node/src/Hydra/API/Server.hs#L79
I'll work on a solution to make the path decide what we want to serve from the api.

[ch1bo]

@v0d1ch Yeah that's no problem. We need to store/load the history nontheless (in case one client requests it). We only need to make the forwardHistory optional.

[v0d1ch]

Ok since things have changed from the time this idea was created I am keeping the old wording here just as a reference and I'll update the original to be inline with the actual changes I plan on doing.

## Why

Clients might have different requirements about the data they want from our API Server:
* Some might not want to be fed the history of previous messages when they connect
* Some might want to send/receive transactions in JSON or CBOR

## What

* Provide a _negotiation phase_ when a client connect to API Server
* Server sends `Greeting` message as before, possibly with a version
* Client sends a `Greeting` message with options set
* Server configures the connection according to the options.
* _(optional)_: Server sends _Available options_ in its initial `Greeting` message
* Then, we want to use this negotiation mechanism in the two example configurations we encountered so far:
  + Configurable transaction format
  + Configurable latching of history

## How

- [ ] Create a negotiation mechanism
- [ ] #371
- [ ] #379