Sync <-> Async bridge - REST Gateway for messages/event correlation

[peterbroadhurst]

Checklist:

• Webhooks implementation
• Front-end request/reply
• Docs

Current FireFly support using REST to submit messages to a remote party, then a pluggable interface for events to trigger in that party where the only implementation right no is WebSockets (long lived bi-directional pipe from a remote runtime).

This issue proposes adding two features, that combine to achieve the above pattern:

1. A REST interface on the front-end that is a sync<->async bridge

• Takes the input payload and packages it into a message+data
• Sends to a single participant, or group, as specified in the URL of the call
• Blocks the REST call until a response with a cid (correlation ID) matching the request ID is received as an event
• To begin with will be RPC style input/output POST rather than a full generated REST interface

2. A WebHook plugin for events

• Takes messages that match the subscription, and extracts the data
• Packages that data in a HTTP POST to a configured server
• Waits for the response to the HTTP call
• Packages that response with the correct cid into a message on the same group hash that the message arrived on

[peterbroadhurst]

A more detailed view of how the end-to-end flow works, with an example where the request is not pinned, and the response is pinned.

The blue parts show the code that would otherwise need to be in the application, that is abstracted away by using FireFly with this set of features.

Source (sequencediagram.org):

title Sync <-> Async Example: Unpinned request, pinned response

participant "Member1 App\nRequester" as app1
database "FireFly DB" as db1
boundary "FireFly Core\n(Multiparty GW API)" as firefly1
entity "Data\nExchange 1" as dx1

entity "Blockchain" as blockchain

entity "Data\nExchange 2" as dx2
boundary "FireFly Core\n(Multiparty GW Proxy)" as firefly2
database "FireFly DB" as db2
participant "Member2 App\nResponder" as app2

app1->firefly1: POST /call/tag/my-request

activate firefly1 #20639b
activate firefly2 #20639b
firefly1-->firefly1: Sync<->Async\nsetup for reply
firefly1<->db1: Store\n"my-request"\nrequest
group Unpinned request
firefly1->dx1: Send message
dx1-->dx2: Secure private transfer
dx2->firefly2: New message
firefly2<->db2: Store\n"/my-request"\nrequest [EVENT]
end
firefly2<->firefly2: Match subscription\non "webhook"\ntransport
firefly2->app2: POST webhook "/my-request"
deactivate firefly1
deactivate firefly2

group Responder app
app2<->app2: Awesome app\nprocessing
end

app2->firefly2: POST [200]\n(with data)
activate firefly2 #20639b
activate firefly1 #20639b
firefly2<->db2: Store\n"my-response"\nresponse
group Pinned response
firefly2->dx2: Send message
firefly2->blockchain: Pin response\nto blockchain
group The order of these events is non-deterministic
dx2-->dx1: Secure private transfer
dx1-->firefly1: New message
firefly1<-->db1: Interim DB updates 
blockchain-->firefly1: Detect pin
firefly1<-->db1: Interim DB updates 
end
firefly1->firefly1: aggregate pin\nand data
firefly1<->db1: Store confirmed\n"my-response"\nresponse [EVENT]
end
firefly1-->firefly1: Sync<->Async\ncorrelation with\nrequest msg
firefly1->app1: POST [200]\n(with data)
deactivate firefly2
deactivate firefly1