Notification subsystem

[roman-khimov]

Overview

We need a notification system for smart contract developers. At the same time it might be interesting to have something similar to plugin system of the C# node (especially in the form described in neo-project/neo#1225), although its IP2PPlugin and IStoragePlugin can directly affect node behaviour and plugins have synchronous model while we're mostly concerned with asynchronous notifications about chain state.

Requirements

Primary requirements:

• networking-based
• asynchronous
• allowing subscription to a set of events (including basic server-side event filtering)
• extensible

Primary usage model:

• smart contract backend service listening for chain events and making some additional requests based on that
• the service is expected to be using RPC protocol already for its purposes

General solution

As we already have JSON-RPC support in our node it seems to be a good idea to extend it for notification service because it allows for some code reuse and makes our RPC client be the sole interface for interacting with node in various ways (it's expected that event listener might make additional requests to node based on events received). Still there are some caveats:

• notifications put additional burden on the server and thus one may want to restrict this service availability
  At the same time smart contract developers are not expected to be using public RPC nodes and a private node can be restricted in any way by regular network firewalling. Still we should make this subsystem optional and future synchronous call extensions should also add an additional option. We could make it a separate service on a different port, but it doesn't seem to be a good option for now because of expected usage model.
• sending server-side notifications requires some generic messaging layer like websockets
  Which probably is OK given that we're starting to extend our existing JSON-RPC interface, but note that it differs from websockets usage in Neo 3 where P2P protocol is layered above them.

Initial events list (all about chain processing):

• new block added
  Contents: block.
  Filters: none for Neo 2, primary index for Neo 3.
• new transaction in the block
  Contents: transaction.
  Filters: type and verification script hash for Neo 2, transaction Sender and Cosigner for Neo 3.
• notification generated during execution
  Contents: container hash, contract script hash, stack item.
  Filters: contract script hash. Note that for Neo 2 only transaction can be container, but for Neo 3 block itself can also produce some notifications.
• transaction executed
  Contents: application execution result.
  Filters: VM state; transaction Sender and Cosigner for Neo 3.

Filters use conjunctional logic.

Ordering and persistence guarantees:

• new block is only announced after its processing is complete and the chain is updated to the new height
  No disk-level persistence guarantees are given.
• new in-block transaction is announced after block processing, but before announcing the block itself
• transaction notifications are only announced for successful transactions
• all announcements are being done in the same order they happen on the chain
  At first transaction execution is announced, then followed by notifications generated during this execution, then followed by transaction announcement (for Neo 2). Transaction announcements are ordered the same way they're in the block.
• unsubscription may not cancel pending, but not yet sent events

Other notes:

• subscription process should include some subscription IDs for client to be able to unsubscribe from certain events streams
• subscription is only valid for connection lifetime, no long-term client identification is being made
• if an event matches several subscriptions, it's only sent once for one client (this is a part of the reason why no subscription ID is sent with notifications)
• during high server load or when dealing with slow client connections the server can throw away some events that can't fit into reasonably-sized event queue. When it's being done missed events counter is being increased for given subscription ID and an event is generated that will be sent to the client containing the number of missed events from the last such event.

Extensions considered

Possible future extensions:

• transaction addition/removal to/from the mempool (similar to IMemoryPoolTxObserverPlugin)
• making log events available (similar to ILogPlugin)
  Can be dangerous as logs might contain some sensitive data about node configuration. At the same time it might be useful if restricted properly.
• P2P (especially consensus) messages passing (similar to IP2PPlugin)
  Might be useful for making neo-go compatible with things like Red4Sec/neo-resilience, but not a priority for now and also note that IP2PPlugin can affect message processing, so it's not just an event subsystem.
• extending filtering capabilities (like matching something from a set of hashes)
  Doesn't seem to be a priority because the primary target audience for this system is smart contract developers and most of the time they're dealing with one or two contracts/addresses.

Things we probably won't do (but they're used in neo-resilience):

• OnCommit() and ShouldThrowExceptionFromCommit() from IPersistencePlugin
ShouldThrowExceptionFromCommit() isn't really applicable to our model and the only value in OnCommit() is that it synchronously exposes the new state before the chain moves on, I think block notification is a good enough alternative.
• OnPersist() callback implementation from IPersistencePlugin
  Doesn't seem to be very useful as it can be substituted by subscribing to our transaction execution and block notifications.

Protocol specification draft

Websocket connection

Connection switch is done with a GET request to /ws path. Standard protocol
upgrade happens after that and all subsequent messaging is being done via this
channel (including regular RPC calls).

Subscription management

subscribe method

Parameters: event stream name, stream-specific filter rules hash (can be
omitted if empty).

Recognized stream names:

• block_added
  No filter parameters for Neo 2, primary with integer value for Neo 3.
• transaction_added
  Neo 2: type as a string containing standard transaction types
  (MinerTransaction, InvocationTransaction, etc); verifier as a string
  containing hex-encoded Uint160. Neo 3: sender and cosigner fields
  containing string with hex-encoded Uint160.
• notification_from_execution
contract field containing string with hex-encoded Uint160.
• transaction_executed
  Both versions: state containing HALT or FAULT string for successful
  and failed executions respectively. Neo 3: sender and cosigner fields
  with the same semantics as for transaction_added.

Response: returns subscription ID (string) as a result.

Example request (subscribe to notifications from contract
0x6293a440ed80a427038e175a507d3def1e04fb67 generated when executing
transactions):

{
  "jsonrpc": "2.0",
  "method": "subscribe",
  "params": ["notification_from_execution", {"contract": "6293a440ed80a427038e175a507d3def1e04fb67"}],
  "id": 1
}

Example response:

{
  "jsonrpc": "2.0",
  "id": 1,
  "result": "55aaff00"
}

unsubscribe method

Parameters: subscription ID as a string.

Response: boolean true.

Example request (unsubscribe from "55aaff00"):

{
  "jsonrpc": "2.0",
  "method": "unsubscribe",
  "params": ["55aaff00"],
  "id": 1
}

Example response:

{
  "jsonrpc": "2.0",
  "id": 1,
  "result": true
}

Events

Events are sent as JSON-RPC notifications from the server with method field
being used for notification names. Notification names are identical to stream
names described for subscribe method.

Verbose responses for various structures like blocks and transactions are used
to simplify working with notifications on client side. Other models
considered:

• passing non-verbose hex blobs
  Using them would require deserializing, which is not a problem for neo-go
  client, but can be problematic in other circumstances
• pushing only hashes
  Which would simplify server's life, but we're suggesting that if there is a
  subscription, then the client really is interested in the contents and so
  requiring an additional request to get the real data reduces overall
  efficiency in the end.

block_added notification

As a first parameter (params section) contains block converted to JSON
structure (similar to verbose getblock response). No other parameters are
sent.

Example:

{
  "jsonrpc": "2.0",
  "method": "block_added",
  "params": [
   {
    "hash": "0x4c1e879872344349067c3b1a30781eeb4f9040d3795db7922f513f6f9660b9b2",
    "size": 686,
    "version": 0,
    "previousblockhash": "0x53ce15a53a184faa058be9008ece887d7f0b30a459342ce42753f73f9ed390e9",
    "merkleroot": "0x9c909e1e3ba03290553a68d862e002c7a21ba302e043fc492fe069bf6a134d29",
    "time": 1476834896,
    "index": 10000,
    "nonce": "aa2c0fd5dc445445",
    "nextconsensus": "APyEx5f4Zm4oCHwFWiSTaph1fPBxZacYVR",
    "script": {
      "invocation": "405e31eb19b1feaeb27c3a5b95f568b9b256fefe0ea61f6296eb8af836c29597617fe81d23a8bf66309000e4c7568b7f43560f61e4ee6cd1f78a2a42f50a5008c240ccf73ce9f7f810273730bdfc786d346086a697cc06239e88e040ed2ec0583c7dbb6eccb8b8a74afbd75cfbaff06c051b7e82abe65f96f50a1673e1536f91a3d540618e43cce18c7c91b54b2a5e44ba1e4a71a8dd0af0ec95c8c4f05343e66129b150057a5f79399a92eda1226fddd254702\
ffc682309787ab241509b2244e410334070a5ac50d897bf39f98780f79fb1a2416c41dc2e202b4ad797bd0c70e2b57f1157c4ff5551ec6df58bec6244dc72a3f25cd1836e8cdd4c0d8c2e5ba7e2d8859b40ae80743c9a2a8e154671eb156266971439a9017e96ea072c848287a71b2d6a99a67ba50fc7935a6de4d8884794291fc6cebd77158954ef03b10d5d0a30b52bc9",
      "verification": "552102486fd15702c4490a26703112a5cc1d0923fd697a33406bd5a1c00e0013b09a7021024c7b7fb6c310fccf1ba33b082519d82964ea93868d676662d4a59ad548df0e7d2102aaec38470f6aad0042c6e877cfd8087d2676b0f516fddd362801b9bd3936399e2103b209fd4f53a7170ea4444e0cb0a6bb6a53c2bd016926989cf85f9b0fba17a70c2103b8d9d5771d8f513aa0869b9cc8d50986403b78c6da36890638c3d46a5adce04a2102ca0e27697b9\
c248f6f16e085fd0061e26f44da85b58ee835c110caa5ec3ba5542102df48f60e8f3e01c48ff40b9b7f1310d7a8b2a193188befe1c2e3df740e89509357ae"
    },
    "tx": [
      {
        "txid": "0x9c909e1e3ba03290553a68d862e002c7a21ba302e043fc492fe069bf6a134d29",
        "size": 10,
        "type": "MinerTransaction",
        "version": 0,
        "attributes":[],
        "vin":[],
        "vout":[],
        "sys_fee": "0",
        "net_fee": "0",
        "scripts":[],
        "nonce": 3695465541
      }
    ],
    "confirmations": 0,
   }
  ]
}

transaction_added notification

In the first parameter (params section) contains transaction converted to JSON
(similar to verbose getrawtransaction response). No other parameters are
sent.

{
  "jsonrpc": "2.0",
  "method": "transaction_added",
  "params": [
   {
    "txid": "0xf4250dab094c38d8265acc15c366dc508d2e14bf5699e12d9df26577ed74d657",
    "size": 262,
    "type": "ContractTransaction",
    "version": 0,
    "attributes":[],
    "vin": [
      {
        "txid": "0xabe82713f756eaeebf6fa6440057fca7c36b6c157700738bc34d3634cb765819",
        "vout": 0
      }
     ],
     "vout": [
      {
        "n": 0,
        "asset": "0xc56f33fc6ecfcd0c225c4ab356fee59390af8560be0e930faebe74a6daff7c9b",
        "value": "2950",
        "address": "AHCNSDkh2Xs66SzmyKGdoDKY752uyeXDrt"
      },
      {
        "n": 1,
        "asset": "0xc56f33fc6ecfcd0c225c4ab356fee59390af8560be0e930faebe74a6daff7c9b",
        "value": "4050",
        "address": "ALDCagdWUVV4wYoEzCcJ4dtHqtWhsNEEaR"
       }
    ],
    "sys_fee": "0",
    "net_fee": "0",
    "scripts": [
      {
        "invocation": "40915467ecd359684b2dc358024ca750609591aa731a0b309c7fb3cab5cd0836ad3992aa0a24da431f43b68883ea5651d548feb6bd3c8e16376e6e426f91f84c58",
        "verification": "2103322f35c7819267e721335948d385fae5be66e7ba8c748ac15467dcca0693692dac"
      } 
    ],
    "blockhash": "0x9c814276156d33f5dbd4e1bd4e279bb4da4ca73ea7b7f9f0833231854648a72c",
    "confirmations": 0,
    "blocktime": 1496719422
   }
  ] 
}

notification_from_execution notification

Contains three parameters: container hash (Uint256 encoded in a string),
contract script hash (Uint160 encoded in a string) and stack item (encoded the
same way as state field contents for notifications from getapplicationlog
response).

Example:

{
  "jsonrpc": "2.0",
  "method": "notification_from_execution",
  "params": [
    "0x92b1ecc0e8ca8d6b03db7fe6297ed38aa5578b3e6316c0526b414b453c89e20d",
    "0x78e6d16b914fe15bc16150aeb11d0c2a8e532bdd",
    {
      "type": "Array",
      "value": [
        {
          "type": "ByteArray",
          "value": "7472616e73666572"
        },
        {
          "type": "ByteArray",
          "value": "d086ac0ed3e578a1afd3c0a2c0d8f0a180405be2"
        },
        {
          "type": "ByteArray",
          "value": "002ba7f83fd4d3975dedb84de27345684bea2996"
        },
        {
          "type": "ByteArray",
          "value": "0065cd1d00000000"
        }
      ]
    }
  ]
}

transaction_executed notification

Contains the same result as from getapplicationlog method in the first
parameter and no other parameters.

Example:

{
  "jsonrpc": "2.0",
  "method": "transaction_executed",
  "params": [
    {
      "txid": "0x92b1ecc0e8ca8d6b03db7fe6297ed38aa5578b3e6316c0526b414b453c89e20d",
      "executions": [
        {
          "trigger": "Application",
          "contract": "0x6ec33f0d370617dd85e51d31c483b6967074249d",
          "vmstate": "HALT",
          "gas_consumed": "2.912",
          "stack": [
            {
              "type": "Integer",
              "value": "1"
            }
          ],
          "notifications": [
            {
              "contract": "0x78e6d16b914fe15bc16150aeb11d0c2a8e532bdd",
              "state": {
                "type": "Array",
                "value": [
                  {
                    "type": "ByteArray",
                    "value": "7472616e73666572"
                  },
                  {
                    "type": "ByteArray",
                    "value": "d086ac0ed3e578a1afd3c0a2c0d8f0a180405be2"
                  },
                  {
                    "type": "ByteArray",
                    "value": "002ba7f83fd4d3975dedb84de27345684bea2996"
                  },
                  {
                    "type": "ByteArray",
                    "value": "0065cd1d00000000"
                  }
                ]
              }
            }
          ]
        }
      ]
    }
  ]
}

Draft client-side API

// All subscription functions accept parameters via pointers to allow passing nil
// as 'no filter' value. They return subscription ID or error.
func (c *Client) SubscribeForNewBlocks(primary *int) (string, error)
func (c *Client) SubscribeForNewTransactions(txType *transaction.TXType, signer *util.Uint160) (string, error) // Neo 2 version
func (c *Client) SubscribeForNewTransactions(sender *util.Uint160, cosigner *util.Uint160) (string, error) // Neo 3 version
func (c *Client) SubscribeForExecutionNotifications(contract *util.Uint160) (string, error)
func (c *Client) SubscribeForTransactionExecutions(state string) (string, error) // Neo 2 version
func (c *Client) SubscribeForTransactionExecutions(state string, sender *util.Uint160, cosigner *util.Uint160) (string, error) // Neo 3 version

// Unsubscription may close event channel if it's the last subscription.
func (c *Client) Unsubscribe(id string) error
func (c *Client) UnsubscribeFromAll()

// Can return error if there are no subscriptions. You can read events from returned channel.
// One channel (instead of per-type queues) is used to ensure correct overall event sequence.
// name can be substituted with client-internal ID.
func (c *Client) GetEventStream() (<-chan struct{name string, value interface{}}, error)

[roman-khimov]

Done.