Add RPC subscriptions

blockworks-foundation · Sep 22, 2023 · 08239ef · 08239ef
1 parent 357e63a
commit 08239ef
Show file tree

Hide file tree

Showing 2 changed files with 141 additions and 75 deletions.
diff --git a/docs/architecture.md b/docs/architecture.md
@@ -5,30 +5,43 @@
 ### Validator
 Manage real time generated data by the validator.
 
-Implements these calls:
+Implements these RPC calls:
  * GetVoteAccounts
  * getLeaderSchedule
  * getEpochInfo
  * getSlot
  * getSignatureStatuses at process commitment.
  * new tx sent: call by the SendTx module when a Tx is processed by sendTx module.
 
-Provide these subscription:
+#### Subdomain Validator subscription
+Implements these RPC subscriptions
+ * programSubscribe
+ * slotSubscribe
+ * blockSubscribe
+ * transactionSubscribe
+ * logsSubscribe
+ * signatureSubscribe
+ * slotsUpdatesSubscribe
+ * voteSubscribe
+#### Sub domain Cluster
+Manage cluster information.
+
+Implement the call: getClusterNodes
+
+Provide the subscription: cluster info.
+
+#### Inter module messages notification (Stream)
+
+Provides these inter module message notification:
  * Full block
  * BLock info
  * Slots
  * Leader schedule
- * sent Tx at confirmed and / or finalized: notify when a Tx sent is confirmed or finalized. 
+ * sent Tx at confirmed and / or finalized: notify when a Tx sent is confirmed or finalized. Use by RPC transactionSubscribe
 
 A new subscription is added: Sent Tx confirmed/ finalized. SendTx module send Tx signature to the Validator domain and when a Tx sent is confirmed (or finalized), it is notified on this subscription.
 
 It avoids to call getSignatureStatuses in a pull mode.
-#### Sub domain Cluster
-Manage cluster information.
-
-Implement the call: getClusterNodes
-
-Provide the subscription: cluster info.
 
 ### SendTx
 Manage the whole send Tx process. Represent the current Lite RPC process.
@@ -37,9 +50,9 @@ Implements the sendTx call.
 
 
 ### History
-Manage history function like getBlocks.
+Manage history function like getBlock/getBlocks/getTransaction.
 
-A special use case is the getSignatureStatuses because on process its the Validator domain that provide tha data.
+A special use case is the getSignatureStatuses because on process its the Validator domain that provide the data.
 
 ### RPC
 It's an entry point for all call and dispatch the call to the right function.
@@ -48,12 +61,6 @@ It's an entry point for all call and dispatch the call to the right function.
 
 ```mermaid
 flowchart TD
-    subgraph Send Tx Domain
-        SendTx("SendTx Domain
-
-              send_transaction()")
-    end
-
     subgraph History Domain
         History("History Domain
 
@@ -67,32 +74,50 @@ flowchart TD
         Storage["2 epoch Storage"]
     end
 
+    subgraph Send Tx Domain
+        SendTx("SendTx Domain
+
+              send_transaction()")
+    end
+
     subgraph Validator Host
         Validator["Solana Validator
             Validator process
             + GRPC Geyser"]
         
         Consensus("Validator Domain
-
+        RPC calls:
             getVoteAccounts()
             getLeaderSchedule()
             getEpochInfo()
             getSlot()...
         At process:
             getSignaturesForAddress()
             getSignatureStatuses()
-
               ")
-        Cluster("Cluster Domain
+        Subscription("Subscription Sub Domain
+            
+            programSubscribe
+            slotSubscribe
+            blockSubscribe
+            transactionSubscribe
+            logsSubscribe
+            signatureSubscribe
+            slotsUpdatesSubscribe
+            voteSubscribe
+              ")
+        Cluster("Cluster Sub Domain
             
            getClusterNodes()")
     end
     
 
     Validator-- "geyser data" -->Consensus
+    Validator-- "geyser data" -->Subscription
     Validator-- "Cluster info" -->Cluster
     Consensus-- "Block Info/Slot/Leader Schedule" -->SendTx
     Consensus-- "confirmed Tx" -->SendTx
+    Consensus-- "confirmed Tx" -->Subscription
     Cluster-- "Cluster info" -->SendTx
     Consensus-- "Full Block/Slot/Epoch" -->History
     History<-. "old data" .-> Faithfull
@@ -110,7 +135,6 @@ flowchart TD
     class Cluster greengray
 ```
 
-
 ## Interaction diagram
 
 ```mermaid
@@ -139,6 +163,11 @@ flowchart TD
               Manage realtime produced data
         by the validator")
 
+        Subscription("Subscription Sub Domain
+            
+            All RPC subscriptions
+              ")
+
         Cluster("Cluster Info
                [SubDomain]
         
@@ -158,11 +187,13 @@ flowchart TD
     Validator-- "geyser Stakes and Votes account Sub" -->Consensus
     Validator== "geyser getBlockHeight" ==>RPC
     Validator-- "geyser Cluster info Sub" -->Cluster
+    Validator-- "geyser Sub" -->Subscription
     Consensus<== "getVoteAccounts/getLeaderSchedule/getEpochInfo/getSlot" ==>RPC
     Consensus<== "At Process getSignaturesForAddress/getSignatureStatuses" ==>RPC
     Consensus-- "Block Info Sub" -->SendTx
     Consensus-- "Leader Schedule Sub" -->SendTx
     Consensus-- "Sent Tx confirmed Sub" -->SendTx
+    Consensus-- "Sent Tx confirmed Sub" -->Subscription
     Cluster-- "Cluster info Sub" -->SendTx
     Consensus-- "Full Block / Epoch Sub" -->History
     RPC== "SendTx" ==> SendTx
@@ -182,6 +213,7 @@ flowchart TD
     class SendTx sendtx
     class History redgray
     class Consensus consensus
+    class Subscription consensus
     class Cluster greengray
 ```
 
@@ -201,6 +233,8 @@ flowchart TD
         Cluster("Cluster Info")
             
         Consensus("Validator")
+
+        Subscription("Subscription")
     end
 
     Stream("Stream
@@ -213,6 +247,7 @@ flowchart TD
    
     Stream-- "[Slot, Leader Schedule, Block and Epoch info, Tx confirmed] sub" -->SendTx
     Stream-- "[Sent Tx] sub" -->Consensus
+    Stream-- "[Sent Tx] sub" -->Subscription
     Stream-- "[Full Block, Slot, Epoch info] sub" -->History
 
     classDef consensus fill:#1168bd,stroke:#0b4884,color:#ffffff
@@ -224,6 +259,7 @@ flowchart TD
     class SendTx sendtx
     class History redgray
     class Consensus consensus
+    class Subscription consensus
     class Cluster greengray
 ```
 
@@ -253,11 +289,13 @@ flowchart TD
     subgraph Validator Host1
         Validator1["Solana Validator"]
         Consensus1("Validator impl")
+        Subscription1("Subscription impl")
         Cluster1("Cluster impl")
     end
     subgraph Validator Host2
         Validator2["Solana Validator"]
         Consensus2("Validator impl")
+        Subscription2("Subscription impl")
         Cluster2("Cluster impl")
     end
 
@@ -270,6 +308,9 @@ flowchart TD
     RPC== "getVoteAccounts" ==>Consensus1
     RPC== "getVoteAccounts" ==>Consensus2
 
+    RPC== "Subscription" ==>Subscription1
+    RPC== "Subscription" ==>Subscription2
+
     RPC== "getBlock" ==>History1
     RPC== "getBlock" ==>History2
     RPC== "getBlock" ==>History3
@@ -287,9 +328,10 @@ flowchart TD
     class SendTx2 sendtx
     class Cluster1 greengray
     class Cluster2 greengray
+    class Subscription1 consensus
+    class Subscription2 consensus
     class Consensus1 consensus
     class Consensus2 consensus
     class History1 redgray
     class History2 redgray
     class History3 redgray
-```
diff --git a/docs/rpcv2.md b/docs/rpcv2.md
@@ -120,6 +120,30 @@ Method calls:
  - [getsupply](https://docs.solana.com/api/http#getsupply) account API but needed. not in geyser plugin
    need to add stake supply in the return response.
 
+#### Subscriptions
+Subscription is implemented in the validator domain because it corresponds to current processed data by the validator.
+
+Each method has its unsubscribe call.
+
+##### Subscription Mapped by a geyser subscription
+These calls map geyser subscription so they can be implemented by using a geyser subscription and a corresponding specific filter
+
+ - [programSubscribe](https://docs.solana.com/api/websocket#programsubscribe)
+ - [slotSubscribe](https://docs.solana.com/api/websocket#slotsubscribe)
+ - [blockSubscribe](https://docs.solana.com/api/websocket#blocksubscribe)
+ - [transactionSubscribe](https://github.com/solana-foundation/solana-improvement-documents/pull/69)
+
+
+##### Subscription with no current geyser subscription
+These calls doesn't correspond to a current geyser subscription and need a specific development that can be: 
+ * data transformation from geyser notification in the validator module.
+ * a new subscription to geyser plugin.
+
+ - [logsSubscribe](https://docs.solana.com/api/websocket#logssubscribe)
+ - [signatureSubscribe](https://docs.solana.com/api/websocket#signaturesubscribe)
+ - [slotsUpdatesSubscribe](https://docs.solana.com/api/websocket#slotsupdatessubscribe)
+ - [voteSubscribe](https://docs.solana.com/api/websocket#votesubscribe)
+
 #### General update
 
 number like lamports that can be more than 48 bits should be in a string format to avoid number overflow. 
@@ -446,6 +470,59 @@ Algo:
 
 Call geyser plugin.
 
+### Cluster Sub domain
+Manage all data related to the solana validator cluster.
+
+##### Data
+ * Cluster: define at the beginning of the epoch all cluster's validator data (ip and port) 
+ * Leader schedule: define for all epoch slot the validator that will be leader.
+ * Epoch data: data related to the epoch.
+ * Vote account: Account data (staking) at the beginning of the epoch for all account that can vote for the blocks.
+
+##### Process
+##### Cluster storage
+Store all cluster data per epoch like for block processing module.
+
+##### Cluster processing
+Process epoch data to extract cluster data (ex: Total staking from the vote accounts).
+
+See the CLuster RPC call to have more details.
+
+###### getclusternodes
+    - Returns information about all the nodes participating in the cluster
+    - Parameters: None
+    - Result: The result field will be an array of JSON objects, each with the following sub fields:
+        * pubkey: <string> - Node public key, as base-58 encoded string
+        * gossip: <string|null> - Gossip network address for the node
+        * tpu: <string|null> - TPU network address for the node
+        * rpc: <string|null> - JSON RPC network address for the node, or null if the JSON RPC service is not enabled
+        * version: <string|null> - The software version of the node, or null if the version information is not available
+        * featureSet: <u32|null > - The unique identifier of the node's feature set
+        * shredVersion: <u16|null> - The shred version the node has been configured to use
+
+Sources: not provided by geyser plugin.
+
+In Solana the cluster data are stored in the ClusterInfo share struct 
+
+1) Geyser (currently not accepted by Solana) :
+  * a) propose the same get_cluster_nodes impl to get the cluster a any time
+  * b) notify at the beginning of the epoch. Change the call name, enough for us but perhaps not for every user: change notifier to add a ClusterInfo notifier like BlockMetadataNotifier for block metadata.
+    Need to change the plugin interface, add notify_cluster_at_epoch. We can notify every time the cluster info is changed but not sure Solana will accept (too much data).
+2) Use gossip and use the GossipService impl like the bootstrap::start_gossip_node() function. Need to use ClusterInfo struct that use BankForks to get staked_nodes.
+For node info it seems that the stake is not needed. The gossip process is:
+ * For shred data: crds_gossip::new_push_messages() -> crds::insert() that update peers with CrdsData::LegacyContactInfo(node). So by getting gossip message with LegacyContactInfo we can update the cluster node info.
+ * For peers data. 
+    GossipService start ClusterInfo::Listen()
+    -> ClusterInfo::listen()
+    -> ClusterInfo::run_listen()
+    -> ClusterInfo::process_packets()
+    -> ClusterInfo::handle_batch_pull_responses(
+    -> ClusterInfo::handle_pull_response()
+    -> CrdsGossip::filter_pull_responses()
+    -> crdsgossip_pull::filter_pull_responses()
+    -> crds::upserts() update the cluster table.
+
+
 #### History domain
 This domain include all function related to get past data of the blockchain.
 
@@ -676,58 +753,6 @@ This function need some evolutions with the current implementation:
     - can get the same signature twice (fork).
 
 
-### Cluster domain
-Manage all data related to the solana validator cluster.
-
-##### Data
- * Cluster: define at the beginning of the epoch all cluster's validator data (ip and port) 
- * Leader schedule: define for all epoch slot the validator that will be leader.
- * Epoch data: data related to the epoch.
- * Vote account: Account data (staking) at the beginning of the epoch for all account that can vote for the blocks.
-
-##### Process
-##### Cluster storage
-Store all cluster data per epoch like for block processing module.
-
-##### Cluster processing
-Process epoch data to extract cluster data (ex: Total staking from the vote accounts).
-
-See the CLuster RPC call to have more details.
-
-###### getclusternodes
-    - Returns information about all the nodes participating in the cluster
-    - Parameters: None
-    - Result: The result field will be an array of JSON objects, each with the following sub fields:
-        * pubkey: <string> - Node public key, as base-58 encoded string
-        * gossip: <string|null> - Gossip network address for the node
-        * tpu: <string|null> - TPU network address for the node
-        * rpc: <string|null> - JSON RPC network address for the node, or null if the JSON RPC service is not enabled
-        * version: <string|null> - The software version of the node, or null if the version information is not available
-        * featureSet: <u32|null > - The unique identifier of the node's feature set
-        * shredVersion: <u16|null> - The shred version the node has been configured to use
-
-Sources: not provided by geyser plugin.
-
-In Solana the cluster data are stored in the ClusterInfo share struct 
-
-1) Geyser (currently not accepted by Solana) :
-  * a) propose the same get_cluster_nodes impl to get the cluster a any time
-  * b) notify at the beginning of the epoch. Change the call name, enough for us but perhaps not for every user: change notifier to add a ClusterInfo notifier like BlockMetadataNotifier for block metadata.
-    Need to change the plugin interface, add notify_cluster_at_epoch. We can notify every time the cluster info is changed but not sure Solana will accept (too much data).
-2) Use gossip and use the GossipService impl like the bootstrap::start_gossip_node() function. Need to use ClusterInfo struct that use BankForks to get staked_nodes.
-For node info it seems that the stake is not needed. The gossip process is:
- * For shred data: crds_gossip::new_push_messages() -> crds::insert() that update peers with CrdsData::LegacyContactInfo(node). So by getting gossip message with LegacyContactInfo we can update the cluster node info.
- * For peers data. 
-    GossipService start ClusterInfo::Listen()
-    -> ClusterInfo::listen()
-    -> ClusterInfo::run_listen()
-    -> ClusterInfo::process_packets()
-    -> ClusterInfo::handle_batch_pull_responses(
-    -> ClusterInfo::handle_pull_response()
-    -> CrdsGossip::filter_pull_responses()
-    -> crdsgossip_pull::filter_pull_responses()
-    -> crds::upserts() update the cluster table.
-
 #### Cross domain Validator/History
 
 Some function are implemented inside several domain. THe main reason is because the RPC call aggregate teh 2 domains.
@@ -773,7 +798,6 @@ If the  searchTransactionHistory is set to true:
  * the local storage is search first
  * if not found the Faithful plugin is queried using getTransaction.
 
-
 ###### getBlockCommitment
     - Returns commitment for particular block
     - Parameters: