[State Sync] State Sync v2!

[JoshLind]

Last Update: 22 December 2021
Owner: Joshua Lind

Goals

This issue is to track: (i) the development and deployment of state sync v2; (ii) the deprecation and removal of state sync v1.1; and (iii) the development and deployment of the storage service.

Task Completion and Checklist

• Publish an initial draft of the state sync v2 DIP.
• Propose a new storage API for state sync v2 and identify the information currently missing from the implementation and APIs.
• Clean up the interfaces and abstractions between state sync, mempool and consensus.
• Fix the panic in the existing reconfiguration notification code ([Bug] State Sync On-chain Configuration Updates #9121).
• Clean up the reconfiguration subscription service and add support for event subscriptions.
• Build a simple template for the Storage Service (server-side). Stub out unsupported API calls.
• Define the Diem Data Client API and implement a simple interface that can used as a starting point.
• Build a Data Streaming Service that supports (bulk) epoch and transaction fetching.
• Extend the Data Streaming Service to support (bulk) transaction outputs and account fetching.
• Extend the Data Streaming Service to support (continuous) transaction and transaction output fetching.
• Extend the Data Streaming Service to support stream termination and multiple stream creation.
• Add simple integration and unit tests to the Data Streaming Service.
• Add logging for the Data Streaming Service.
• Add metrics for the Data Streaming Service.
• Add configs for the Data Streaming Service.
• Extend the Data Streaming Service to handle data advertising attacks (i.e., handle each case in a simple way).
• Clean up some of the naming around the Data Streaming Service.
• Build a simple State Sync Multiplexer that wraps State Sync v1.
• Build a simple Storage Service.
• Add configs to the Storage Service.
• Add metrics and logging to the Storage Service.
• Implement strategies B1 and S1 (transaction execution) in state sync v2.
• Mark state sync v1.1 for deprecation.
• Modify storage to serve account state chunks dynamically.
• Implement strategy B4 (waypoint account state syncing) in state sync v2.
• Implement strategy B3 (epoch-skipping and waypoint syncing) in state sync v2.
• Modify storage to persist and serve transaction outputs.
• Implement strategies B2 and S2 (applying transaction outputs) in state sync v2.
• Remove state sync v1.1 from the codebase.
• Update strategies B1 and S1 to use pipelined execution (i.e., decouple execute and commit).
• Update runbooks to explain different state syncing strategies
• Update metric dashboards to include state sync v2

Post MVP Tasks

• Reason about Data Advertising Attacks
• Explore issues around maximum data sizes in a single network packet