Storage optimization

[thanethomson]

High-level tracking issue for general storage optimization efforts. This issue can be expanded over time.

At present (late 2022), depending on their configuration, Tendermint-based nodes use large quantities of storage space. This has significant cost implications for operators. We aim to implement strategies to reduce and/or offload certain data stored in order to reduce operators' costs.

• Investigate which types of data are consuming the most storage space
  • Provide a way to obtain a clear breakdown of the different types of data a node is storing and how much storage they use.
• Define storage use cases and workloads #9943 Preliminary investigation to identify the users and workloads of the storage backend. The result of this step is supposed to be a high level list containing information on who and how (at a very high level) accesses the storage.
  • What components of Tendermint access the storage and how do they access it (read/write)
  • How are our users interacting with the storage (operators, relayers etc.)
• Extend the previous point with: Establish and implement the relevant metrics to understand storage workloads #9773
  • Define the metrics we need to understand the usage of storage.
  • Implement the metrics defined.
• Simplify Tendermint database engine support #9749
  • Gather data using previously implemented metrics to identify the storage requirements for Tendermint.
  • Research different databases based on the obtained requirements.
  • Refactor Tendermint to use a single database backend.
• Identify the functionalities we want to support within Tendermint and offload non-critical data and functionality
  • ADR-082: Data Companion Push API #9437
  • Provide a mechanism to offload event indexing based on ADR-082: Data Companion Push API #9437 or a different design.
• Short-term fixes to alleviate pressure for operators
  • Add in-process compaction support to LevelDB #9743
  • abci++: Define and implement pruning strategy for stored extended commits #8458
• Develop an understanding on how to manage a blockchain's history across the whole network
• Storage management is complex (a UX/docs-related set of problems)
  • docs/ux: Simplifying storage management #9906