Document feature UTxO-HD v0.1

[dnadales]

Blog post

UTxO-HD will have an impact on the Cardano node performance for no apparent immediate benefit. We need to:

• Explain why UTxO-HD is important for the long-term sustainability of the Cardano node.
• Warn about the expected performance loss.
• Explain the problem and solution
• Implications of not solving the problem, in particular, what would the mem requirements be.
• Explain what this means for sustainability and decentralization.

We need performance metrics so that the community can make an informed decision.

We will relay this information in the form of a draft blogpost to the Cardano Steering Committee.

Checklist

• Port UTxO-HD documents to the new feature branch #151
• Develop documentation for other developers/teams (inc. READMEs) (UTxO-HD for-developers documentation #250).
• Develop documentation for the general public (Add documentation for the general public #252).
• utxo-db-api design document should get an editorial review (Update utxo-db-api.pdf #326).
• utxo-db and utxo-db-api documents should be accessible somewhere (Deploy utxo-db and utxo-db-api docs #330).
• Develop technical documentation, like ouroboros-consensus/docs/utxo-hd/report.pdf low-level report (superseded by haddocks documentation).

Important information to include

• Why do we have a changelog diff anchor? Differences between volatile anchors
  and diff anchors.

Structure

Currently, the docs folder is structured as follow:

|-docs
  |-report
  |-website
    |-contents
      |-about-ouroboros
      |-for-developers

As mentioned above, we need some documentation for the general public and some technical documentation aimed at developers.

• Documentation for the general public (in about-ouroboros): A utxo-hd.md file describing the motivation for UTxO-HD, what users can expect from using either in-memory or LMDB, configuration options, installation requirements
• Documentation for developers/technical documentation (in for-developers):
  • utxo-hd.md: high-level description of the feature and files structure. Maybe a section on how downstream packages are affected? (I think that's mostly having to use ledger tables and mapkinds in some places)
  • utxo-hd/ : markdown files for important concepts/components.
    • Design concepts (re-iterating parts of the utxo-db-api document, like the partitioned in-memory/on-disk representation)
    • Ledger states with holes
      • Ledger tables
      • Hard fork combinator
    • The new LedgerDB
      • BackingStore
      • Changelog
    • Ledger-HD: more on-disk tables
      ...

[dnadales]

Documentation must wait until version 2 of the LedgerDB API is finished.