Cardano Database Incremental Snapshots MVP

[rezabaram]

Why

We aim to provide incremental Cardano DB snapshots to allow clients to download and verify only the parts they are missing or interested in. This will make our current solution more useful by enabling users to download only the immutable files they lack, along with the relevant ledger state. For instance, a full node wallet user could sync their node in minutes or even seconds using Mithril.

What

Implement a new type of Mithril certified data which enables the retrieval and verification of a range of immutable files.

How

• Create new signed entity type for Incremental Cardano DB #2121
• Implement signable builder for Incremental Cardano DB #2122
• Implement artifacts builder for Incremental Cardano DB #2151
• Implement artifact routes for Incremental Cardano DB in aggregator REST API #2152
• Implement digests route for Incremental Cardano DB in aggregator REST API #2174
• Implement artifacts cloud synchronization in Incremental Cardano DB with GCP #2211
• Incremental Cardano DB artifacts production enhancements #2234
• Implement a retry mechanism for the FileUploader #2213
• Implement Incremental Cardano DB in client library #2214
• Implement Incremental Cardano DB in client CLI #2246
• Implement Incremental Cardano DB in WASM client #2247
• Implement monitoring for Incremental Cardano DB #2249
• Update explorer for Incremental Cardano DB #2212
• Enhance artifact structure for Incremental Cardano DB #2291
• Support evolving cloud artifact locations type to avoid client breaking change #2293
• Enhance computation of required disk space for Incremental Cardano DB in client CLI #2292
• Add integration test for Incremental Cardano DB client #2328
• Use consistent naming in the client Cardano database APIs #2248
• Optimize artifact downloads when restoring an Incremental Cardano DB #2327
• Implement an example for Incremental Cardano DB #2330
• Document Cardano incremental database signature in website #2331
• Document Incremental Cardano DB usage in the clients #2329

Later

• Implement artifacts uploader configuration of aggregator in Incremental Cardano DB
• PoC BitTorrent archives storage
• PoC IPFS archives storage
• Support archives evolutive source selection and fallback mechanism (and monitoring)