Logbook

This page records a log of the team’s work, thinking, discussions, pains, joys, events, and experiences that happen on a daily basis. It’s supposed to be a kind of Stream of consciousness that can later be searched, analyzed, reviewed in more formal retrospectives.

Rules:

• Log days in reverse chronological order, recent first

• When the stream is getting too big, such that github refuses to render do the following:

  • extract biggest logical section (half-year or quarter) out of this into a correspondingly named Logbook-Year-Hn.md or Logbook-Year-Qn.md page
  • update links on top of this page to make it easier to navigate try to fix all links in the wiki at least
  • notify the team

• Move (and link) recurring sections to relevant dedicated pages (TBD).

March 2022

2022-03-24

Mithril Session

Discussions/thoughts about MVP and +:

• In order to simplify the registration process, each signer must (re)register its keys at epoch n-1 in order to be able to sign at epoch n
• The aggregator server could later keep track of the operations taking place and thus be able to produce reports & statistics
• How can we verify the genesis state? The genesis state (first certificate in chain) must be signed by a genesis key. It will be verified by the associated public key delivered on the Cardano node (TBV)
• Signers could store their single signatures on-chain
• Aggregator could store the protocol parameters on-chain
• Certificates (or hash of them) could be stored on-chain
• Verification could use the on-chain storage (after restoring it on a node) to proceed to full verification (all the previous certificates chain)
• The drawback of using on-chain transactions, is that it has a cost. We could maybe think of a "cashback" mechanism where the aggregator could refund (+reward) these transactions later by using the treasury

2022-03-22

Mithril Session

We talked about the project organization with Roy:

• We need to work on an estimated roadmap for the Mithril project:

  • This will help check if we are on track with the goals
  • For now let's focus on the MVP, from the end goal: allow people who use Cardano to bootstrap a node fast and securely
  • The MVP is expected at +6 months on the real testnet (+3 months on the development testnet)
  • Let's focus on priorities instead of deadlines
  • We can work as for Hydra with a matrix of Features vs Priorities (must have, should have, could have, nice to have)
  • Short term goal: have something working for people to start using (even though it is not full features yet)
  • We will focus on the layout of the plan during the next session on Thursday

• Agile organization:

  • Let's work with agile sprints and ceremonies (duration 2 weeks)
  • Ceremonies: 30 min for the demo (together with Galois for the Test Lab), 1h for the backlog grooming and sprint planning (on Thursday)
  • First sprint would start on Thursday, March 24
  • First demo would be on Thursday, April 07

• Target features for the MVP would be:

  • Create a snapshot from the Cardano node
  • Create a certificate for a snapshot
  • Store a certificate and a snapshot (centralized on a server)
  • Verify a certificate and download a snapshot
  • Bootstrap a Cardano node from a downloaded & verified snapshot

Mithril architecture meeting

Questions:

1. what is the thing we want to certify? => that's the thing a node would need to bootstrap
   • point on the chain + corresponding ledger state at some epoch boundary
   • node.db does contain stuff we don't need -> volatile db
   • it contains the blocks up to some point + ledger state

Right now node assumes it has all blocks => can't run without previous blocks

• In principle, it could "forget" past of the chain

Need to select parts of the node.db:

• Node can already take a snapshot whenever we like -> current policy is snapshot 10K blocks, could be adapted Takes in memory state in a non-blocking way and serialises it to disk: contains only the ledger state ledger-state directory in the node.db
• ledger state is internal format but block format is not block format is guaranteed to be portable

SPOs could post their signatures on the chain -> follow the chain to produce the

TODO:

• look at consensus code (LedgerDB) for the snapshot policy, need to understand what are the data structures stored on need

• details on how to make sure what does the node need to set up faster

• use Mithril certificate to sign just segment from the previous certificate -> could download/verify segments in parallel

• Sign point = (slot nr, hash) -> enough to ensure we have the valid chain

• include list of hashes for all epoch files?

• Index files can be reconstructed from the epoch files -> needed for fast bootstrap

• tradeoff between speed and robustness

• sign db immutable files independently from the ledger state

Q: epoch files?

• Praos => large chunks (X nr of slots) per file
• Need to check alignment on the epoch boundary -> https://hydra.iohk.io/build/13272534/download/1/report.pdf We could use a 10K or whatever limit as long as it includes the epoch boundary -> it's ok to have more blocks worst case = epoch boundary + 10k ?

2022-03-21

First Session with JP, some Q&A:

• What's the role of the TestLab?
• What about the repository's organisation? It feels a bit messy right now, with lot of inconsistencies here andther
  • JP should feel free to reorganise the code to be cleaner

Work from past week:

• Understanding how it works by Looking at the go node, it's interesting to see a first stab at the node, even though it's definitely not suited for production and there are quite a few problems
• Developed a simple CLI utility in rust-demo to be an easy to use executable for mithril primitives
  • currently, it can "run" mithril protocol with several parties, replicating what's done in the integration tests but more configurable
  • Could be the basis for a more elaborate tool to be used by SPOs without needing a full blown network

Big question: What's a "signable state" from a cardano-node?

• The node's DB might not be identical between nodes even at the same block no.
• How do we ensure all potential signers sign the same "thing"?
• We need to add stake distribution to the signed "thing" as this is part of the validity of the certificate?

How does bootstrapping would work:

• Sign certificate with genesis keys at epoch X => Then parties can produce signature and certificates starting from epoch X + 1
• Issuing a new genesis certificate could be a "version boundary". A new genesis certificate is issued with metadata for versions, that makes it clearer when protocol changes, nodes get upgraded...

What if nodes come and go?

• Mithril nodes need Stake + Keys to produce signatures => Nodes that come within an epoch boundary need to wait next epoch
• Each epoch boundary the parameters are recomputed and redistributed to all participants

2022-03-18

TestLab Demo

• Had a first demo integrating the monitor framework with a primitive rust-node: The test lab spins up a number of nodes (actual processes), connects them and intercepts all messages between them, and checks basic properties.
  • Tried verifying a simple fault injection during registration process
  • We noted it's perfectly fine to "extend" the node's API in order to make testing easier, esp. as there aren't any node right now, as long as we take care of not breaking the "black box" abstraction: The test lab should talk to the node through an API that regular clients could make sense of
• We decided to postpone next meeting till 2022-03-28 in order for everyone to have time to regroup, and think about what the next steps would look like