Ledger encoding

Ledger encoding

Upcoming. This page describes the ledger-node branch. It is not part of a release, and details may change.

Part of Ledger implementation. This page gives the bytes of everything the ledger signs, stores or sends, what each signature covers, and how hashes, roots and references are made.

Byte layouts

All integers are big-endian. Every fixed-size field is checked on construction and copied in and out, so a record cannot be changed through an array it handed out.

Item	Layout	Bytes
Address	type 1 and a key of 32, or type 2 and a legacy address of 20	33 or 21
Mint	kind 1, recipient address, amount 8, reference 32	74 or 62
Vote	kind 2, voter 32, candidate 32, action 1 (1 add, 2 remove), round 8, signature 64	138
Claim	kind 3, public key 33 or 65, recipient key 32, signature 65	131 or 163
Transfer	kind 4, sender 32, recipient address, amount 8, fee 8, nonce 8, signature 64	154 or 142
Block header	height 8, view 4, previous hash 64, time 8, state root 64, transaction root 64, last commit hash 64, proposer 32	308
Endorsement	signer 32, signature 64	96
Certificate	view 4, endorsement count 2, endorsements	up to 96,006
Block	header, certificate, last commit (a certificate), transaction count 2, transactions	up to 355,322
Genesis	UGLEDGR3, time 8, round length 4, key count 2, keys 32 each, snapshot hash 64, snapshot total 8
Heartbeat	validator 32, height 8, tip hash 64, time 8, signature 64	176
Status	chain id 64, height 8, tip hash 64	136
Block request	first height 8, most blocks 4	12
Block batch	block count 2, then each block as a length 4 and its bytes	up to 16 blocks, 8 MiB
Ballot	kind 1, type 1 (0 prevote, 1 precommit), height 8, view 4, block hash 64, validator 32, signature 64	174
Proposal	kind 2, height 8, view 4, valid view 4, sender 32, signature 64, block length 4 and block, proof count 2, proof ballots of 173 bytes each
Decision	kind 3, block length 4 and block

A mint has no signature, and an amount of zero or less is refused. A transaction's id is the SHA-512 of its encoding, so the id of a vote, claim or transfer includes its signature. The three consensus messages are told apart by their first byte. A heartbeat, a status and a request are encoded alone, and the frame they travel in gives their kind, see Ledger networking.

The genesis above is the binary form that the chain id is computed from. Operators handle it as a JSON file, described on Running a ledger node.

Decoding refuses a block with more than 1000 endorsements or transactions, truncated input, an unknown transaction kind, an unknown address type, trailing bytes, and a header that names another last commit than the block holds. A list is grown one decoded entry at a time and never sized from a count read off the wire, so a forged count costs nothing until the bytes it promises are there. The largest block is two full certificates of 1000 validators and 1000 claims.

What a signature covers

Each signature starts from SHA-512(tag, chain id, …), so none can be replayed on another chain:

flowchart LR
    Genesis[genesis encoding] -->|SHA-512| Chain[chain id]
    Tag["tag, such as Unigrid transfer v1"] --> Digest
    Chain --> Digest["SHA-512(tag, chain id, fields)"]
    Fields[fields of the record] --> Digest
    Digest -->|Ed25519 with the signer's key| Signature[64-byte signature]

Loading

Signature	Tag	Covers after the chain id
Vote	Unigrid vote v1	voter, candidate, action, round
Transfer	Unigrid transfer v1	sender, recipient with its type byte, amount, fee, nonce
Heartbeat	Unigrid heartbeat v1	validator, height, tip hash, time
Precommit, which signs a block	Unigrid precommit v1	height, view, block hash
Prevote	Unigrid prevote v1	height, view, block hash or nil
Proposal	Unigrid proposal v2	height, view, valid view, block hash, and a digest of the proof

A claim is signed with the legacy wallet instead, over the text claim <chain id> <recipient key> in hex. LegacySignature accepts a 65-byte signature on secp256k1 whose header byte says whether the key is compressed (31 to 34) or not (27 to 30) and matches the key supplied, with the low s only. The message hash is the double SHA-256 of DarkNet Signed Message:\n and the text, each with a length byte, so the text must be plain ASCII and at most 252 bytes. The recovery id is not checked, so a claim is identified by its address and never by its signature bytes. The compressed and uncompressed encodings of one key are two different addresses. The key must hash (RIPEMD-160 of SHA-256) to the address being claimed.

Hashes, signatures and roots

• SHA-512 comes from the JDK, in Digests, over the concatenation of its parts.
• Ed25519 is the plain algorithm, through the lightweight API of BouncyCastle in Ed25519. A key pair is made from a 32-byte seed, and verify is false for a signature that is not 64 bytes.
• The Merkle root has the tree shape of RFC 6962 with SHA-512. A leaf is hashed with a 0x00 prefix and an inner node with 0x01, the left subtree takes the largest power of two below the leaf count, and a lone leaf is never duplicated. The root of nothing is the hash of nothing.
• The block hash is the SHA-512 of the 308-byte header alone, so a block has one identity whatever its certificate. The transactions are bound through the transaction root, a Merkle root over their ids, and the last commit through its hash in the header.
• The chain id is the SHA-512 of the genesis encoding, which includes the snapshot hash and total. The tip of an empty chain is the chain id, so the first block names the chain in its previous hash, and every later block names it through the one before, see the figure on Ledger.
• The snapshot hash in the genesis is the SHA-512 of the bootstrap snapshot without its signature, the same digest that the foundation signs, see Legacy chain snapshot.
• Endorsements are kept sorted by signer, so the same signatures always give the same bytes.
• Verified signatures of claims and transfers are remembered in a table of 10,000, oldest dropped first. Only successes are kept.

References

A reference is 32 bytes that a mint uses once. Two kinds are derived, each the first 32 bytes of a SHA-512:

Reference	Hash of	Why
Of a claim	Unigrid claim v1, the snapshot hash, the 20-byte legacy address	The snapshot balance of an address is taken once
Of a spork mint	Unigrid mint v1, the recipient's address encoding, the entry's number as 8 bytes	An entry of the mint spork is minted once, whatever node proposes it

The number of a spork entry is a height of the legacy chain or a time in seconds, depending on the address, see Ledger sporks. Two entries for one address with different numbers are two grants with two references.

In the source

• ledger/src/main/java/org/unigrid/hedgehog/ledger/TransactionCodec.java: decoding of the four transactions, defined in Mint.java, Vote.java, Claim.java and Transfer.java
• ledger/src/main/java/org/unigrid/hedgehog/ledger/BlockCodec.java: blocks and certificates, with Block.java and Certificate.java
• ledger/src/main/java/org/unigrid/hedgehog/ledger/ConsensusCodec.java: proposals, ballots and decisions
• ledger/src/main/java/org/unigrid/hedgehog/ledger/WireCodec.java: heartbeat, status, request and batch
• ledger/src/main/java/org/unigrid/hedgehog/ledger/LegacySignature.java: the proof of a claim; Address.java and LegacyAddress.java define the accounts
• ledger/src/main/java/org/unigrid/hedgehog/ledger/MerkleRoot.java, Digests.java and Ed25519.java: hashing and signing
• ledger/src/main/java/org/unigrid/hedgehog/ledger/Reference.java: the two derived references
• ledger/src/main/java/org/unigrid/hedgehog/ledger/LedgerGenesis.java: the genesis encoding and the chain id