Implementation Spec: Augmenting recovery-snapshot receipts with cross-recovery service-identity endorsements

[achamayou]

Implementation Spec: Augmenting recovery-snapshot receipts with cross-recovery service-identity endorsements

• Status: v0.1 — locked for discussion (no implementation yet); revised to carry endorsements in an out-of-band sidecar file rather than a rewritten snapshot
• Component: node recovery path, snapshots, receipts, network-identity subsystem
• Primary files touched (anticipated): src/node/snapshot_serdes.h, src/node/node_state.h,
  src/node/rpc/network_identity_subsystem.h, src/node/historical_queries_utils.cpp,
  src/service/tables/previous_service_identity.h
• Related existing machinery: CoseEndorsement, PREVIOUS_SERVICE_IDENTITY_ENDORSEMENT,
  InternalTablesAccess::endorse_previous_identity, NetworkIdentitySubsystem,
  populate_cose_service_endorsements

---

1. Summary

When a CCF node is started in recovery mode and is provided with a snapshot, it currently
verifies the receipt embedded in that snapshot against a single previously-trusted service
identity (the operator-supplied previous_service_identity — the identity that signed the
snapshot).

This spec proposes that, during recovery, the node additionally scans the ledger chunks that
follow the snapshot, collects the COSE "back-endorsements" that every disaster recovery (DR)
emits, and uses that chain — persisted alongside the snapshot in a small sidecar file — to
augment verification of the snapshot's receipt. The
result is a snapshot whose receipt is verifiable against the latest service identity In — the
operator-supplied recovery identity (§5.2.1), which is the identity current at the end of the recovered
ledger — even when multiple DRs happened between the snapshot and that identity.

This feature covers snapshots whose embedded receipt is a COSE receipt (COSE_Sign1, the current
format). Verification is augmented with a COSE endorsement chain — the back-endorsements already
written by endorse_previous_identity — carried in a sidecar file, so that the snapshot verifies
against In.

Old-style / JSON snapshot receipts are out of scope for augmentation (see §5.5). A snapshot that
still carries an old-style receipt is left untouched; if a DR has moved past it so that it can no longer
be verified against In, it is handled by the usability gate (§5.6.1) — the snapshot is not used and
recovery falls back to full-ledger replay.

Design principle: the endorsement chain is collected from the KV (the COSE endorsement table)
and persisted out-of-band, in a small sidecar file next to the snapshot — the snapshot itself is
never copied or mutated. Verification consumes the snapshot's existing receipt together with the
sidecar chain (exactly how CCF already delivers COSE endorsements out-of-band for transaction
receipts), reusing existing endorsement plumbing. The sidecar is cheap, disposable, and fully
re-derivable from the ledger: if it fails validation it is simply dropped and recovery falls back to
today's behaviour.

---

2. Background

2.1 Snapshots and their embedded receipts

A serialized snapshot file is two concatenated segments (snapshot_serdes.h::separate_segments):

• header_and_body — the serialized KV store snapshot.
• receipt — a receipt that commits to the digest of header_and_body.

verify_snapshot(segments, prev_service_identity) dispatches on the first receipt byte:

• 0xD2 → verify_cose_snapshot_receipt: decodes the COSE_Sign1, checks the receipt's
  claims_digest equals SHA-256(header_and_body), and (if prev_service_identity is supplied)
  verifies the detached COSE signature against that identity's public key.
• { → verify_json_snapshot_receipt: parses a ProofReceipt, recomputes the Merkle root,
  verifies signature with the receipt's node cert, and (if prev_service_identity is
  supplied) checks the node cert is endorsed by that service cert.

New snapshots are produced with COSE receipts (snapshot_serdes.h::build_and_serialise_receipt
→ describe_cose_signature_v1 + Merkle inclusion proof in the unprotected header). JSON receipts
appear only in snapshots created by older CCF versions.

2.2 Service identity and disaster recovery (DR)

Each DR establishes a new service identity. Denote the identity that signed the snapshot as
I0, and each subsequent DR's identity as I1, I2, …, In, where In is the latest identity
present in the recovered ledger (the one current immediately before the current recovery attempt
begins). Intuitively In is the ledger's latest identity, but formally this spec takes In to be
the operator-supplied recovery config certificate, which must equal that latest identity and is
cross-checked against it (§5.2.1); the config value — not the ledger's claim — is the trust anchor.

2.3 The COSE back-endorsement chain (already implemented)

On every recovery, when the service is opened, NodeState calls
InternalTablesAccess::endorse_previous_identity(tx, service_key), which writes a CoseEndorsement
to the public single-value table
public:ccf.internal.previous_service_identity_endorsement
(Tables::PREVIOUS_SERVICE_IDENTITY_ENDORSEMENT).

CoseEndorsement (service/tables/previous_service_identity.h) contains:

• endorsement — a COSE_Sign1 in which the new service key signs the previous service
  public key, carrying epoch.start.txid / epoch.end.txid headers.
• endorsing_key — the signing (new) service public key (DER).
• endorsement_epoch_begin / endorsement_epoch_end — TxID range the endorsed identity was valid
  for.
• previous_version — KV version of the previous write to the table (a back-pointer used to
  walk the chain). Absent for the very first, self-endorsement entry.

Because the table is single-valued, each DR overwrites it; the historical chain is recovered by
following previous_version (or by capturing each write during ledger replay — see §5.3).

Chain integrity rules already exist in network_identity_chain_helpers.h
(verify_endorsements_connected, validate_chain_front_connection, is_self_endorsement) and are
enforced by NetworkIdentitySubsystem while building a per-epoch set of trusted_keys.

2.4 How transaction receipts are already augmented (the analogy to follow)

For historical query receipts, historical_queries_utils.cpp already augments a receipt so a
verifier trusting the current identity can verify a receipt signed under a past identity:

• populate_cose_service_endorsements(...) sets receipt->cose_endorsements to the chain returned
  by NetworkIdentitySubsystem::get_cose_endorsements_chain(seqno) (COSE path).

TxReceiptImpl already carries std::optional<CoseEndorsementsChain> cose_endorsements for exactly
this purpose.

This spec applies the same idea to the snapshot's embedded receipt, but sources the endorsements
from the recovered ledger rather than from live KV / a running subsystem.

---

3. Problem statement

verify_snapshot today can only anchor trust in the single identity that signed the snapshot
(I0). A party (operator, auditor, or a downstream node) that only trusts the latest identity
In cannot verify a snapshot whose receipt was signed by I0, even though the ledger contains an
unbroken cryptographic chain I0 ← I1 ← … ← In of back-endorsements proving In vouches
(transitively) for I0.

We want the recovering node — which is uniquely positioned because it replays the whole ledger and
therefore observes every DR endorsement — to close this gap by augmenting the snapshot receipt with
that chain.

---

4. Goals

• G1. During a recovery that consumes a snapshot, collect all COSE back-endorsements emitted by
  DRs that occurred after the snapshot seqno and up to the end of the recovered ledger.
• G2. Produce a sidecar endorsements file carrying this chain such that the snapshot — combined
  with the sidecar at verification time — verifies against the latest recovered service identity In
  (not only against I0). The snapshot itself is neither copied nor modified.
• G3. Support snapshots whose embedded receipt is a COSE receipt, augmenting them with a
  COSE endorsement chain.
• G4. Handle snapshots whose embedded receipt is an old-style/JSON receipt safely: augmenting
  such receipts is out of scope (§5.5). An old-style snapshot is never mutated; if a DR has moved
  past it so it cannot verify against In, it is rejected by the usability gate (§5.6.1) and recovery
  falls back to full-ledger replay.
• G5. Preserve backward compatibility: the endorsement chain is additive. The snapshot must
  still verify against I0 using the existing single-identity path, and a snapshot with no sidecar
  must continue to verify exactly as today.
• G6. Prefer material already present in / derivable from the recovered ledger; do not require
  interactive cooperation from past identities. (The COSE back-endorsement material is present in the
  ledger.)
• G7. Reuse existing chain-integrity validation (network_identity_chain_helpers.h) so
  verification is rejected if the collected chain is broken, out-of-order, or does not connect
  I0 to In.
• G8. Never mutate or copy the input snapshot on disk. The endorsement chain is written to a
  small sidecar file next to the snapshot; the original snapshot stays byte-for-byte unchanged.
  A stale or invalid sidecar can be dropped freely (§5.6.1).
• G9. The sidecar must reuse the existing serialized-endorsement format
  (SerialisedCoseEndorsements — a CBOR array of endorsement COSE_Sign1 blobs), so verification goes
  through the standard endorsement-verification path. No bespoke or hybrid container is introduced.

5. Detailed design

5.1 When the feature runs

The collection runs very early in StartType::Recover startup — before the startup snapshot's
KV state is deserialized/consumed and before PartOfPublicNetwork (or any recovery-share /
private-recovery phase) is reached. (The snapshot's receipt segment is parsed at this point — it
is needed for the final signature / claims-digest check — but the KV state body is not deserialized
until a usable snapshot has been selected.) As soon as the node has (a) a startup snapshot at seqno S
(find_local_startup_snapshot) and (b) detects any committed ledger suffix after S, it performs
a dedicated pre-pass that scans that suffix, collects the endorsements, and (if the chain is complete)
writes the sidecar endorsements file that is then consumed alongside the original snapshot (§5.3,
§5.6).

Rationale for running this early: the node must decide which snapshot to deserialize before it
deserializes one. Since the trust anchor is In, the receipt can only be checked once the endorsement
chain is available, so the chain must be gathered up-front rather than after replay. The scan only
needs public ledger tables (see §5.3), which are readable without ledger secrets, so it is possible
this early.

It does not run for StartType::Join or StartType::Start, and is a no-op when no snapshot is
supplied or when there is no ledger suffix after S (then In == I0 and the snapshot already
verifies).

5.2 Definitions

• S = snapshot seqno (startup_snapshot_info->seqno).
• I0 = identity that signed the snapshot receipt (epoch containing S).
• In = identity current at the end of the recovered ledger — i.e. the service identity just before
  this recovery. In is the operator-supplied config.recover.previous_service_identity (the
  previous_service_identity_file startup input), which is a trusted, ledger-independent value (see
  §5.2.1 and O8). It is also the value that the later transition_service_to_open governance proposal
  must match against the KV PREVIOUS_SERVICE_IDENTITY (node_state.h:2296).
• I(n+1) = the new identity this recovery creates (the recovering node generates a fresh keypair
  and adopts In's subject name at node_state.h:1035-1042). It is not the target.
• Target identity = In. The snapshot receipt, together with its endorsement chain, must verify
  with In as the trust anchor.

5.2.1 Trust anchor for In — the existing recovery config input (O8 resolved)

CCF recovery already requires the operator to supply the previous service identity certificate at node
startup via previous_service_identity_file → config.recover.previous_service_identity
(src/host/configuration.h, HostConfig::Command::Recover). Today this cert is used to (a) seed the
recovering node's NetworkIdentity subject name (node_state.h:1035-1042) and (b) verify the startup
snapshot directly: verify_snapshot(segments, config.recover.previous_service_identity)
(node_state.h:534). For a snapshot taken in the current generation, that cert is the snapshot
signer, so verification passes; for a snapshot taken before one or more DRs, the snapshot is signed
by an older I0 != In and verification fails today — this is exactly the gap this feature closes.

This feature reuses that same operator input as the target In and relaxes the requirement that
the snapshot be directly signed by it: an older I0-signed snapshot is accepted when the collected
endorsement chain connects In (config) down to I0. Because In comes from operator config — not
from the not-yet-verified ledger suffix — a forged latest-identity planted in a tampered suffix cannot
be accepted: the endorsement chain is validated under the config-supplied In's key
(validate_chain_front_connection against In), so a chain terminating at a bogus identity simply
fails the gate and the node falls back to ledger replay (§5.6.1). The startup input and the
transition_service_to_open governance argument remain distinct inputs at distinct times, but must
carry the same identity value (cross-checked at node_state.h:2296), giving end-to-end consistency.

5.3 Collecting back-endorsements from the ledger

The endorsement material is gathered by a dedicated early scan of the committed ledger suffix after
S (§5.1) — not by the main recovery replay. The scan reads only public ledger tables, which are
stored unencrypted in each entry's public domain and are therefore readable before any ledger
secrets are recovered:

• public:ccf.internal.previous_service_identity_endorsement — the per-DR COSE endorsement.
• public:ccf.gov.service.info — the ledger's latest ServiceInfo, whose cert is cross-checked to
  equal the config-supplied In (§5.2.1) and which provides the create-TxID used to anchor
  validate_chain_front_connection. The trust anchor remains the config In; the ledger value is only
  used to locate/validate the chain front, and any mismatch fails the gate (§5.6.1).

(The existing python/src/ccf/ledger.py already reads public-domain tables entry-by-entry via
get_public_domain().get_tables(), confirming this scan is feasible without secrets and giving the
client-side verifier, O6, the same source.)

COSE receipt → collect the COSE endorsement chain. As the scan walks public entries with seqno
> S in order, it records each write to PREVIOUS_SERVICE_IDENTITY_ENDORSEMENT, yielding the
endorsement entries E1 (I1→I0), E2 (I2→I1), …, En (In→I(n-1)) with their epoch ranges. (Equivalently,
once the final table value is known, the previous_version back-pointers can be walked exactly as
NetworkIdentitySubsystem does, stopping at the endorsement whose endorsement_epoch_* covers S;
the forward scan is preferred here because the node is already reading the suffix.) The entries are
collected oldest-first (E1…En) here and later serialized newest→oldest when embedded in the
receipt (§5.4.1).

The collected set is validated with the existing helpers, anchored to the config-supplied In:

• the newest endorsement's endorsing_key must be In's key (config), via
  validate_chain_front_connection — this is what ties the whole chain to the operator's trust anchor;
• each adjacent pair passes verify_endorsements_connected;
• the oldest relevant endorsement's epoch range must cover S (i.e., it endorses I0).

Old-style receipt → out of scope. If the snapshot carries an old-style/JSON receipt, no endorsement
collection is performed for it — augmenting old-style receipts is out of scope (§5.5). Such a snapshot
is either usable as-is (In == I0) or rejected by the usability gate (§5.6.1).

The chain is only needed when S predates a DR. If S falls within the latest epoch (no DR
happened after the snapshot), the collected chain is empty and no augmentation is required (the
snapshot already verifies against In == I0).

5.4 Producing the endorsements sidecar

Given the snapshot's COSE_Sign1 receipt R (signed by I0) and the validated COSE chain
[E1 … En], the node serializes the chain into a sidecar file — it does not modify or re-emit
R, and does not copy the snapshot body:

1. Serialize the endorsements (CoseEndorsement::endorsement bytes — each a full COSE_Sign1) as a
   CBOR array ordered newest→oldest, i.e. exactly the existing SerialisedCoseEndorsements
   representation returned by NetworkIdentitySubsystem::get_cose_endorsements_chain. This is the same
   value CCF already delivers out-of-band for transaction receipts (describe_cose_endorsements_v1 /
   the /receipt/cose_endorsements endpoint), so no new wire/encoding format is introduced.
2. Write that byte string to the sidecar file (§5.6). The snapshot's receipt R, its Merkle inclusion
   proof, its ccf.v1 protected map, and its I0 signature are all left untouched — they live only in
   the original snapshot file.

No new COSE header label is introduced. Because the chain is carried out-of-band in the sidecar
(not embedded in R's unprotected header), the verifier receives it as a separate input; there is no
change to src/crypto/cose.h and no re-serialization of the receipt.

Verification (target = In). The verifier is given (R, endorsements): it verifies each
endorsement in the chain (each COSE_Sign1 signed by the newer key over the older key, with contiguous
epoch ranges) anchored at the config-supplied In, derives I0's public key at the tail of the chain,
and finally verifies R's detached signature over the recomputed Merkle root with I0's key. This
reuses the trusted-key chain built in NetworkIdentitySubsystem::build_trusted_key_chain and the
historical_queries_utils.cpp receipt-verification logic that already accepts an out-of-band
endorsement list — only the source of that list differs (the sidecar vs. a live subsystem).

5.5 Old-style / JSON receipt augmentation — out of scope

Augmenting old-style/JSON snapshot receipts is out of scope for this feature. The endorsement
material such a receipt would need to reach In is not persisted in current ledgers: the only
previous-identity tables written at DR are PREVIOUS_SERVICE_IDENTITY (the previous service's
self-signed cert) and PREVIOUS_SERVICE_IDENTITY_ENDORSEMENT (the COSE endorsement). Old-style
cross-identity endorsements were historically minted live by a node holding the current private
key, which a recovering node cannot reproduce for a past identity, and CCF must never fabricate
endorsements without genuine signing material or require a past identity's private key.

Consequently, this feature does not augment old-style snapshot receipts:

• If no DR occurred after S (In == I0), the un-augmented old-style snapshot already verifies
  against In and is used as today.
• If a DR has moved past the snapshot (In != I0), it cannot be verified against In; per the
  usability gate (§5.6.1) it is rejected and recovery falls back to full-ledger replay. The ledger
  and the original snapshot are left untouched. This outcome is surfaced to the operator
  (log/telemetry).

Persisting the material needed to also augment old-style snapshots (e.g. writing a companion
endorsement in the same DR transaction) is a possible future enhancement, but is not part of this
feature.

5.6 Output: a sidecar endorsements file (snapshot untouched)

Output invariant — a small, disposable, re-derivable sidecar. The augmentation writes only a
sidecar file containing the serialized endorsement chain (SerialisedCoseEndorsements, §5.4). The
snapshot file is neither copied nor modified. The sidecar is a cache of ledger-derived material: it can
be deleted at any time (by the operator, or by the node itself on a failed gate, §5.6.1) with no effect
on correctness — worst case the node re-derives it or falls back to ledger replay.

Why out-of-band rather than a rewritten receipt: CCF already delivers COSE endorsements out-of-band
for transaction receipts (a separate SerialisedCoseEndorsements list). Persisting the same list in
a sidecar mirrors that exactly and avoids duplicating a potentially multi-GB snapshot body just to
add a few KB of endorsements. It also keeps the snapshot byte-identical for any external consumer
that ignores the sidecar.

• COSE receipts: the sidecar carries the endorsement chain; at consume time the node verifies
  (snapshot receipt, sidecar) against In.
• Old-style receipts: out of scope (§5.5) — no sidecar is written. When In == I0 the original
  snapshot is used as-is; when a DR occurred after S, the snapshot is rejected (§5.6.1) and recovery
  falls back to full-ledger replay.

File handling:

1. Do not mutate or copy the input snapshot. The original committed snapshot file
   (snapshot_<S>_<evidence>.committed) is left on disk byte-for-byte, in any directory (writable or
   read-only). It remains the single source of the snapshot body and receipt.
2. Write a sidecar file next to it. The endorsement chain is written to a separate, small file
   whose name is derived from the snapshot's, so the pairing is unambiguous and it does not match
   the committed-snapshot glob. Proposed naming (see O1, §10):
   snapshot_<S>_<evidence>.committed.endorsements. It is written to a writable snapshots directory; if
   none is writable the feature is simply skipped (fall back to replay).
3. Consume snapshot + sidecar together. When the sidecar is present and valid, the recovering node
   loads the snapshot as today and passes the sidecar chain as the extra verification input (§5.4), so
   the snapshot is accepted under In. The snapshot bytes handed to the rest of the recovery flow are
   unchanged.
4. No double-selection problem. Because the sidecar does not match the committed-snapshot glob used
   by find_committed_snapshots_in_directories, snapshot discovery is unchanged; the node just checks
   for an adjacent .endorsements sidecar for the chosen snapshot. See §5.6.2 (O2).

5.6.1 Usability gate and fallback (O7)

Usability is a binary decision made up-front, not a way to relax verification. The candidate is
(snapshot receipt, endorsements), where endorsements is the freshly-collected chain (or an existing
sidecar), or empty when In == I0:

1. Verify the candidate against the target identity In using the standard
   endorsement-verification path (chain validated and anchored at In, then I0 detached signature
   over the Merkle root).
2. If it verifies against In → the sidecar is written (if not already present) and the snapshot is
   consumed alongside it; recovery proceeds from the snapshot as an optimisation.
3. If it cannot verify against In — for any reason: broken/incomplete/out-of-order chain,
   endorsement epoch not covering S, tampered material, or an old-style receipt moved past by a DR
   (out of scope, §5.5) — verification is rejected. No sidecar is persisted (and any pre-existing
   stale sidecar is dropped). There is no I0-only acceptance: the snapshot is simply not used,
   and the node falls back to the current recovery behaviour — replaying the ledger, which remains
   unchanged. The original snapshot file is left on disk untouched.

This keeps the ledger as the ultimate source of truth: the feature only decides, on a limited basis,
whether this snapshot can be trusted under In. A rejected snapshot never weakens recovery; it just
forgoes the snapshot optimisation for that start-up, and the discardable sidecar means there is nothing
heavy to clean up.

5.6.2 Sidecar reuse and idempotency (O2)

On startup, after selecting the snapshot as today, the node looks for an adjacent
snapshot_<S>_<evidence>.committed.endorsements sidecar. If one exists, it is re-verified against
In (never trusted purely on presence): on success it is reused (no re-derivation); on any failure it
is dropped and the node re-derives from the ledger (or falls back, §5.6.1). If no sidecar exists, the
node derives one via the pre-pass. This makes the operation idempotent across restarts and robust to a
stale or tampered sidecar, with no snapshot-selection ambiguity (the sidecar never participates in
snapshot discovery).

5.7 API / signature changes (anticipated)

• Extend snapshot verification to accept a target identity and an out-of-band endorsement chain,
  e.g. verify_snapshot(segments, prev_service_identity = I0, target_identity = In, cose_endorsements = <SerialisedCoseEndorsements>), keeping the current 2-arg behavior as the backward-compatible default.
  The endorsement chain is passed as a separate input (from the sidecar) — not embedded in the COSE
  receipt — so there is no new COSE header label and no receipt re-serialization.
• A small sidecar read/write helper: serialize the collected chain to
  SerialisedCoseEndorsements bytes and write/read the .endorsements sidecar next to the snapshot
  (§5.6).
• An early collection routine invoked at the very start of the recover flow (node_state.h, before
  set_startup_snapshot consumes the snapshot and before PartOfPublicNetwork), which: detects a
  ledger suffix after S, scans public entries (§5.3), validates the chain, applies the usability gate
  (§5.6.1), and either writes the sidecar and proceeds from the snapshot, or drops any stale sidecar and
  falls back to full-ledger replay.
• Adjacent-sidecar lookup on startup: after snapshot selection, check for a matching
  .endorsements sidecar and re-verify it against In before reuse (§5.6.2).
• A lightweight public-ledger-suffix reader (or reuse of existing ledger-reading plumbing) able to
  enumerate public-domain table writes without ledger secrets.
• Client-side verifier updates (python/src/ccf/receipt.py, ledger tooling) to verify a COSE snapshot
  against In given the snapshot plus its sidecar endorsement chain (O6).

6. Non-goals

• N1. Not changing how snapshots are generated during normal (non-recovery) operation.
• N2. Not using any past identity's private key and not fabricating endorsements without real
  signing material. Collection uses only COSE back-endorsement material already present in the
  recovered ledger.
• N3. Not augmenting arbitrary transaction receipts — that is already covered by the historical
  query path (populate_cose_service_endorsements).
• N4. Not enabling this for Join or fresh Start; recovery-with-snapshot only.
• N5. Not mutating, copying, or deleting the input snapshot file; the original always remains on
  disk byte-for-byte. Only a small, disposable sidecar file is written next to it.
• N6. Not augmenting old-style/JSON snapshot receipts; that is out of scope (§5.5). Such a snapshot
  is used as-is only when In == I0, and otherwise rejected in favour of ledger replay.
• N7. Not changing the COSE DR/endorsement writing mechanism (endorse_previous_identity) or the
  CoseEndorsement schema, and not adding any new endorsement-writing mechanism at DR time. Not
  embedding the chain in the receipt or introducing any new COSE header label — the chain is carried
  out-of-band in the sidecar, reusing the existing SerialisedCoseEndorsements format.
• N8. Not weakening or bypassing existing I0-anchored verification; the endorsement chain is
  strictly additive supplementary material.

7. Security considerations

• The usability gate fails closed: if the collected chain is incomplete, non-contiguous, or fails
  signature/epoch validation (network_identity_chain_helpers.h), the snapshot is rejected and the
  node falls back to full-ledger replay (§5.6.1). There is no I0-only acceptance of a snapshot that a
  DR has moved past; the ledger (unchanged) is the source of truth.
• Trust anchor for In under early execution (O8 resolved). In is the operator-supplied
  config.recover.previous_service_identity (the existing previous_service_identity_file startup
  input) — a trusted value independent of the not-yet-verified ledger suffix, not whatever the
  suffix claims as the latest identity. The endorsement chain is self-verifying and is validated under
  that config-supplied In (validate_chain_front_connection against In), so forged endorsements or
  a bogus latest identity in a tampered suffix cannot yield a chain that validates — the gate fails and
  the node falls back to ledger replay (§5.2.1, §5.6.1).
• Collection must never require a past identity's private key and must not fabricate
  endorsements without genuine signing material (see N2). This is why augmenting old-style receipts is
  out of scope (§5.5).
• Sidecar integrity. The sidecar is untrusted cache material: it is re-verified against In on
  every load (§5.6.2) and never trusted on mere presence. A tampered or forged sidecar cannot make a
  snapshot verify against In (the chain is self-verifying and anchored at the config-supplied In),
  so the worst a bad sidecar can do is force a re-derivation or a fall-back to ledger replay. The
  sidecar carries only public endorsement material (no secrets).
• Trust is not expanded: In verifying I0 transitively is exactly the guarantee CCF already makes
  for cross-epoch transaction receipts; this feature only makes it available for snapshot receipts.
• No confidentiality impact: the endorsements, the previous/latest service certs, and the public
  snapshot body are all public material readable without ledger secrets.

8. Backward compatibility

• Non-augmented snapshots verify exactly as before for external consumers.
• Snapshot files stay byte-identical for any external consumer — nothing is embedded in the
  receipt. The endorsement chain lives in a separate .endorsements sidecar that reuses the existing
  SerialisedCoseEndorsements encoding, so consumers that ignore the sidecar are unaffected and those
  that use it go through the standard endorsement-verification path.
• Recovering-node behaviour change: the node's own accept/reject decision now targets In. A
  snapshot that a DR has moved past and whose chain cannot reach In (notably any old-style snapshot,
  which is out of scope) is no longer used to bootstrap recovery — the node falls back to full-ledger
  replay (§5.6.1). This can make some recoveries fall back to replay where they previously used a
  snapshot, but never weakens the trust anchor.

9. Testing (outline)

• Unit: COSE chain collection and validation over a synthesized sequence of CoseEndorsements
  (contiguous, broken, out-of-order, missing-I0-coverage, empty/no-DR).
• Unit: sidecar round-trip — serialize the collected chain to SerialisedCoseEndorsements, write and
  re-read the sidecar, and verify (snapshot receipt, sidecar) against In, and still against I0.
• Unit: old-style snapshot moved past by a DR is rejected (out of scope) and recovery falls back to
  ledger replay (§5.5, §5.6.1).
• Unit/file: the sidecar is written with the expected .endorsements name next to the snapshot, does
  not match the committed-snapshot glob, the snapshot file is left byte-for-byte unchanged, and an
  adjacent sidecar is reused after re-verification / dropped and re-derived when tampered (§5.6.2).
• Unit: usability gate (§5.6.1) — a snapshot whose chain cannot reach In is rejected, no sidecar is
  persisted (any stale one dropped), and recovery falls back to full-ledger replay (ledger + snapshot
  untouched).
• Client-side verifier (O6): extend python/src/ccf/receipt.py / ledger tooling to verify a COSE
  snapshot against In given the snapshot plus its sidecar endorsement chain, and use it as the
  primary test oracle for the e2e scenario.
• Integration/e2e: multi-DR recovery scenario — COSE snapshot from I0, ledger containing DRs to
  I1…In, recover with the I0 snapshot, assert the derived sidecar makes the snapshot verify against
  In. Also
  cover an old-style snapshot moved past by a DR falling back to ledger replay.
• Negative: tampered endorsement, gap in the chain, endorsement whose epoch does not cover S, and a
  forged In in the suffix that must not validate under the real In (O8).

10. Design decisions (resolved)

All questions raised during design review have been resolved; this section records the decisions and
their rationale for implementers.

• O1 (RESOLVED — snapshot_<S>_<evidence>.committed.endorsements sidecar). The endorsement chain is
  written to a small sidecar file whose name is the snapshot's name plus an .endorsements suffix. It
  keeps the snapshot_<seqno>_<evidence> prefix (so the pairing is obvious and parseable) but,
  critically, does not match the committed-snapshot glob used by
  find_committed_snapshots_in_directories — so it never participates in snapshot discovery and the old
  double-selection problem disappears. It is easily operator-recognizable and droppable.
• O2 (RESOLVED — adjacent sidecar, re-verified on every load). Snapshot discovery is unchanged.
  After selecting the snapshot, the node looks for an adjacent .endorsements sidecar and, if present,
  re-verifies it against In before reuse (never trusts it on presence): success → reuse, failure →
  drop and re-derive (or fall back). Absent → derive via the pre-pass. This is the idempotency
  mechanism across restarts and is robust to a stale/tampered sidecar (§5.6.2).
• O3 (RESOLVED — very early, before snapshot deserialization). The collection runs as
  a dedicated pre-pass at the very start of recovery — as soon as a startup snapshot and a committed
  ledger suffix after S are detected, before the snapshot is deserialized/consumed and before
  PartOfPublicNetwork. It reads only public ledger tables, so no ledger secrets are required (§5.1,
  §5.3).
• O4 (RESOLVED — old-style augmentation is out of scope). The material an old-style receipt would
  need to reach In is not persisted in current ledgers (main persists only the previous-service
  self-signed cert and the COSE endorsement; old-style cross-identity endorsements were historically
  minted live with the current private key, unavailable to a recovering node for a past target
  In). Decision: covering old-style snapshots is out of scope (§5.5); an old-style snapshot
  that a DR has moved past is rejected in favour of ledger replay (§5.6.1). Persisting
  the material to also cover old-style snapshots is a possible future enhancement, not part of this
  feature.
• O5 (RESOLVED — out-of-band sidecar, no new COSE header label; supersedes the earlier v0 decision).
  Rather than embedding the chain in the receipt's unprotected header, the COSE endorsement chain is
  carried out-of-band in the .endorsements sidecar (O1), reusing the existing
  SerialisedCoseEndorsements encoding: a CBOR array of the endorsement COSE_Sign1 blobs ordered
  newest→oldest, matching get_cose_endorsements_chain. This mirrors how CCF already delivers COSE
  endorsements for transaction receipts, keeps the snapshot and its receipt byte-identical, and means
  no new COSE header label and no change to src/crypto/cose.h. (This supersedes the v0 decision to
  add a ccf.v1.service_endorsements unprotected-header label.) Verification takes the receipt and the
  sidecar chain as separate inputs (§5.4).
• O6 (RESOLVED — yes, update client-side verifiers). python/src/ccf/receipt.py and the ledger
  tooling are updated to verify a COSE snapshot against In given the snapshot plus its sidecar
  endorsement chain. Although this is primarily consumed inside the node, updating the client verifiers
  is convenient as the e2e test oracle and is worth having even if not used in the general case.
• O7 (RESOLVED — reject and fall back to ledger replay). Reaching In is a binary usability gate
  (§5.6.1): if the snapshot receipt (with its collected endorsement chain) cannot be verified against
  In, the snapshot is rejected, no sidecar is persisted, and the node falls back to the current
  recovery behaviour — replaying the ledger, which stays unchanged. There is no I0-only acceptance;
  the decision is purely can this snapshot be used under In or not.
• O8 (RESOLVED — In is the operator-supplied recovery config identity). The target In is the
  existing config.recover.previous_service_identity startup input (previous_service_identity_file,
  src/host/configuration.h), already used at node_state.h:534 (verify_snapshot) and
  node_state.h:1035-1042 (node subject name). This feature reuses it as the trust anchor and relaxes
  the "snapshot must be directly signed by it" requirement to "snapshot must reach it via the augmented
  chain". Since In is operator-supplied (not read from the unverified ledger suffix), a forged
  latest-identity in a tampered suffix cannot be accepted (§5.2.1, §7). This startup input is a
  distinct input from the transition_service_to_open governance previous_service_identity
  argument, but the two must carry the same value (cross-checked at node_state.h:2296).