perf(room-state): carry a signature digest in the member_info summary, not the signature - #572
Conversation
sanity
left a comment
There was a problem hiding this comment.
Review: #572 (head b1db4ed1)
Verdict: Needs Changes. One blocking finding. The change is well-built otherwise: consumer coverage is complete, the summary/delta equivalence holds, determinism is preserved, and the size claim is real and independently re-derived (78 → 21 bytes/entry, ~37 KB → ~10 KB at 470 records).
Lenses run: skeptical (adversarial bug-hunt) and code-first (read-code-before-description), independently, blind to each other. Coverage gap, stated honestly: the testing lens was stopped part-way to relieve build contention and the wire-compat/CRDT-semantics lens did not return, so this is 2 of 4. Both surviving lenses reached the blocking finding independently and cited the same precedent, which is why it is reported with confidence; the coverage gap means test-strength findings below may be incomplete.
BLOCKING — the 64-bit digest is inside attacker reach, and the repo already decided this question the other way
common/src/room_state/member_info.rs:144 truncates blake3 to 8 bytes. The doc comment at :136-137 states the threat model correctly and then draws the wrong conclusion:
A member self-signs their own record, so they could otherwise mint colliding pairs at will. blake3 removes that.
At 64 bits it does not remove it. It prices it at a ~2^32 birthday search, which is hours on commodity hardware. The attacker has unlimited grinding entropy: preferred_nickname is free-form and deputies entries are never validated for membership.
Why a collision is unrecoverable rather than merely annoying. Both lenses traced this independently and agree:
apply_delta(member_info.rs:356-363) replaces only on strict>, so each peer keeps whichever record arrived first.summarize(:240-253) emits an identical(V, d)on both peers, so anti-entropy sees agreement.delta(:273-280) filters on strict>, so neither peer ever offers its record to the other.- Full-state merge does not rescue it: freenet-scaffold implements
mergeas summarize → delta → apply_delta, andcontracts/room-contract/src/lib.rs:139-145routesUpdateData::Statethrough exactly that path.
common/src/room_state/member.rs:378-393 resolves ban authority through deputies_of → canonical, so the two halves of the network permanently disagree on who may ban whom, silently. That is #411 round 4 B reinstated by construction, i.e. the precise bug this discriminator was added to fix. Under the old raw-signature compare it was impossible (different records ⟹ different signatures ⟹ different ranks). This converts an impossibility into a priced attack.
The precedent, which is the strongest argument here. common/src/room_state/direct_messages.rs:162-167 faced the same choice and picked 16 bytes:
128 bits gives a ~2^64 birthday bound - adequate against worst-case attacker-chosen signature grinding (an attacker who can sign as themselves cannot influence which token any other member's purge list contains, and the recipient is the sole signer of their own purge list).
That reasoning describes a weaker threat model than this one, and chose twice the width. Here the attacker controls both sides of the comparison.
Fix: widen to [u8; 16]. Cost is ~9 CBOR bytes per entry (21 → ~30), still ~2.6x better than the 78 being replaced, and still inside the new test's 32-byte bound. Worth taking now specifically because this PR re-keys the contract: the width cannot be revised cheaply afterward.
Should fix
Tie-direction disagreement between two selectors. canonical() uses max_by_key (:39-44), which returns the last maximum; dedup_to_canonical() (:72-87) and apply_delta keep the first. A peer that GETs a full state containing both records answers deputies_of one way, then flips after the next unrelated apply_delta runs dedup. Widening makes this unreachable, but two selectors disagreeing on ties is a latent trap worth closing on its own.
The byte-order pin is only ~50% effective. :139-143 calls from_le_bytes load-bearing, but the oracles at common/tests/deputy_ban_test.rs:1579 and :1661 compare randomly-keyed signatures, so a switch to from_be_bytes would agree about half the time and fail intermittently. By this project's own standard that is a broken detector. A golden vector — one fixed 64-byte signature to one fixed expected digest — makes it deterministic. No golden vector exists anywhere today.
The size guard measures unrealistically cheap keys. common/tests/summary_determinism_test.rs:344-360 builds MemberId(FastHash(i)) for i in 0..470, which CBOR-encodes in 1-3 bytes against a real MemberId's ~9, so it measures ~13-16 bytes/entry where production is ~21. The assertion still holds, but the reported figure understates production by ~30%. It also never calls summarize() — it hand-builds the type, so it cannot catch a regression in how summarize populates it, only a type-shape change that would be a compile error anyway.
Doc claims that no longer survive their own arithmetic. :122-125 says the signature was "~84% of the summary's bytes and made the whole room summary ~29 KB", but 470 × 66 = 31 KB of signature bytes alone, exceeding the stated total. Reconcile against the actual measured record count (records exist only for members who set a nickname, which is likely the explanation). Also :137 "blake3 removes that" per the blocking finding, and :95 "Total, deterministic ordering" is now a total preorder, whose ties are exactly the dangerous case.
Nine stale comments describing the old rule, five inside the rewritten file: member_info.rs:230, :236, :931, :1286, :1291; ui/src/room_data.rs:877, :6339, :6416; ui/src/components/members/member_info_modal/nickname_field.rs:193. Also .claude/rules/contract-summary-determinism.md:59 still describes the value type as Signature. AGENTS.md asks for stale docs fixed in the same PR.
Bytes traded for contract CPU, unmentioned in a perf(...) PR. is_ban_authorized calls deputies_of at member.rs:378, :384, :393, each an O(|member_info|) hash sweep through canonical, inside the per-ban loop at :279 — roughly O(bans × chain_depth × member_info_count) blake3 hashes per validation, under Cranelift OptLevel::None. Since 0.2.105 per-contract WASM CPU is an eviction axis, so this deserves a sentence. Separately dedup_to_canonical (:77-78) recomputes the incumbent's digest on every collision; storing (rank, info) in the map fixes that for free.
The body's test claim is stale. It says "All 406 river-core tests pass", but commit b1db4ed1 then documents that equal_version_member_info_diff_detected_by_anti_entropy was a ~45% flake (89 failures in 200 runs) and that river-ui did not compile — the green run won the coin flip. That fix is good work, honestly written up in the commit message; the body just needs to stop carrying the original claim. 406 also matches none of the counts in that commit.
Consider
sig_digest's doc and the body cite "#571" as though it were the PR that changed behavior; #571 is the issue, this is #572. The body's "three of the five sub-summaries" undercounts: there are nine type Summary declarations under common/src/room_state/, including SecretsSummary (secret.rs:251) alongside the acknowledged DirectMessagesSummary.
Checked and clean — recorded so coverage is visible
expect()on the digest slice (:146-150): unreachable.[..8]of a fixed 32-byte array;try_intois infallible.- Summary/delta agreement: compared line-by-line against the pre-change form. Equivalent outside a digest collision; the "summary value IS the rank" refactor removes a derivation step rather than adding one.
- No missed call sites. Verified by grepping all uses of the changed types rather than reading the diff —
member_info_rank,sig_digest,type Summary(all nine),summarize/StateSummaryacrossui/,cli/,contracts/,delegates/, plus(u32, Signature)prose repo-wide. All five rank consumers route through the singlemember_info_rank; notably the PR does not take the tempting shortcut of keeping the raw-signature compare on non-wire paths, which would have silently reinstated #411 round 4 B. - Determinism: still
BTreeMap; the onlyHashMap(dedup scratch,:72) is drained to aVecand sorted bymember_id. The #4857 HashMap-ordering class is not reintroduced. - No vacuous tests: reverting the type breaks compilation rather than passing silently, and the oracles are genuine independent recomputations. A constant
sig_digestmakesequal_version_member_info_resolves_deterministically_across_apply_orderfail on its first assertion. - No client/contract skew: nothing persists a summary; the UI's summary is consumed locally to build a delta, and riverctl sends
summary: None. - No new WASM dependency:
blake3already ships in bothriver-coreandriver-uiand already runs inside the room contract.
Note on the green checks
All 7 checks pass at b1db4ed1, but check-room-contract-migration is vacuous for this diff: it compares the committed ui/public/contracts/room_contract.wasm, which this PR does not touch, so it reports "no migration entry needed" for a change that alters contract behavior. The body already acknowledges the re-key and holds the PR as draft; flagging only so the green tick is not read as migration clearance. The eventual WASM-rebuild commit will be caught correctly, since it compares against main's old hash.
[AI-assisted - Claude]
|
Correction to my review above, on the byte arithmetic and therefore on the recommended fix. I verified the serializers rather than continuing to reason from the figure everyone (including me) had been repeating.
Corrected per-entry figures (
Two consequences. The recommended fix changes shape. My review said Finding C is resolved, and in the PR's favour. The arithmetic was impossible (470 × 66 = 31 KB of signatures exceeding a stated 29 KB total) because 66 was the wrong per-signature figure. The real reduction is larger than the PR claims. The body and the Note the deferred Credit where due: this came from the wire-format lens, which traced the serializers instead of accepting the number the other two lenses and I had all repeated. Worth recording that the figure survived three independent reviews before anyone checked it. The blocking finding itself is unaffected — 64 bits is still inside a ~2^32 birthday grind, the divergence is still unhealable, and the 128-bit precedent in [AI-assisted - Claude] |
sanity
left a comment
There was a problem hiding this comment.
Re-review: #572 at head e8092b0e (the two post-review fix commits)
Verdict: no blocking findings. One medium (disclosure), four low, four info. Adversarial lens with every claim re-verified by execution, not reading.
Checked and clean — the load-bearing set
- Round-trip: the
SigDigestserde pair is a byte-for-byte copy of the shippedPurgeTokenpattern. Probe tests drove the REAL decode path (fullChatRoomStateV1Summarythrough ciborium and back, byte-identical, thendelta()off the decoded summary — exactly whatget_state_deltadoes, and grep confirms it is the only real decode path; nothing persists a summary). Wrong lengths (0/8/15/17/32) all rejected. One benign asymmetry: the deserializer also accepts the 16-element array form; the serializer never emits it, so no divergence is possible. - Golden vector reproduced externally:
b3sumover0x00..0x3fmatchesEXPECTEDexactly. All three mutations run for real: bytes reversed FAILS, last-16 selection FAILS, derived-impl FAILS on both the vector and the size guard (42.53 B/entry, over the bound). - All five rank consumers first-wins on strict
>, checked one by one, including the hand-rolledcanonicalloop (no off-by-one). The tie branch is unreachable without a genuine collision —verifychecks every record's signature, so the "latent trap, not reachable bug" framing is accurate. - The in-test baseline is not a drifted reconstruction:
git showof the base commit confirms the old type and the oldsummarizeinserted exactly what the test rebuilds, same real serde impl. - Size guard deterministic (3 identical runs; seeded keys, no timestamps, fixed-width encoding): 134.08 → 28.01 B/entry, 4.8x.
- The 2^64 doc claim is conservative — it omits that each birthday candidate costs an ed25519 signing op, and the retired 64-bit figure ("hours on commodity hardware") also checks out arithmetically.
Medium — three of four green CI ticks are blind to this change; the body names only one
The body discloses check-room-contract-migration's vacuity. Also blind: "Check CLI WASM Sync" only cmps the two committed WASM copies against each other (never against source — while its own trigger list claims common/src/** coverage), and the riverctl bump guard doesn't fire on common/src/ changes. Concrete drift while the rebuild is deferred: committed WASM runs the old tiebreak, river-core source runs the new one. Extend the existing disclosure paragraph to name check-cli-wasm.yml.
Low
apply_delta's tie direction is asserted in prose, pinned by nothing —>→>=there leaves all 409 tests green (hard to reach behaviorally since that branch verifies signatures). Extract a sharedoutranks()and test it once, or source-pin the three sites.canonical's digest arm has no river-core test — a version-only-keep-first mutation passes all 409; only a UI test catches it. Cheap deterministic test: twowith_signaturerecords, same version, fixed signatures, greater digest must win. This PR rewrotecanonical, so now is the moment.- The loser-first refixture trades detection of last-wins for detection of first-match/version-only — acceptable, since
canonical_and_dedup_break_rank_ties_identicallywas confirmed to catch last-wins at workspace level. - "The absolute cost is small" is the one unmeasured claim in a measure-don't-derive PR. Measured:
summarize()470 records ~18 → ~97 µs; ≈7 ms per UPDATE under the Cranelift no-opt penalty across the three passes. Fine — but put the numbers in. (TheHashSetpre-pass optimization is noted and deliberately NOT requested — scope.)
Info
The "32 bytes" derive figure is an expectation for random content, not fixed (worth one word); the "37% miss" figure for the reversed-digest oracle reproduced at 1/12 rather than 11/30 — conclusion unchanged (intermittent = broken), figure softened; one stale comment at room_synchronizer.rs:186 ("signature map" → digest map); and a pre-migration check for the bundle: an equal-version duplicate surviving in the Official room's state would flip ban authority under the new tiebreak at re-PUT — post_apply_cleanup makes this unlikely, and a one-line riverctl duplicate check before migrating converts unlikely into verified.
Scope
Source-only review. The deferred bundle (WASM rebuild + legacy_room_contracts.toml + riverctl bump + moderation-key pin) still stands before leaving draft and needs its own pass — and per the medium finding, CI will not catch its absence for you.
[AI-assisted - Claude]
…, not the signature Closes #571 The room summary is ~29 KB and is re-sent to every interested peer on every state change. On the live Freenet network interest_sync_summaries is 49.8% of ALL outbound bytes, making this the single largest bandwidth consumer on the network — against a chat message capped at 1,000 bytes. The metadata describing a change was ~30x the change. MemberInfoV1::Summary carried a raw 64-byte ed25519 Signature per member, which is ~66 of ~78 CBOR bytes per entry (84%). The signature is never verified in the summary path; it exists only as an equal-version tiebreak discriminator and is compared for ordering and equality. A digest does that identically. Summary is now BTreeMap<MemberId, (u32, u64)>, i.e. exactly what member_info_rank returns, so the summary value IS the rank and delta() compares against it directly instead of re-deriving one from a stored signature. Roughly 78 -> 21 bytes per entry; a 470-member room's member_info summary goes ~37 KB -> ~10 KB. blake3 rather than freenet_scaffold::util::fast_hash: fast_hash is a base-31 polynomial, adequate for the accidental collisions MessageId/BanId care about but trivially collidable by construction. A collision here is not cosmetic — two same-version records whose discriminators tie are indistinguishable to anti-entropy, so peers would silently and permanently disagree on that member's deputies, i.e. on ban authority (#411 round 4 B). Members self-sign their own records, so they could otherwise mint colliding pairs. Semantic change: the equal-version tiebreak now orders by digest rather than by raw signature bytes, so a different record can win a tie. Safe because every peer applies the same deterministic rule and this re-keys the contract anyway. Tests: determinism suite updated to the new shape; deputy_ban_test's winner oracle recomputes the digest from blake3 independently rather than calling into river-core, so a change to the digest function or its byte order fails the assertion instead of silently agreeing. New member_info_summary_stays_small_per_entry pins bytes/entry under 32 — the pre-change encoding cannot pass, since a 64-byte signature alone is 66 CBOR bytes. All 406 river-core tests pass.
…digest The digest tiebreak this PR introduced changed which record wins an equal-version member_info collision, but two tests still predicted the winner with the OLD rule (raw signature-byte ordering). Both are ~50/50 coin flips against the new implementation, not deterministic failures, which is why they were not caught locally. - ui/src/room_data.rs `apply_deputy_change_uses_canonical_base_and_version` retries fresh D keys until "clean" outranks "stale_grant", then asserts `canonical` selects "clean". The retry predicate compared raw signature bytes, so it selected a key satisfying the wrong condition and the sanity assertion failed whenever the two orderings disagreed. This is the build failure on this PR: the crate is river-ui, which `cargo test -p river-core` never compiles. - common/tests/deputy_ban_test.rs `equal_version_member_info_diff_detected_by_anti_entropy` picked its expected winner by raw signature bytes. The sibling test one screen up was updated by this PR; this one was missed. It is a latent ~45% flake in river-core (measured: 89 failures in 200 runs before this commit, 0 in 200 after), and the green 406-test run simply won the coin flip. Both oracles now recompute the blake3 digest independently rather than calling into river-core, matching what this PR already did for the sibling test: an oracle that reuses the implementation under test only proves self-consistency, and in the UI case it would additionally make the `canonical` sanity assertion tautological. The shared helper is hoisted to module scope in deputy_ban_test.rs so the two tests cannot drift apart again. No production logic changed. The digest function, its width, and its byte order are untouched, as is the Summary shape. The remaining edits are comments that still described the tiebreak as raw signature bytes; they are line-for-line replacements (net 0 lines in common/src), so the room-contract WASM is unaffected. Verified: cargo test -p river-ui --bins (814) and --bins --features example-data,no-sync (819), -p river-core --tests / --lib (206) / --lib --features ecies-randomized,migration,mentions (248), -p riverctl --lib (314) all pass; cargo fmt --check clean; each repaired test run 200x with zero failures. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SeAw3G4bFhTT242xoEeMxN
The 64-bit digest this PR introduced is inside attacker reach. A member self-signs their own MemberInfo and controls unlimited grinding entropy for it -- preferred_nickname is free-form and deputies entries are never validated for membership -- so they control BOTH sides of the equal-version tiebreak comparison. At 64 bits that is a ~2^32 birthday search, hours on commodity hardware. A collision is unrecoverable rather than merely annoying. summarize advertises an identical (version, digest) on both peers; delta filters on strict `>` so neither peer offers its record to the other; apply_delta replaces only on strict `>` so each keeps whatever arrived first; and full-state merge routes through summarize -> delta -> apply_delta, so it does not rescue either. The two halves of the network then disagree permanently and silently on that member's deputies, i.e. on ban authority. That is #411 round 4 B reinstated by construction, in a case the old raw-signature compare made impossible rather than unlikely. SigDigest is a 16-byte blake3 digest with a hand-written Serialize emitting a CBOR byte string, following direct_messages::PurgeToken, which chose 128 bits for the same reason under a weaker threat model. The two are kept as near-duplicates with a cross-reference rather than factored together: they are independent wire-format commitments. The derived Serialize would emit a 16-element CBOR array (32 bytes, measured 42.5 B/entry) instead of a byte string (17 bytes), giving back most of the saving. Measured through the real summarize() at 470 entries with realistic MemberIds: ~78 -> 28.0 CBOR bytes per entry. Also in this commit: - canonical() kept the LAST maximum (Iterator::max_by_key) while dedup_to_canonical and apply_delta keep the FIRST. Widening makes a tie between distinct records infeasible to mint, but two selectors of "the canonical record" disagreeing is a latent trap: a state holding duplicates would answer deputies_of one way on a freshly-GET'd full state and the other way after the next apply_delta ran dedup. All three now keep the first. - sig_digest_golden_vector pins one fixed signature to one fixed digest and one fixed CBOR encoding. The existing oracles compare randomly-keyed signatures, so a byte-order change leaves them agreeing about half the time: measured, with the digest reversed they passed 11 of 30 runs. An intermittent detector is a broken one. - canonical_and_dedup_break_rank_ties_identically reaches the tie branch via with_signature (two records sharing a signature, differing in deputies) -- what a collision would look like to these pure orderings. Records built normally cannot test it: ed25519 signing is deterministic over the whole MemberInfo, so rank-tied records are byte-identical and the assertion would hold under either tie direction. - member_info_summary_stays_small_per_entry now calls the real summarize() on built state with MemberIds from real VerifyingKeys (deterministically seeded). MemberId(FastHash(i)) for small i encodes in 1-3 bytes against a real key's ~9 and understated the entry by ~30%. - apply_deputy_change_uses_canonical_base_and_version's fixture flips its push order to loser-first. It relied on canonical's old last-wins tie-break to make a version-only selector return the wrong record; with canonical now first-wins, clean-first would let that regression pass by accident. Loser-first restores it and additionally catches the first-match .find() regression the test's header claims to target. - Doc corrections: the summary's per-entry cost is stated as 66 of ~78 CBOR bytes rather than "made the room summary ~29 KB", which could not be true as written (470 x 66 = 31 KB of signatures alone). The 29.1 KB is a fleet-wide mean across all rooms and is not any single room's summary size; no per-room measurement is on record and none is claimed. member_info_rank is a total order on the (version, digest) pair but only a total preorder on records. Nine stale comments describing the old (version, signature) rule are updated, and the determinism rule gains a section on summary VALUES as a wire-format commitment. Verified: cargo fmt --check clean; river-core --tests (409), --lib (208), --lib --features ecies-randomized,migration,mentions (250), riverctl --lib (314), river-ui --bins (814) and --bins --features example-data,no-sync (819) all pass. The four affected test groups were each run 40 consecutive times (1,440 executions, 0 failures), since the previous head carried a ~45% flake here and one green run proves nothing. Both new tests were verified to FAIL under the implementation they guard against (reversed byte order, last-16-instead-of-first-16, derived Serialize, canonical reverted to max_by_key, canonical reverted to .find()). Not in this commit, deliberately: the room_contract.wasm rebuild and the migration entry. This PR is held as draft for that. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SeAw3G4bFhTT242xoEeMxN
…seline The "66 bytes per signature / ~78 bytes per entry" figure carried by issue #571, this PR's body, my code comments, and the first review round is wrong. 66 is the CBOR BYTE STRING encoding -- what River's own SignatureBytes newtype produces via serialize_bytes. ed25519::Signature does not do that: its Serialize calls serialize_tuple(64), ciborium maps a tuple to a CBOR ARRAY, and a uniformly random byte costs 2 bytes there whenever it is >= 24. The real figure is ~124. Measured at 470 entries with realistic MemberIds: before (u32, Signature) 63019 B = 134.08 B/entry after (u32, SigDigest) 13163 B = 28.01 B/entry 4.8x So the win is ~1.7x larger than claimed, not smaller. This is also the entire explanation for the impossible arithmetic flagged in review, where 470 x 66 = 31 KB of signatures exceeded a stated 29 KB total: 66 was the wrong number, so the sum could not close. The Official room's member_info term is ~63 KB, not ~37 KB. The fix is not a better derivation, it is a measurement. member_info_summary_stays_small_per_entry now rebuilds the pre-change (u32, Signature) shape from the SAME records it summarizes and prints both figures, so the before/after claim is produced by the test rather than asserted in prose. A second assertion pins the baseline above 100 B/entry with the explanation inline, because the failure mode is specifically measuring against the wrong encoding: swapping the baseline to SignatureBytes reproduces the old claim exactly (77.01 B/entry), and that assertion catches it. Also recorded in .claude/rules/contract-summary-determinism.md: the two encodings of the same 64 bytes, which one each type uses, and the rule that a summary size claim must measure the old shape in the same test. The deferred DirectMessagesSummary follow-up genuinely is ~66 B/entry since it does use SignatureBytes; the two must not be reasoned about with one number. No production behaviour changes here. Verified: cargo fmt --check clean; river-core --tests (409), --lib (208), --lib --features ecies-randomized,migration,mentions (250), riverctl --lib (314), river-ui --bins (814) and --bins --features example-data,no-sync (819) all pass. The new baseline assertion was verified to FAIL when pointed at SignatureBytes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SeAw3G4bFhTT242xoEeMxN
… digest arm Re-review follow-ups on e8092b0. No behaviour change: `outranks` is `candidate > incumbent`, exactly what the five call sites already spelled inline. LOW-1. The tie direction was asserted in prose at five separate `>` sites (canonical, dedup_to_canonical, summarize, delta, apply_delta) and pinned by nothing -- relaxing any single one to `>=` left all 409 tests green, because reaching the tie branch behaviourally needs a genuine SigDigest collision and `verify` checks every stored record's signature. That is the same prose-only state that let canonical (last-wins) and dedup_to_canonical (first-wins) disagree until the previous commit. All five now route through one `outranks()` with one test. Verified: `>` -> `>=` in outranks fails 4 tests including the new pin. LOW-2. canonical's digest arm had no river-core coverage: a mutation comparing only `version` while keeping the first on a tie -- i.e. matching the real tie direction -- passed all 409, caught only by a river-ui test. The new test is deterministic by construction rather than by retry loop: two `with_signature` records at the same version carrying fixed signatures [1; 64] and [2; 64], whose blake3 digests (29c04cc4... and fe969aba..., computed with b3sum) are known to differ, with the loser placed first so vector position and rank disagree. Verified: the version-only mutation now fails exactly this test (209 passed, 1 failed). INFOs: the "32 bytes" figure for the serde-derive encoding is an expectation for random content (each byte >= 24 costs two), now worded as such; the reversed-digest oracle miss rate is reported as both samples taken (11/30 and 1/12) rather than one, since the rate depends on the keys a run draws and only its being non-zero matters; and one stale comment at room_synchronizer.rs:186 ("a signature map" -> "a digest map") that the earlier sweep missed. Verified: cargo fmt --check clean; river-core --tests (411), --lib (210), --lib --features ecies-randomized,migration,mentions (252), riverctl --lib (314), river-ui --bins (814) and --bins --features example-data,no-sync (819) all pass. Clippy unchanged at 54 pre-existing sites across 17 files, none on lines this diff touches (river#573). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SeAw3G4bFhTT242xoEeMxN
1c2fddc to
fe07a72
Compare
PRE-MIGRATION GATE — must clear before the re-key PUT, not before mergeThe migration bundle has landed (
Why it gates. This PR changes the equal-version tiebreak from raw signature bytes to a 128-bit digest. If two records for the same member at the same Status: NOT satisfied. What I have is indicative, not gating: Reassuring, but the snapshot is stale — it reports 477 records while a live It cannot be run live today. Before the re-key PUT, one of these must hold
Deliberately not a merge blocker: merging changes no live room state. Publishing does. Also note — three of four green ticks are blind to the source changeDetailed in the PR body. [AI-assisted - Claude] |
…r-core (V31/V30) Registers the outgoing generations BEFORE the rebuild, so clients can probe the old keys and recover data dormant across the upgrade (#292): * room contract V31: dd63bcc9... -> e765339b... * chat delegate V30: 6f65e45c... -> a44c6401... river-core 0.1.18 -> 0.1.19, and riverctl's `=` pin with it. This is NOT cosmetic and was the P1 in review: the V31 registry is generated by the build script and `include!`d into river-core (`migration.rs:40`), riverctl pins `river-core = { version = "=0.1.18" }` (`cli/Cargo.toml:53`), and 0.1.18 is ALREADY PUBLISHED on crates.io (verified via `cargo search`). So `release-riverctl.yml`'s publish_if_needed would have skipped river-core, and `cargo install riverctl@0.2.10` would have resolved the published 0.1.18 — a river-core with no V31 entry. Installed CLIs could then not probe dd63bcc9... to recover a dormant room, which is the exact failure the entry exists to prevent. `.claude/rules/river-publish.md` requires this bump for the same reason. WHY THE DELEGATE MOVES, since it is not the obvious reason. The summary change itself is dead-code-eliminated from the delegate: rebuilding with it, before the version bump, left chat_delegate.wasm byte-identical at 6f65e45c..., and the delegate entry was reverted on that evidence. It is the river-core VERSION BUMP that re-keys the delegate (0.1.18 -> 0.1.19 moves both WASMs), and the bump is mandatory per the above. So the entry is required after all, for a different cause than first assumed. Delegate key = BLAKE3(BLAKE3(wasm) || params); without the entry every user's stored rooms_data would be orphaned. The room-contract key therefore moved twice during this work, dd63bcc9 -> c2b77b55 -> e765339b. Only the FIRST is a registry concern: c2b77b55 was never published or registered, and the registry records generations that were live and are now superseded, so dd63bcc9 remains the correct V31 code_hash. Confirmed: the room registry pins did NOT move (len 31, fingerprint b5f02d45b6370b4d) — the bump adds no room-registry content. `legacy_set_fingerprint` 323a2f640bfd7a7f -> c43e66ee147e3739 for the new delegate entry. That fingerprint suffixes the migration localStorage keys, so every user re-probes the legacy delegates once — the intended behaviour for a real new generation. Both room-contract copies verified byte-identical after every rebuild. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SeAw3G4bFhTT242xoEeMxN
fe07a72 to
e38bf58
Compare
|
Pre-migration gate: CLEARED with live data (see the gate comment above). Ran the live equal-version-duplicate check tonight using the #577 tool (draft PR #578), against the live Official room via a node websocket (raw records, no canonicalization, read-only run):
Cross-check: Conclusion: the #572 tiebreak change (raw signature bytes → SigDigest ordering) cannot flip any winner in the live Official room. The re-key publish is clear on this risk; remaining publish requirement is only the two-contract sequencing (riverctl + UI together). [AI-assisted - Claude] |
Problem
The room summary is re-sent to every interested peer on every state change, and on the live Freenet network
interest_sync_summariesis 49.8% of all outbound bytes — the single largest consumer, ahead of contract updates themselves. The measured mean summary message is 29.1 KB, against a chat message capped at 1,000 bytes.MemberInfoV1::Summarycarried a raw ed25519Signatureper member. Measured: 134.08 CBOR bytes per entry, of which ~124 is the signature — about 92%. At the Official room's ~470 records that term alone is ~63 KB.(Those two figures come from different measurements and must not be multiplied together. The 29.1 KB is a fleet-wide mean across all rooms, most of which are far smaller than the Official room, which is why a single large room's member_info term exceeds it. There is no per-room summary measurement on record, so this PR makes no claim about what any one room's total summary weighed.)
Correction: the earlier "66 bytes per signature" figure was wrong
Issue #571, this PR's original body, and the first review round all used 66 bytes for the signature and ~78 bytes/entry. That is the CBOR byte string encoding — what River's own
SignatureBytesnewtype produces viaserialize_bytes. It is not whated25519::Signaturedoes: itsSerializecallsserialize_tuple(64), ciborium maps a tuple to a CBOR array, and a uniformly random byte costs 2 bytes there whenever it is >= 24. The real figure is ~124.This is also the whole explanation for the impossible arithmetic flagged in review (470 x 66 = 31 KB of signatures exceeding a stated 29 KB total): 66 was simply the wrong number.
Demonstrated rather than argued — swapping the test's baseline to
SignatureBytesreproduces the old claim exactly:So the win is ~1.7x larger than this PR originally claimed, not smaller. The deferred
DirectMessagesSummaryfollow-up genuinely is ~66 bytes/entry, because it does useSignatureBytes— two different encodings of the same 64 bytes, and they must not be reasoned about with one number.The signature is never verified in the summary path. It exists only as the equal-version tiebreak discriminator (#411 round 4 B) and is compared for ordering and equality. A digest does that identically.
Approach
type Summary = BTreeMap<MemberId, (u32, SigDigest)>— exactly whatmember_info_rankreturns. The summary value now is the rank, sodelta()compares against it directly rather than re-deriving a rank from a stored signature.SigDigestis a 128-bit BLAKE3 digest of the signature, serialized as a CBOR byte string. Measured on 470 entries with realisticMemberIds, through the realsummarize(): 134.08 → 28.01 CBOR bytes per entry, a 4.8x reduction, so the Official room's member_info term goes ~63 KB → ~13 KB.Both figures are measured in the test, not derived:
member_info_summary_stays_small_per_entryrebuilds the pre-change shape from the same records and prints both. Given that a wrong per-entry figure survived an issue, a PR body, and a review round, a size claim that is the entire justification for a change should not rest on arithmetic.Why 128 bits and not 64. A collision here is not cosmetic and does not self-correct. Two same-version records whose discriminators tie are indistinguishable to anti-entropy:
summarizeadvertises an identical pair on both peers,deltafilters on strict>so neither peer offers its record to the other,apply_deltareplaces only on strict>so each keeps whatever arrived first, and full-state merge routes through summarize → delta → apply_delta so it does not rescue either. The two halves of the network then disagree permanently and silently on that member'sdeputies, i.e. on ban authority — precisely the bug #411 round 4 B added the discriminator to fix, and something the old raw-signature compare made impossible rather than merely unlikely.A member self-signs their own record and has unlimited grinding entropy for it (
preferred_nicknameis free-form,deputiesentries are never validated for membership), so the attacker controls both sides of the comparison. At 64 bits that is a ~2^32 birthday search — hours on commodity hardware. 128 bits puts it at ~2^64. This follows the precedent already in this repo:direct_messages::PurgeTokenderives a 16-byte BLAKE3 value from a signature under a strictly weaker threat model and still chose 128 bits.Why BLAKE3 and not
freenet_scaffold::util::fast_hash.fast_hashis a base-31 polynomial — fine for the accidental collisionsMessageId/BanIdcare about, trivially collidable by construction, which would price the attack above at roughly nothing regardless of width.Why a hand-written
Serialize. This is the same trap as the 66-vs-124 one above, in the other direction. A[u8; 16]through the serde derive emits a 16-element CBOR array (32 bytes), not a byte string (17) — measured, that puts the entry at 42.53 B/entry, over the 32-byte bound. The hand-writtenSerializecallingserialize_bytesis what makes 28.01 achievable, and it is the reason a bare[u8; 16]was not used.Measured alternatives, all at 470 entries with realistic
MemberIds:(u32, Signature)— before(u32, u64)— 64-bit digest(u32, SigDigest)— shipped(u32, u64, u64)— two halves(u32, [u8; 16])via serde deriveConsistent tie direction.
canonical()usedmax_by_key, which returns the last maximum, whilededup_to_canonical()andapply_deltakeep the first. Widening the digest makes a tie between distinct records infeasible to mint, but leaving two selectors of "the canonical record" disagreeing is a latent trap: a state holding duplicates would answerdeputies_ofone way on a freshly-GET'd full state and the other way after the nextapply_deltaran dedup. All three now keep the first.Cost traded the other way: contract CPU
This buys bytes with WASM CPU, which since freenet-core 0.2.105 is itself an eviction axis, so it is worth stating rather than leaving implicit.
Before this change
member_info_rankreturned(version, signature.to_bytes())— a 64-byte copy, no hashing. Now every rank costs one BLAKE3 hash of 64 bytes:dedup_to_canonicalruns on everyapply_delta(and again inpost_apply_cleanup) and computes one digest per storedmember_inforecord — ~470 hashes per apply in the Official room, where before it copied 470 signatures.is_ban_authorizedcallsdeputies_oftwo-plus-ancestor-depth times per ban validated; each goes throughcanonical, which scans the record vector as before but now hashes the matching record — normally exactly one.Measured natively, 470 records, best-of-5 over 200 reps:
summarize()before (64-byte copy)summarize()after (blake3)signature.to_bytes()blake3(64 bytes)What is NOT measured, stated plainly: the cost inside the contract WASM under Cranelift
OptLevel::None. These are native timings, and native profiling of this codebase's contracts is known to mislead badly (roughly 30x, which is why.claude/rulessays to profile contracts underwasmtime -O opt-level=0rather than natively). So no absolute per-UPDATE figure is claimed here. The durable number is the 1.4x onsummarize(), which holds across both native profiles and is a property of the work being done rather than of the compiler.Two mitigations are in this PR:
canonicalcomputes each candidate's rank once rather than once per comparison, anddedup_to_canonicalstores(rank, info)in its map instead of recomputing the incumbent's digest on every collision. A furtherHashSetpre-pass was raised in review and deliberately not taken — out of scope for this PR.Behaviour change to be aware of
The equal-version tiebreak now orders by digest rather than raw signature bytes, so a different record can win a tie. Safe because every peer applies the same deterministic rule, and this re-keys the contract regardless.
This re-keys the room contract, so it needs the standard River migration. Left as draft until that is sequenced. The bundle, as one commit per repo precedent:
room_contract.wasm(both committed copies)common/legacy_room_contracts.tomllen() == 30count and the fingerprint pin incommon/src/migration.rsmember_infoduplicate. A surviving duplicate would flip ban authority at re-PUT, because the winner is chosen by a different rule after this change.post_apply_cleanupdedups, so this is unlikely — a one-line riverctl check converts unlikely into verified, and it is cheap next to the cost of getting it wrong on the live room.Three of the four green CI ticks are blind to this diff. Flagging so the green row is not read as clearance:
check-room-contract-migrationcompares the committedui/public/contracts/room_contract.wasm, which this PR deliberately does not rebuild, so it reports "no migration entry needed" for a change that alters contract behaviour.check-cli-wasm.yml's sync step onlycmps the two committed WASM copies (ui/public/contracts/vscli/contracts/) against each other, never against source — so it passes while both are stale, even though its own trigger list claimscommon/src/**coverage.contracts/room-contract/src/,contracts/room-contract/Cargo.toml, or a committed.wasm.common/src/is not in that list, so it does not fire on this diff at all.The concrete drift while the rebuild is deferred: the committed WASM runs the old tiebreak, river-core source runs the new one. The eventual WASM-rebuild commit will be caught correctly by the first check, since it compares against main's old hash — but CI will not tell you the bundle is missing before then.
Scope note on the other sub-summaries
There are nine
type Summarydeclarations undercommon/src/room_state/. Three are fixed-size scalars (configuration.rs,version.rs,upgrade.rs) and do not scale with content. Of the six that do, four already carry compact hashes or small tuples (member.rs,ban.rs,message.rs,secret.rs). Two carried raw 64-byte signatures: this one, andDirectMessagesSummary.message_signatures: BTreeSet<SignatureBytes>(66 bytes per DM held), which the same fix applies to. That one is deferred so this change can be reviewed on its own, and because member_info is the measured dominant term.Testing
sig_digest_golden_vector(new) — ONE fixed signature, ONE fixed expected digest, ONE fixed expected CBOR encoding. This pins the four things every peer must agree on: that the hash is BLAKE3 oversignature.to_bytes(), that the digest is the first 16 bytes, that they are kept in natural order, and that the value encodes as a 17-byte byte string. Expected bytes were produced with theb3sumCLI outside the crate, not by running the code under test.This exists because every other check on the digest compares randomly-keyed signatures, which detects a byte-order change only intermittently. Measured: with the digest bytes reversed, the existing oracles passed 11 of 30 runs — a 37% miss rate. The golden vector fails 100% of the time. (Verified by toggling: reversed order, last-16-instead-of-first-16, and derived-
Serializeeach fail it.)canonical_and_dedup_break_rank_ties_identically(new) — builds two records that share a signature but differ indeputiesviawith_signature, which is what a digest collision would look like to these two pure orderings, and asserts both keep the same one. Verified to fail whencanonicalis reverted tomax_by_key. Records built the normal way cannot test this: ed25519 signing is deterministic over the wholeMemberInfo, so two records that tie on rank are byte-identical and the assertion would hold either way.member_info_summary_stays_small_per_entry— rewritten to be realistic, and to measure its own baseline. It now calls the realsummarize()on built state instead of hand-constructing the summary type; derivesMemberIds from realVerifyingKeys (deterministically seeded, so the measurement is reproducible and cannot flake); and rebuilds the pre-change(u32, Signature)shape from the same records so the before/after figures are both measured in one place. The oldMemberId(FastHash(i))for smalliencodes in 1-3 bytes against a real key's ~9 and understated the entry by ~30%.It carries a second assertion pinning the baseline above 100 B/entry, whose whole job is to catch the encoding confusion above: verified to fail (at 77.01 B/entry) when the baseline is measured against
SignatureBytesinstead ofSignature. Verified to fail on the forward direction too, whenSigDigestuses the derivedSerialize(42.53 B/entry against the 32 B bound).deputy_ban_test's winner oracles androom_data's key-search fixture recompute the digest from BLAKE3 independently rather than calling into river-core, so a change to the digest function fails the assertion instead of silently agreeing.apply_deputy_change_uses_canonical_base_and_version's fixture had to flip its push order. It relied oncanonical's old last-wins tie-break to make a version-only selector return the wrong record; withcanonicalnow keeping the first, clean-first would have let that regression pass by accident. Loser-first restores the detection and additionally catches the first-match.find()regression the test's own header claims to target. Verified: regressingcanonicalto.find()fails it.outranks_keeps_the_incumbent_on_a_tie(new) — the tie direction was asserted in prose at five separate>call sites and pinned by nothing: relaxing any one to>=left all 409 tests green, because reaching the tie branch behaviourally needs a real digest collision (verifychecks every stored record's signature). All five now route through a singleoutranks()with one test. Verified:>→>=inoutranksnow fails 4 tests including this one.canonical_selects_by_digest_not_just_version(new) —canonical's digest arm had no river-core coverage; a version-only-keep-first mutation passed all 409 and was caught only by a river-ui test. Deterministic by construction: twowith_signaturerecords at the same version with fixed signatures[1; 64]and[2; 64], whose digests (29c04cc4…andfe969aba…, fromb3sum) are known to differ, loser placed first. Verified: the version-only mutation now fails exactly this test (209 passed, 1 failed)..claude/rules/contract-summary-determinism.mdgains a "summary values are a wire-format commitment too" section covering hash choice, width against the threat model, truncation/order, and encoding — plus the golden-vector and realistic-size-guard requirements.Results
cargo test -p river-core --testscargo test -p river-core --libcargo test -p river-core --lib --features ecies-randomized,migration,mentionscargo test -p riverctl --libcargo test -p river-ui --binscargo test -p river-ui --bins --features example-data,no-synccargo fmt --checkclean.Flake check. The previous head carried a ~45% flake in
equal_version_member_info_diff_detected_by_anti_entropy, so a single green run proves nothing here. The four affected groups —deputy_ban_test equal_version_member_info,river-core --lib member_info,summary_determinism_test, andriver-ui apply_deputy_change— were each run 40 consecutive times: 1,440 test executions, 0 failures, noFAILEDlines, every exit code 0.Clippy.
cargo clippy --workspace --all-targets -- -D warningsdoes not pass on this repo and did not before this change — the clippy workflow is checked in asclippy.yml.disabled. It stops at 7 errors inriver-core(ban.rs×5,dm_body.rs,message.rs— none touched by this PR); allowing those three lints (doc_lazy_continuation,assertions_on_constants,repeat_once) lets it run the whole workspace, where it finds 54 more across 17 files inui/andcli/. All 61 sites were cross-referenced against the lines this diff touches: none land on a changed line, and nothing new comes from this change. Left for a separate PR rather than fixed here, since unrelated lint cleanup riding along in aperf(...)PR is the scope creep AGENTS.md warns about.Closes #571
[AI-assisted - Claude]