Skip to content

Room summary is ~29 KB because it embeds 64-byte Ed25519 signatures where a u32 hash would do #571

Description

@sanity

Summary

The room summary is ~29-33 KB. It should be a small set of identifiers — the minimum needed to compute a delta — and its identifiers already are compact hashes. The bulk is raw 64-byte Ed25519 signatures carried alongside those hashes, used only as content fingerprints. A u32/u64 hash serves that purpose identically at 5 CBOR bytes instead of 66.

This matters well beyond River: on the live Freenet network, interest_sync_summaries is 49.8% of all outbound bytes (freenet/freenet-core#4965), and the room is the dominant contract driving it. Measured mean summary message: 29.1 KB, against a room chat message capped at 1,000 bytes. The metadata describing a change is ~30x the change itself.

Where the bytes are

component per-entry notes
MemberInfoV1::Summary = BTreeMap<MemberId, (u32, Signature)> (member_info.rs:126) MemberId ~9 B + u32 ~2 B + Signature 66 B signature is 84% of the entry
DirectMessagesSummary.message_signatures: BTreeSet<SignatureBytes> (direct_messages.rs:1381, built at :940) 66 B each — SignatureBytes(pub [u8; 64]) one per DM held
MessagesSummary.message_ids: BTreeSet<MessageId> (message.rs:112) ~9 B — MessageId(pub FastHash) already a hash, fine
MembersV1::Summary = BTreeSet<MemberId> (member.rs:34) ~9 B already a hash, fine
BansV1::Summary = BTreeSet<BanId> (ban.rs:290) ~9 B already a hash, fine

So the design is right in three of five components. The two that carry signatures dominate the byte count. With roughly 350-470 member records the member_info term alone accounts for most of the observed summary size.

Why a hash is sufficient

member_info.rs:184-210 documents the intent directly:

Carry the signature alongside the version so anti-entropy can detect a SAME-version content difference and correct it (#411 round 4 B).

Difference detection is exactly what a hash is for. The summary never verifies the signature — it only compares it for equality against a peer's advertised value. Likewise direct_messages.rs:992 uses message_signatures.contains(...) purely as a membership test.

Proposed change

Replace the embedded signatures with fixed-width hashes:

  • MemberInfoV1::Summary: BTreeMap<MemberId, (u32, Signature)>BTreeMap<MemberId, (u32, u32)>, the second u32 being a hash of the signature bytes.
  • DirectMessagesSummary::message_signatures: BTreeSet<SignatureBytes>BTreeSet<u32> (or u64 if collision headroom is wanted; see below).

Projected: member entry ~79 B → ~18 B (4.4x), DM entry 66 B → 5 B (13x), room summary roughly 29 KB → 5-7 KB.

Things to get right

  1. Determinism is load-bearing. .claude/rules/contract-summary-determinism.md and State updates permanently lost for rarely-changing fields: silent ContractQueueFull drop + sender-side neighbor-summary poisoning freenet-core#4857: freenet-core byte-compares summarize_state output for staleness, so the collections must stay BTreeSet/BTreeMap and the hash must be a fixed, endianness-explicit function. A DefaultHasher would be catastrophic here — its output is not stable across processes or releases.

  2. member_info_rank tie-breaking. member_info.rs:110 is fn member_info_rank(version: u32, signature: &Signature) -> (u32, [u8; 64]) and ranks duplicates by raw signature bytes. The summary must advertise the pair that deputies_of enforces on, so if the summary carries a hash, the ordering peers agree on has to be derived consistently. Ranking by hash instead of raw bytes changes which duplicate wins in a tie. That is acceptable if every peer does it identically, but it is a semantic change and needs a deliberate decision plus a test, not a mechanical swap.

  3. Collision risk. A u32 over a few hundred member records is comfortable; over a large DM set the birthday bound gets less comfortable (~50% at ~77k entries). A collision means a genuine difference is not detected, so anti-entropy silently skips a heal. u64 costs 4 more bytes per entry and removes the concern; recommend u64 for message_signatures and u32 for member_info unless there is a reason to unify.

  4. This re-keys the contract. Changing summarize_state output changes the WASM, so the room contract key rotates and the standard migration applies (river#292 auto-migration; register the outgoing hash in common/legacy_room_contracts.toml). Coordinate with the river-publish flow, and bump the moderation key pin per the two-signal-staleness procedure.

  5. Old and new summaries are mutually unintelligible. A peer on the old contract and one on the new advertise different summary shapes. That is inherent to a contract re-key and handled by migration, but worth stating.

Relationship to the core-side work

Complementary, not redundant:

Even with hash-first, a mismatch still ships the full summary, so a 29 KB summary stays expensive on every genuine divergence. Both are worth doing, and this one needs no core protocol change or fleet-wide version gating.

Verification

  • Assert the serialized summary size for a synthetic room of N members and M DMs, before and after, so the reduction is pinned rather than assumed.
  • Round-trip determinism test: two independently-constructed identical states must summarize to identical bytes (the #4857 failure mode).
  • A test that a same-version content difference is still detected via the hash, i.e. the feat: member deputies for ban authority (invite-subtree moderation) #411 round 4 B behaviour the signature was added for.
  • After publish, re-measure interest_sync_summaries mean message size on the fleet; it should fall from ~29 KB toward single digits.

[AI-assisted - Claude]

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions