Skip to content

test(cli): close three holes in the #441 membership-guard source pins - #546

Merged
sanity merged 2 commits into
mainfrom
fix-441-pins
Jul 29, 2026
Merged

test(cli): close three holes in the #441 membership-guard source pins#546
sanity merged 2 commits into
mainfrom
fix-441-pins

Conversation

@sanity

@sanity sanity commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Follow-up to #541, addressing findings from independent review. No production behaviour change — the authorization logic from #541 is untouched. This fixes the source pins that were supposed to protect it, and one inaccurate docstring.

Problem

Independent review found the #541 pins did not hold. I confirmed all three by executing the mutation against the full riverctl lib suite; it stayed green (294 passed) in every case.

1. The call-site needle stopped at a comma, before )?;

The needle was authorize_send(&room_state,room_owner_key,&sender_vk,. Two mutations matched it and survived:

  • let _ = authorize_send(&room_state, room_owner_key, &sender_vk, members_delta.as_ref());the authorization decision is discarded entirely. let _ = also silences unused_must_use, and there is no -D warnings anywhere in CI, so nothing else catches it either.
  • authorize_send(&room_state, room_owner_key, &sender_vk, None)?; — compiles via inference, and silently breaks the inactivity-rejoin path: a pruned member holding valid stored credentials can no longer send.

This is precisely the failure mode the pin's own doc comment claimed to prevent.

2. The scan count was method-sensitive, not merely name-sensitive

It matched only .any(|m| m.member.member_vk == ...) and missed .any(|m| m.member.id() == ...), which is equally owner-blind, is the more common idiom in this codebase (dm.rs:270,300,434, debug.rs), and has sender_member_id already in scope at the call site. A guard written that way keeps the count at 2 and passes.

The #541 commit message framed this whole class as closed. It was not — the fix there addressed variable renaming and stopped short of the scan method.

Separately, production was "everything before the test module", which still contained 15 mid-file #[cfg(test)] modules, so an unrelated future test could trip the count spuriously.

3. The docstring named the wrong contract-side backstop

It cited MessagesV1::verify. Verified against contracts/room-contract/src/lib.rs: update_state calls apply_delta (or merge) and never calls verify. What screens a message on the update path is MessagesV1::apply_delta's author retain — a MemberId comparison with no signature check. verify runs from validate_state, i.e. when a peer validates state, not as the gate on an update.

This does not weaken the owner branch, which is self-enforcing (it requires the caller to hold the owner's private key), but the comment overstated the backstop and would mislead the next reader.

Approach

The pins are now pure functions over source textproduction_source and membership_guard_violations — rather than inline assertions. That is the load-bearing change: it makes the pins themselves testable, which is exactly what was missing. Six pin_catches_* meta-tests feed deliberately-mutated source through membership_guard_violations and assert it objects.

A pin nobody has watched fail is not a pin. That is how all three of these shipped.

Specifics:

  • The call-site needle now runs through members_delta.as_ref() and the ?;, accepting both trailing-comma spellings so a future rustfmt that collapses the call onto one line cannot false-fail it.
  • Both scan idioms are counted.
  • production_source strips test modules line-wise on column-0 markers. Deliberately not brace-matching: api.rs holds 43 raw strings, and a matcher would have to lex them all correctly to avoid desynchronising. Every test module here starts with #[cfg(test)] at column 0 and ends with } at column 0, so this needs no lexer and cannot be confused by a brace inside a string or comment. Guarded by production_source_strips_test_modules_but_keeps_production, plus an early bail inside membership_guard_violations if the strip ever eats the send path (which would otherwise make every check pass vacuously).

Testing

301 tests pass. Every mutation below was applied to the real file and the suite re-run — not argued from inspection:

Mutation Before After
let _ = authorize_send(...) (authz discarded) survived caught
authorize_send(..., None) (rejoin path broken) survived caught
.any(|m| m.member.id() == sender_member_id) (MemberId idiom) survived caught
Call site fully re-inlined (the original #441 shape) caught caught
room_owner_key / sender_vk arguments swapped caught caught
Owner clause removed from room_has_member_key (reverts #541) caught caught

The first three are the review findings; the last three confirm this rewrite did not regress what the old pin did catch.

cargo fmt clean. No new clippy warnings — the six in riverctl are all pre-existing and outside this diff.

On the fourth review finding

The review also flagged cli/src/commands/identity.rs:290-300 as having the same owner-blind scan, telling the owner "Try sending a message first to populate membership data" — advice that can never work.

I could not reproduce this and believe it is a false positive, so I have not filed it. The scan sits in the final else of a three-way chain at identity.rs:264-300:

let is_owner = signing_key.verifying_key() == room_owner_key;   // :258
let (authorized_member, invite_chain) =
    if let Some(am) = room_info.self_authorized_member.clone() { ... }   // :265
    else if is_owner { /* synthesizes the owner's AuthorizedMember */ }  // :267
    else { /* :275 — the scan and that error message */ };

The owner is caught by else if is_owner at :267 and never reaches the scan. is_owner is computed from the resolved signing key, so a --signing-key-file override pointing at the owner's key still lands in the owner branch. The error string occurs exactly once in the file, reachable only when !is_owner, and for its actual audience — a pruned non-owner — "try sending a message first" is accurate advice, since a send triggers the rejoin path.

Happy to be shown wrong if the reviewer has a path I have missed.

Refs #441

[AI-assisted - Claude]

Independent review of #541 found the pins did not hold. All three were
confirmed by executing the mutation against the full riverctl lib suite,
which stayed green (294 passed) in every case.

1. The call-site needle stopped at a comma, before `)?;`. Two mutations
   survived it: `let _ = authorize_send(...)` discards the authorization
   decision entirely (and silences `unused_must_use`, with no `-D warnings`
   in CI to catch it), and `authorize_send(..., None)` compiles via
   inference while silently breaking the inactivity-rejoin path so a pruned
   member with valid stored credentials can no longer send. The needle now
   runs through `members_delta.as_ref()` and the `?;`, accepting both
   trailing-comma spellings so a future rustfmt cannot false-fail it.

2. The scan count was method-sensitive, not just name-sensitive: it matched
   only `member_vk ==` and missed `.any(|m| m.member.id() == ...)`, which is
   equally owner-blind, is the more common idiom in this codebase (dm.rs,
   debug.rs), and has `sender_member_id` already in scope at the call site.
   Both idioms are now counted. The previous commit message framed this
   class as closed; it was not.

   Also, `production` was everything before the test module, which included
   15 mid-file `#[cfg(test)]` modules, so an unrelated future test could
   trip the count. `production_source` now strips test modules line-wise
   (column-0 markers, no lexer — brace matching would have to handle 43 raw
   strings correctly).

3. The `authorize_send` docstring claimed `MessagesV1::verify` backs the
   guard contract-side. It does not: `update_state` runs `apply_delta`,
   never `verify`. What screens a message on the update path is
   `apply_delta`'s MemberId-based author retain, with no signature check;
   `verify` runs from `validate_state`. Corrected. The owner branch is
   unaffected — it is self-enforcing.

The pins are now pure functions over source text, so they are themselves
tested: six `pin_catches_*` meta-tests feed mutated source through
`membership_guard_violations` and assert it objects. A pin nobody has
watched fail is not a pin, which is exactly how these three got shipped.

Verified end-to-end: all six mutations (the three above plus call-site
re-inlining, argument swap, and reverting the owner clause) now turn the
suite red.

Refs #441

[AI-assisted - Claude]
@sanity

sanity commented Jul 29, 2026

Copy link
Copy Markdown
Contributor Author

Independent review — pin robustness lens

Reviewed at 7796c015 by a reviewer that did not write the code. Because it could not run cargo, it ported production_source and membership_guard_violations to Python verbatim and validated the port by reproducing the known baseline and all three previously-confirmed results. I then executed its two headline claims directly.

Verdict: this closes all three confirmed mutations. The pin machinery itself is not yet robust — three fixes recommended before merge.

What this PR fixes — confirmed

mutation (each previously left all 294 tests green on #541) now
let _ = authorize_send(...) CAUGHT
authorize_send(..., None)? CAUGHT
.any(|m| m.member.id() == sender_member_id) CAUGHT (count 3, expected 2)

The needle running through )?; and accepting both trailing-comma spellings is correct, as is counting both scan idioms.

Finding 1 — commenting the call out survives the pin proper

Replacing the call with // was: authorize_send(&room_state, room_owner_key, &sender_vk, members_delta.as_ref())?; leaves check 1 satisfied: the commented text squashes to a string containing both the prefix and the )?; tail.

Executed: the suite does go red — 299 passed, 2 failed — but the failures are pin_catches_discarded_authorization_result and pin_catches_rejoin_delta_replaced_by_none, both panicking must be caught, got: []. Those are the meta-tests failing because their str::replace can no longer find REAL_CALL to mutate. The pin itself is fooled; the red is a staleness side effect that disappears the moment REAL_CALL is updated or the meta-tests are refactored.

Fix: strip lines whose trimmed form starts with // before the substring search.

Finding 2 — the stripper can silently stop covering production

production_source skips from a line exactly #[cfg(test)] to the next column-0 }. Any #[cfg(test)] item that is not brace-terminateduse, const, type, mod foo; — makes the skip run past the item and swallow production code.

Executed, and this is the dangerous direction. Inserting

#[cfg(test)]
use std::fmt::Debug;

before unseal_nickname_display (cli/src/api.rs:715), together with a planted owner-blind helper in the swallowed window:

pub(crate) fn sneaky_owner_blind_guard(state: &ChatRoomStateV1, vk: &VerifyingKey) -> bool {
    state.members.members.iter().any(|m| m.member.member_vk == *vk)
}

leaves the full suite green — 301 passed. The scan-count never saw the new scan. A pin that silently stops covering production reads as protection while providing none.

Two related notes: the doc at :9412 says the file holds 15 #[cfg(test)] modules — it is 12 modules plus 3 #[cfg(test)] fns (:501, :785, :814), so the stripper already depends on non-module items happening to close at column 0. And the stated reason for rejecting brace-matching does not hold: the reviewer wrote a raw-string/comment-aware reference stripper and diffed it against the line-wise one over the real file — byte-identical, zero hunks. The 43 raw strings desynchronise nothing.

Fix (keeping the line-wise approach): after consuming #[cfg(test)], require the next line to open a braced item and push a violation otherwise.

Finding 3 — check 1 is not scoped to send_message_with_key

:9474 searches the whole production text while its failure message at :9483 claims it pins that function. Contrast check 3, which correctly scopes to fnbuild_rejoin_delta(. With eight near-identical send paths in this file, a sibling copy carrying the guard would satisfy a pin about the original.

Finding 4 — the scan count misses live idioms (defence-in-depth only)

Surviving the count: .find(..).is_some(), .position(..), swapped operands, renamed closure binding, and two idioms already in this codebase — HashSet<MemberId>::contains, which build_rejoin_delta itself uses at :3339-3347, and members_by_member_id(), the dominant idiom (api.rs:4949, debug.rs:76, deputies.rs:215, member_info.rs:135/289/324). A re-inline at the send path is still caught by check 1, so this is bounded — but the comment at :9495 should stop describing the count as robust against renaming.

Scope gap outside api.rs

cli/src/private_room.rs:264-268 is owner-safe only via the early return at :259 — the same order-dependent shape that is pinned in build_rejoin_delta, but with no pin. Deleting that early return reintroduces the #441 class in the private-room heal path with nothing to catch it.

Credit

The meta-tests operate on include_str!("api.rs") — the real file — and hard-fail if the target text is absent, so the committed tests carry end-to-end proof rather than only proving the checker objects to synthetic input. That is the strongest part of this PR. (Caveat: the owner-blind-scan meta-tests str::replace on a comment that occurs 12 times, so they insert 12 scans rather than 1 — assertions hold, but they are blunter than they read.)

The reviewer also independently reached the same conclusion as the author on the identity.rs finding from the earlier review: it is a false positive, the owner is caught by else if is_owner and never reaches that scan.

[AI-assisted - Claude]

…nged one

Round-3 review found the round-2 pins were not robust. All three findings
reproduced by execution before fixing.

F1 — a guard COMMENTED OUT IN PLACE satisfied the needle: the commented
text still contains the call verbatim. The suite did go red, but only
because the meta-tests' `str::replace` target had gone stale, so the
mutation no-opped and they failed on an empty violation list. That is
protection by accident — any later edit to `REAL_CALL` removes it, and
the pin proper stayed green throughout. Comment lines are now stripped
before the search.

F2 — the needle searched the whole file, so lifting the guard out of
`send_message_with_key` into an unrelated sibling left the pin green
while the send path ran unguarded. The search is now scoped to the send
path's own method body.

Q2 — `production_source` assumed every `#[cfg(test)]` guards a braced
item. A `#[cfg(test)] use ...;` made it skip to the next column-0 `}`,
eating the production code between. A whole owner-blind helper planted in
that window went undetected with the full suite green: a pin silently
ceasing to cover production.

The stripper is now a string-aware brace matcher rather than a line scan.
Two corrections behind that: the review disproved the stated reason for
avoiding brace matching (a raw-string-aware matcher produces
byte-identical output on this file), and then the line scan was disproved
outright — a meta-test added here embeds Rust source containing a
column-0 `}` inside a string literal, which the line scan read as the
module's closing brace, spilling test code into the production scan. Line
scanning was never safer; it had a different bug. Item kind is now
determined (braced item vs terminated statement) instead of assumed.

F3 — widened to `.find`/`.position` and swapped-operand spellings, and
the comment no longer claims robustness it does not have. Deliberately
still excludes `HashSet::contains` and `members_by_member_id()`: those
appear here but ask a different question (`build_rejoin_delta`'s
invite-chain filter asks which members to ADD, not whether the sender may
send), so counting them would trip on unrelated code and train people to
bump the number. The pins are now documented as tripwires for known-bad
shapes, not proofs of absence.

Also fixed a meta-test bluntness the review flagged: the scan-planting
tests anchored on a comment shared verbatim by all seven send/edit/
delete/react/unreact/reply paths, so `str::replace` planted seven copies
while reading as though it planted one. They now anchor on the guard call,
pinned unique by `guard_call_anchor_is_unique_in_production`.

All nine mutations now turn the suite red, six of them via the pin proper
rather than a meta-test side effect: `let _ =`, `None`, commented-out,
moved-to-sibling, hidden-behind-cfg-test, MemberId idiom, reverted owner
clause, swapped args, deleted early-return.

Refs #441

[AI-assisted - Claude]
@sanity
sanity merged commit 4c3641b into main Jul 29, 2026
6 checks passed
@sanity
sanity deleted the fix-441-pins branch July 29, 2026 19:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant