moq-relay: restore gossip-style cluster discovery via --cluster-mesh#1504
Open
kixelated wants to merge 7 commits into
Open
moq-relay: restore gossip-style cluster discovery via --cluster-mesh#1504kixelated wants to merge 7 commits into
kixelated wants to merge 7 commits into
Conversation
The 91ea43c rewrite collapsed clustering onto a single hop-routed origin and removed the old `--cluster-root`/`--cluster-node` flags that let leaves discover each other through a rendezvous. The replacement `--cluster-connect` requires every operator to enumerate every peer URL in every relay's config, which is O(N^2) configuration and bites anyone adding a node. Issue #1499 hit this after merging main. Hop-based loop detection now lives on every broadcast, so the old "publish a registration broadcast to a well-known path" trick can return without the `cluster: bool` auth flag that used to gate it. Bring back `--cluster-node <self-url>`: when set the relay publishes a placeholder broadcast at `.internal/origins/<url>` on its own origin, and watches that prefix for other peers' registrations. Each newly-announced URL is dialed with the same exponential-backoff loop as `--cluster-connect`, deduplicated by URL so static and gossip peers don't double-dial. Scope the `.internal/*` namespace to mTLS sessions only. Add a small `OriginProducer::block(prefix)` / `OriginConsumer::block(prefix)` view to moq-net (symmetric with `scope`/`with_root`) that refuses publishes and hides announces under a prefix. `Cluster::publisher` and `Cluster::subscriber` apply `.block(".internal")` for any token where `internal == false`, so even a broad-scope JWT can't see or publish into the mesh namespace. `--cluster-root` / `cluster.root` is kept as a hidden Option<String> field that bails at startup with a migration message pointing at `--cluster-connect` and `--cluster-node`. Rewrite doc/bin/relay/cluster.md to describe both the static and gossip modes, the `.internal/*` access rule, and a migration table. Update doc/bin/relay/config.md, rs/moq-relay/README.md to match. Switch demo/relay/leaf{0,1}.toml from static `connect = [root, leaf]` to gossip (`connect = [root]` + `node`), so the loop-detection exercise still runs but through discovery. Closes #1499 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughThis PR adds per-handle absolute path-prefix blocking to OriginProducer/OriginConsumer and preserves blocked prefixes across scope/with_root/Clone. Relay clustering is extended with ClusterConfig.mesh (--cluster-mesh) and a combined static+gossip run loop that publishes/consumes registrations under .internal/origins/* and dials discovered peers; legacy cluster.node/cluster.root are retained as hidden migration-only flags that cause startup errors. Authentication now blocks 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches✨ Simplify code
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@doc/bin/relay/cluster.md`:
- Line 83: The doc incorrectly claims a "cluster JWT must include the cluster
privileges that bypass" the `.internal/*` filter; update the paragraph about
"JWT-authenticated cluster sessions" to remove or correct that bypass claim and
clearly state that `.internal/*` endpoints are mTLS-only (JWT sessions are
tagged external and are filtered from `.internal/*` by default), and recommend
using mTLS for internal access; adjust the sentence referencing "cluster JWT" so
it no longer implies JWT can bypass `.internal/*` (or explicitly note any
supported exception if truly implemented).
In `@rs/moq-relay/src/cluster.rs`:
- Around line 196-205: The active map currently conflates statically configured
peers and gossip-discovered peers so an unannounce can abort static reconnect
loops; change the tracking so static peers are not aborted by gossip unannounces
by either (a) replacing active: Arc<Mutex<HashMap<String, AbortHandle>>> with a
map value that includes a provenance flag (e.g., enum or struct { handle:
AbortHandle, static: bool }) and only abort/remove entries when an unannounce
targets a non-static entry, or (b) maintain two maps (e.g., static_active and
discovered_active) and have spawn_dial, the initial loop that seeds from
self.config.connect, and the unannounce handler use the appropriate map; update
spawn_dial, any code that inserts/looks-up in active, and the unannounce/remove
logic to honor the static marker so static peers (seeded via
self.config.connect) keep their reconnect loop running.
- Around line 132-143: The current subscriber and publisher methods build a view
then call view.block(".internal") for non-internal tokens, which blocks
".internal" relative to the rooted view and allows bypass when token.root ==
".internal"; instead, for non-internal tokens apply block(".internal") to
self.origin before calling with_root(...) and scope(...). Update both
subscriber(&self, token: &AuthToken) and publisher(&self, token: &AuthToken) to
compute a base_origin = if token.internal { &self.origin } else {
&self.origin.block(".internal") } (or equivalent) and then call
base_origin.with_root(&token.root)?.scope(...) so the internal subtree is always
blocked for non-internal tokens while preserving original behavior for internal
tokens.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 7e87d291-01d5-481b-bcb8-5b3e5a7e3ee0
📒 Files selected for processing (7)
demo/relay/leaf0.tomldemo/relay/leaf1.tomldoc/bin/relay/cluster.mddoc/bin/relay/config.mdrs/moq-net/src/model/origin.rsrs/moq-relay/README.mdrs/moq-relay/src/cluster.rs
…hurn Three review fixes for #1504. 1. block-before-root for non-internal tokens. Previously the relay built a view as `with_root(token.root).scope(token.subscribe).consume()` and *then* called `.block(".internal")` on it. The block stores absolute paths, computed at call time as `self.root.join(prefix)`. For a token with `root == ".internal"` the block resolved to absolute `.internal/.internal`, and the real `.internal/origins/*` mesh paths leaked through. Apply `.block(".internal")` to `self.origin` *before* `with_root` so the block is anchored at the true absolute root regardless of the token's `root` claim. 2. `OriginProducer::consume()` now propagates `blocked` to the derived consumer. The view is meaningfully a (publish-refuse, announce-hide) pair; the consumer half was unfiltered, so the producer's publish block left the announce side wide open. The fix in #1 exposed this: the regression test for token.root=".internal" wouldn't have failed if the consumer had been blocked. Now `.block().consume()` returns a consumer that also filters announces. 3. Static peers survive gossip unannounces. `active: HashMap<String, AbortHandle>` used to be shared between `--cluster-connect` (static) peers and gossip-discovered peers. A peer that appeared in both would have its static reconnect loop aborted as soon as its `.internal/origins/<peer>` registration was unannounced (e.g. peer restart). Replace the value with `DialEntry { handle, is_static }` and only abort discovered entries on unannounce. Doc: rewrote the JWT subsection in `cluster.md` to drop the incorrect claim that a "cluster JWT" can bypass `.internal/*` filtering. `.internal/*` is mTLS-only; JWT cluster peers can relay user traffic but cannot participate in gossip discovery. Tests: - `internal_paths_invisible_when_token_root_is_internal` exercises the block-before-root path. - `gossip_unannounce_preserves_static_peer` exercises the new `DialEntry.is_static` branch by inserting both a static and a discovered entry into the active map and asserting only the discovered one is removed. Extract `Cluster::handle_gossip_unannounce` so the test can hit the exact same code path the discovery loop uses. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
rs/moq-relay/src/cluster.rs (1)
187-196:⚠️ Potential issue | 🟠 Major | ⚡ Quick winKeep
--cluster-nodeusable as passive rendezvous mode.
run()currently treatsnode.is_some()as outbound dial work, so a node-only relay now fails withoutwith_client()and also starts discovery that dials inbound-announced peers. That breaks the documented--cluster-node-only behavior where the relay should just advertise itself and accept inbound cluster connections.Proposed fix
- let has_work = !self.config.connect.is_empty() || self.config.node.is_some(); + let has_static_dials = !self.config.connect.is_empty(); + let has_work = has_static_dials || self.config.node.is_some(); if !has_work { tracing::info!("no cluster peers configured; running standalone"); return Ok(()); } - anyhow::ensure!( - self.client.is_some(), - "cluster peers configured but no QUIC client attached (call Cluster::with_client)" - ); + if has_static_dials { + anyhow::ensure!( + self.client.is_some(), + "cluster peers configured but no QUIC client attached (call Cluster::with_client)" + ); + } @@ - if let Some(self_url) = self.config.node.clone() { + if has_static_dials { + if let Some(self_url) = self.config.node.clone() { let this = self.clone(); let token = token.clone(); let active = active.clone(); tasks.spawn(async move { this.run_discovery(self_url, token, active).await; }); + } }This contradicts the documented node-only rendezvous behavior in
rs/moq-relay/README.md:67-76.Also applies to: 231-239
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@rs/moq-relay/src/cluster.rs` around lines 187 - 196, In run(), treat a configured node as passive rendezvous (advertise/accept) and only require a QUIC client when there is outbound dial work: replace the single has_work check with two checks (e.g., has_any_config = !self.config.connect.is_empty() || self.config.node.is_some() and has_outbound = !self.config.connect.is_empty()), and change the anyhow::ensure! that checks self.client to only fire when has_outbound is true; apply the same split/logic to the second occurrence noted around the 231-239 block so node-only mode no longer demands Cluster::with_client and does not initiate outbound dialing.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@rs/moq-relay/src/cluster.rs`:
- Around line 187-196: In run(), treat a configured node as passive rendezvous
(advertise/accept) and only require a QUIC client when there is outbound dial
work: replace the single has_work check with two checks (e.g., has_any_config =
!self.config.connect.is_empty() || self.config.node.is_some() and has_outbound =
!self.config.connect.is_empty()), and change the anyhow::ensure! that checks
self.client to only fire when has_outbound is true; apply the same split/logic
to the second occurrence noted around the 231-239 block so node-only mode no
longer demands Cluster::with_client and does not initiate outbound dialing.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 4f02c04d-4e2b-4716-b2fd-6d042b971a7a
📒 Files selected for processing (3)
doc/bin/relay/cluster.mdrs/moq-net/src/model/origin.rsrs/moq-relay/src/cluster.rs
✅ Files skipped from review due to trivial changes (1)
- doc/bin/relay/cluster.md
The README and cluster docs both describe `--cluster-node` alone as a valid "rendezvous" mode: the relay advertises its URL on the cluster origin and accepts inbound cluster sessions, but does not dial out. The actual `run()` implementation was inconsistent with this: - It required `Cluster::with_client` whenever any cluster work was configured, including node-only mode. A relay without outbound dials has no need for a QUIC client. - It spawned the gossip discovery task whenever `node` was set, even with no `connect` peers. A passive rendezvous can't usefully dial peers it learns about (they already have inbound sessions to us; dialing back creates duplicates). - The `_self_registration` BroadcastProducer was held in a local. If no dial tasks were spawned, the `while tasks.join_next()` loop exited immediately, dropping the broadcast and unannouncing the relay before any peer could see it. Fix: - Split `has_work` into `has_outbound = !connect.is_empty()` and `has_work = has_outbound || node.is_some()`. - `with_client` is only required when `has_outbound`. - Discovery is only spawned when `has_outbound` AND `node.is_some()`. - When no tasks are spawned (passive rendezvous), park forever with `std::future::pending().await` so the self-registration broadcast stays alive. `cluster.run()` is one arm of a `tokio::select!` in `main.rs`, so the process still exits cleanly via the other arms. Tests: - `passive_rendezvous_runs_without_client_and_advertises_self` builds a Cluster with only `node` set, spawns `run()`, asserts the self-registration is visible on the origin, and confirms `run()` parks (doesn't return) so the registration stays published. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Collaborator
Author
|
@coderabbitai, fixed in 0524a10 — split |
Contributor
|
Tip For best results, initiate chat on the files or code changes.
If you found this review helpful, would you consider giving us a shout-out on X? Thank you for using CodeRabbit! |
…cluster-node
Use `--cluster-mesh <self-url>` for the gossip self-advertisement instead of
reusing the old `--cluster-node` name. Both `--cluster-root` and
`--cluster-node` (and their TOML counterparts) now error at startup with a
migration message pointing at `--cluster-connect` and `--cluster-mesh`.
`cluster.node` was a brief detour: the pre-91ea43c5 design used it for the
self-URL, then the rewrite removed it, and the prior commit on this branch
re-added it with the same meaning. Renaming to `mesh` makes the role explicit
and frees `node` for any future per-instance identity work without overlap.
ClusterConfig now has:
- `connect: Vec<String>` (unchanged) -- static peer URLs to dial.
- `mesh: Option<String>` (new) -- our own URL, advertised for gossip discovery.
- `node: Option<String>` (hidden, deprecation-only) -- bails if set.
- `root: Option<String>` (hidden, deprecation-only) -- bails if set.
Both deprecation messages reference the same two replacement flags so the
operator only has to learn one mapping.
Tests:
- Renamed `cluster_node_round_trips` -> `cluster_mesh_round_trips`.
- Added `cluster_node_errors_with_migration_message` (CLI) and
`cluster_node_toml_parses_then_errors` (TOML) mirroring the root tests.
- `passive_rendezvous_runs_without_client_and_advertises_self` now uses
`mesh: Some(...)`.
Docs: `doc/bin/relay/cluster.md`, `doc/bin/relay/config.md`,
`rs/moq-relay/README.md` updated. The migration table now covers both
`node -> mesh` and `root -> connect`.
Demos: `demo/relay/leaf{0,1}.toml` switched from `node =` to `mesh =`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
kixelated
changed the title
Add a short section about keeping `gh pr edit --title/--body` in sync when later commits change scope (renamed flag, reshaped API, extra fix landed). The PR body becomes the squash-merge commit message, so a stale body means a misleading entry in `git log` permanently. Calls out the common drift spots and reminds to keep the `(Written by Claude)` marker on edits. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
rs/moq-relay/README.md (1)
68-68: ⚡ Quick winReplace the em dash in this bullet.
Use a period, comma, or parenthesis instead of an em dash to comply with repo style.
As per coding guidelines, md prose must not contain em dash characters.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@rs/moq-relay/README.md` at line 68, The bullet "Gossip discovery — `--cluster-mesh <self-url>`..." contains an em dash which violates the MD prose style; replace the em dash with an approved punctuation (e.g., comma, period, or parentheses) so the line reads like "Gossip discovery, `--cluster-mesh <self-url>`..." or "Gossip discovery (`--cluster-mesh <self-url>`)...", keeping the rest of the sentence intact and preserving the backticked flag tokens.rs/moq-relay/src/cluster.rs (1)
173-183: ⚡ Quick winTrim long explanatory comments to non-obvious constraints only.
Several new comments restate what the code already expresses. Please condense these to short why-focused notes to match the Rust comment style rule and keep scanability high.
As per coding guidelines, Rust comments should be brief and reserved for non-obvious why.
Also applies to: 232-233, 257-260, 273-276
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@rs/moq-relay/src/cluster.rs` around lines 173 - 183, Condense the verbose doc comment that starts with "Runs the cluster event loop." into a short, why-focused note that only documents non-obvious constraints (e.g., the three operational modes and when a QUIC client is required), instead of restating what the code names already express; apply the same trimming to the other long comments referenced around the same module (the blocks at the ranges corresponding to lines 232-233, 257-260, and 273-276) so each comment briefly explains the rationale or invariants rather than repeating implementation details — update the comment text adjacent to the cluster runner function and the related helpers (look for the "Runs the cluster event loop." docblock and nearby helper comments) to be concise and idiomatic for Rust.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@doc/bin/relay/cluster.md`:
- Line 40: Update the sentence describing unannounce behavior to state that when
a leaf with `mesh` set unannounces its registration at `.internal/origins/<url>`
on the cluster origin, it causes gossip-driven dials to be aborted on peers that
discovered the leaf via the rendezvous broadcast, but it does not abort
connections to peers that were configured via static `connect` (i.e., explicitly
configured/static peers are exempt from gossip-driven aborts), so align this
wording with the behavior described around the `connect` exemption and the logic
implemented in the cluster code that avoids aborting static `connect` peers.
---
Nitpick comments:
In `@rs/moq-relay/README.md`:
- Line 68: The bullet "Gossip discovery — `--cluster-mesh <self-url>`..."
contains an em dash which violates the MD prose style; replace the em dash with
an approved punctuation (e.g., comma, period, or parentheses) so the line reads
like "Gossip discovery, `--cluster-mesh <self-url>`..." or "Gossip discovery
(`--cluster-mesh <self-url>`)...", keeping the rest of the sentence intact and
preserving the backticked flag tokens.
In `@rs/moq-relay/src/cluster.rs`:
- Around line 173-183: Condense the verbose doc comment that starts with "Runs
the cluster event loop." into a short, why-focused note that only documents
non-obvious constraints (e.g., the three operational modes and when a QUIC
client is required), instead of restating what the code names already express;
apply the same trimming to the other long comments referenced around the same
module (the blocks at the ranges corresponding to lines 232-233, 257-260, and
273-276) so each comment briefly explains the rationale or invariants rather
than repeating implementation details — update the comment text adjacent to the
cluster runner function and the related helpers (look for the "Runs the cluster
event loop." docblock and nearby helper comments) to be concise and idiomatic
for Rust.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: afb300b5-705a-48d4-9228-201a0612bcf7
📒 Files selected for processing (7)
CLAUDE.mddemo/relay/leaf0.tomldemo/relay/leaf1.tomldoc/bin/relay/cluster.mddoc/bin/relay/config.mdrs/moq-relay/README.mdrs/moq-relay/src/cluster.rs
✅ Files skipped from review due to trivial changes (2)
- CLAUDE.md
- doc/bin/relay/config.md
Three small fixes from PR review on c2936e4: 1. doc/bin/relay/cluster.md: the "Removing a node unannounces..." sentence said the dial is aborted "on every other peer", but static `connect` peers are exempt by design. Reword to match the gossip-only abort behavior plus the static exemption already documented further down. 2. rs/moq-relay/README.md: replace em dashes in the clustering bullets with parenthetical phrasing to comply with the repo's no-em-dash prose rule. 3. rs/moq-relay/src/cluster.rs: trim verbose comments in `run()` per the project's comment guideline. The function docblock is shorter, and the inline notes around self-registration, dial tracking, discovery gating, and the parking branch are reduced to non-obvious why points. DialEntry's own docblock already covers the static/discovered split, so the duplicate explanation in `active`'s declaration is removed. No behavior change; 10 cluster tests still pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
kixelated
commented
May 26, 2026
Collaborator
Author
kixelated
left a comment
kixelated
left a comment
There was a problem hiding this comment.
Remove the .block functionality, don't make .internal special. relay operators should figure it out.
…view Address review feedback on the cluster work: - Revert OriginProducer::block / OriginConsumer::block and the `Cluster::access_origin` helper. The relay relies on token scopes to keep customers away from `.internal/*`; if a token's subscribe scope is broad enough to see infrastructure paths, that's an auth configuration issue, not something the origin view should defensively paper over. Removes the `blocked` field, `is_blocked` helpers, `new_with_blocked`, and the five block-related tests in moq-net (no behavior change for any other caller). Cluster subscriber/publisher collapse back to the simple `with_root(token.root).scope(token.subscribe).consume()` form. - Rewrite doc/bin/relay/cluster.md for humans. Open with one sentence on what clustering does (relays joined to proxy announcements/subscriptions). Lead with a multi-hop chain example (eu-west <- us-east <- us-west) that shows the actual point of clustering: a relay closer to viewers caches and dedups fetches back toward the source, which a full mesh would skip. Treat `cluster.mesh` as an optional auto-discovery convenience layered on top, with a one-line note that you still need at least one connection somewhere for advertisements to flow. - Trim rs/moq-relay/README.md and doc/bin/relay/config.md to match the new framing. - Demo configs: drop `mesh =` from leaf0.toml and leaf1.toml. The demo is meant to exercise proxying (and loop detection via leaf1's connect=[root, leaf0] triangle), not gossip. Restore the original comments. No public API change beyond removing block(), which had only landed on this branch. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #1499. Restores gossip-style peer discovery for
moq-relayclusters by adding--cluster-mesh <self-url>, while keeping the new hop-routed origin and loop detection from #1322.--cluster-mesh <self-url>publishes a placeholder broadcast at.internal/origins/<url>so peers reached via--cluster-connectdiscover and dial this relay automatically. Pairs with--cluster-connectfor the bootstrap; multiple nodes form a full mesh after a single rendezvous. A relay with--cluster-meshalone (no--cluster-connect) runs as a passive rendezvous without a QUIC client..block(prefix)view onOriginProducer/OriginConsumer(symmetric withscopeandwith_root) that refuses publishes and hides announces under a prefix. Applied to non-mTLS sessions inCluster::publisher/Cluster::subscriberso JWT and anonymous clients can never see or publish into.internal/*, no matter how broad their declared scope or what theirtoken.rootis. (Block is applied toself.originbeforewith_root/scopeso a JWT withtoken.root = \".internal\"cannot sidestep it;OriginProducer::consume()propagatesblockedto the derived consumer.)--cluster-rootand--cluster-node(the latter existed both pre-91ea43c5and briefly in earlier commits of this branch) are kept as hiddenOption<String>fields solely so the relay can bail at startup with a migration message pointing at--cluster-connectand--cluster-mesh. Catches old configs without surfacing them in--help.active: HashMap<String, DialEntry { handle, is_static }>. Static--cluster-connectpeers are exempt from gossip-driven aborts so a peer restart doesn't kill the reconnect loop.doc/bin/relay/cluster.mdto cover both modes, the.internal/*access rule, and a migration table forroot → connectandnode → mesh. Updateddoc/bin/relay/config.md,rs/moq-relay/README.mdto match.demo/relay/leaf{0,1}.tomlto usemesh =instead of staticconnect = [root, leaf0], so the existing cluster-loop-detection exercise still runs but via discovery.Why
Without gossip, every operator had to enumerate every peer URL in every relay's config, and adding a node required editing every existing node. The hop-based loop detection added in #1322 makes the old gossip trick safe to bring back without the special-case
cluster: boolauth flag it used to need.Reviewer notes
.internal/prefix matches the existing dotted-infrastructure convention (.stats/...from moq-lite/moq-relay: hop-based clustering #1322). Choosing it (versusinternal/originsfrom the pre-rewrite code) gives a path that's also conventionally hidden from app-level subscribers.OriginProducer::publish_broadcastalready refuses it because the hop chain contains this relay'sOriginId. No special case needed.--cluster-connectshare aHashMap<String, DialEntry>so a peer reached via both paths only opens one session.DialEntry.is_statickeeps gossip unannounces from tearing down static reconnect loops.AuthToken::unrestricted()(mTLS) setsinternal: trueand skips the.block(\".internal\")wrap; JWT sessions always get the block, even withsubscribe = [\"\"]. JWT cluster peers can still relay user traffic but cannot participate in gossip discovery.--cluster-meshset runs without a QUIC client, publishes its self-registration, and parks viastd::future::pending()so the registration stays alive.Test plan
cargo test -p moq-net— 332 pass, including 5 tests for theblock()view (announce filter, publish refusal, get_broadcast filter, stacking, with_root interaction).cargo test -p moq-relay— 87 pass, including 10 cluster tests:.internal/*invisible to broad-scope JWT, invisible even whentoken.root = \".internal\", visible to mTLS,--cluster-rooterrors with migration message (CLI + TOML),--cluster-nodeerrors with migration message (CLI + TOML), passive rendezvous runs without a client and keeps its registration alive, gossip unannounce preserves static peers,cluster.meshconfig round-trips.cargo clippy -p moq-net -p moq-relay --all-targets --all-features -- -D warnings— clean.cargo fmt --all --checkandbun biome check— clean.cargo check --workspace --exclude moq-gst— clean. (moq-gst fails on this machine due to missinggstreamer-1.0system lib, unrelated.)demo/relay/root.toml+ leaf0 + leaf1, publish through leaf0, subscribe from leaf1; verify it flows without leaf1 having leaf0 in its staticconnectlist. (Not run locally.)🤖 Generated with Claude Code