Skip to content

feat(moq-pub-mmtp): demand-driven SSM membership (subscribe-driven re-join) - #33

Closed
kkroo wants to merge 2 commits into
mainfrom
omar/shred-subscribe-rejoin
Closed

feat(moq-pub-mmtp): demand-driven SSM membership (subscribe-driven re-join)#33
kkroo wants to merge 2 commits into
mainfrom
omar/shred-subscribe-rejoin

Conversation

@kkroo

@kkroo kkroo commented Jul 8, 2026

Copy link
Copy Markdown

Problem

The moq-pub-mmtp bridge joins its SSM multicast group once at startup and holds it for the process lifetime. Without an IGMP/MLD querier on the VLAN, the switch's snooping entry ages out (~260s) and forwarding to the pod stops — the stream goes silent while a valid membership still sits in the kernel.

Fix — drive membership from subscriber demand (no timer)

  • moq-transport: Publisher::announce gains an optional on_subscribe: mpsc::Sender<(bool, String)> reporting each subscribe's lifecycle — (true, track) on start, (false, track) on end. None = unchanged behavior. All in-repo callers pass None.
  • moq-pub-mmtp: join the (S,G) group on the first subscribe, refresh it (leave+join → fresh IGMP/MLD report, re-arming the snooping entry) on each subsequent subscribe, and leave when the last subscriber drops. open_udp_socket now binds only for SSM targets; the join is deferred to run_udp_loop.

IPv4 + IPv6

  • IPv4: socket2 join_ssm_v4/leave_ssm_v4.
  • IPv6 (socket2 has no v6 SSM): MCAST_JOIN/LEAVE_SOURCE_GROUP via libc with a repr(C) group_source_req.
  • --mmtp-udp-source is now IpAddr; --mmtp-udp-iface-index selects the v6 interface.

56 crate tests pass; cargo check --workspace clean.

🤖 Generated with Claude Code

…-join)

The bridge held its SSM multicast membership for the lifetime of the
process and joined once at startup. Without an IGMP/MLD querier on the
VLAN, the switch's snooping entry for the group ages out (~260s) and
forwarding to the pod stops — the stream goes silent with a perfectly
valid membership still in the kernel.

Drive membership from subscriber demand instead of a timer:
- moq-transport `Publisher::announce` gains an optional
  `on_subscribe: mpsc::Sender<(bool, String)>` that reports each
  subscribe's lifecycle: (true, track) when it begins, (false, track)
  when the subscriber drops.
- moq-pub-mmtp joins the (S,G) group on the first subscribe, refreshes
  it (leave+join, forcing a fresh IGMP/MLD report) on each subsequent
  subscribe — re-arming the snooping entry — and leaves when the last
  subscriber drops. open_udp_socket now binds only for SSM targets; the
  join is deferred to run_udp_loop.

IPv4 uses socket2's join_ssm_v4/leave_ssm_v4; IPv6 (which socket2 does
not expose) uses MCAST_JOIN/LEAVE_SOURCE_GROUP via libc. --mmtp-udp-source
is now IpAddr; --mmtp-udp-iface-index selects the v6 interface.

56 crate tests pass; whole workspace compiles.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@cursor

cursor Bot commented Jul 8, 2026

Copy link
Copy Markdown

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

@allyblockcast allyblockcast Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ally — Consolidated PR Review

Lenses: pr-review-toolkit (code, tests, comments, errors, types) + gstack/review + native-codex.
reviewed head: afade31

Important Issues (2)

  • [native-codex] moq-pub-mmtp/src/main.rs:486-487refresh_ssm(...)? in the sub_rx.recv() arm propagates any join failure straight out of run_udp_loop, which tears down the whole tokio::select! in main() (session + announce + publisher loop all get cancelled). Before this PR a failed SSM join only happened once, at startup, before any subscriber existed. Now it can fire on every subsequent subscribe (refresh_ssm leave+joins each time), so one transient join hiccup (e.g. a momentary ENOBUFS/interface blip) when subscriber #2 joins kills the session for the already-connected subscriber #1 too. Contrast with leave_ssm (moq-pub-mmtp/src/udp.rs:214), which is explicitly best-effort and only logs on error. Recommend treating refresh_ssm failures the same way (log + leave joined/active_subs state as-is, retry on the next subscribe event) rather than propagating via ?, since a demand-driven rejoin failing shouldn't be worse than the timer-based staleness this PR is fixing.
  • [gstack/review] This PR bundles ~940 lines completely unrelated to the stated SSM-membership change: 8 new files under .planning/ (dev-session planning notes for a different repo's epic, Blockcast/multicast BLO-6020, discussing other agents/tickets) and 5 new files under .playwright-mcp/ (browser console logs / page snapshots from an unrelated portal debugging session). None of this belongs in moq-rs history — looks like local session artifacts picked up by a broad git add. Recommend dropping these 13 files from the PR (and adding .planning/ and .playwright-mcp/ to .gitignore if they're a recurring local-tooling byproduct) before merge.

Suggestions (2)

  • [pr-review-toolkit] moq-pub-mmtp/src/main.rs:475-479 — the let Some((started, track)) = ev else { continue } arm loops forever without yielding once sub_rx closes (mpsc recv() returns None immediately on a closed+drained channel, not pending). It's harmless today only because sub_tx lives inside publisher.announce(), which races run_udp_loop in the same top-level select! in main() — so the whole future tree gets dropped the moment announce() finishes, before this arm can spin. Worth a comment noting that invariant (or return Ok(()) instead of continue) so a future refactor that runs run_udp_loop outside that specific select! doesn't reintroduce a busy loop.
  • [pr-review-toolkit] No CI is configured on this branch (gh pr checks reports nothing), so the PR body's "56 crate tests pass; cargo check --workspace clean" is self-reported and unverified by an automated gate for this diff.

Strengths

  • ssm_membership cleanly separates "is this SSM" from socket state, validates source/group family agreement, and has direct unit coverage for both families plus the mismatch-error case.
  • The IPv6 MCAST_JOIN/LEAVE_SOURCE_GROUP group_source_req layout is done carefully — the repr(C) padding-for-8-align reasoning is spelled out and matches the kernel struct, and the unsafe blocks are scoped tightly with clear safety comments.
  • Demand-driven design (bind-only until first subscribe, leave-on-last-unsubscribe) is a real improvement over the timer-aging problem described in the PR, and the subscribe/unsubscribe accounting in moq-transport/publisher.rs correctly fires the "end" event on any task completion path (error, close, or normal drop), not just orderly unsubscribes.

Recommended Action

  1. Fix the two Important issues before merge — the error-propagation one is a real regression risk for existing viewers, and the bundled unrelated files shouldn't ship in this repo's history.
  2. Consider the Suggestions opportunistically.

…p-alive)

The subscribe-driven refresh in the prior commit doesn't hold membership for
the case that matters. MoQ subscription is not a recurring event: the relay
forwards exactly one upstream subscribe per track on the 0→1 downstream
transition and fans the rest out from the existing TrackReader. A stable single
viewer therefore produces one subscribe at warm-up and then silence, so nothing
re-arms the switch's IGMP snooping entry and forwarding dies at the ~260s
age-out — the exact bug this was meant to fix.

Membership maintenance is inherently periodic (that's what an IGMP querier
normally drives); with no querier on the receive fabric the host must self-drive
it. So: demand GATES membership (join on first subscribe 0→1, leave on last
1→0), a periodic timer MAINTAINS it (re-report while held). The interval is a
required CLI flag (--mmtp-membership-refresh-secs, no default) — it must sit
below the fabric IGMP Query Interval (default 125s); an unset value with SSM
enabled is a hard error, not a silent guess.

- cli.rs: add --mmtp-membership-refresh-secs (Option<u64>, documented).
- main.rs: UdpRecvConfig bundles the udp receive params (clears the
  too_many_arguments smell); membership_refresh_period() extracts the
  required-when-SSM decision as a pure, unit-tested fn; run_udp_loop joins on
  0→1, refreshes on a tokio interval while joined, leaves on 1→0. The timer arm
  is inert (never resolves) for ASM/unicast.
- 4 new tests for membership_refresh_period; 60 pass, crate is clippy-clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@cursor

cursor Bot commented Jul 8, 2026

Copy link
Copy Markdown

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

@kkroo

kkroo commented Jul 8, 2026

Copy link
Copy Markdown
Author

Design correction (790824d): the per-subscribe refresh in afade31 doesn't hold membership for a stable single viewer — the relay forwards one upstream subscribe per track (0→1) and fans the rest out, so there are no recurring subscribe events to refresh on, and the snoop entry still ages out at ~260s. Switched to the correct model: demand gates (join 0→1 / leave 1→0), a periodic timer maintains (--mmtp-membership-refresh-secs, required, no default, must be < fabric IGMP Query Interval ~125s). 60 tests pass, clippy-clean. Verifying on staging past the 260s window before merge.

@allyblockcast allyblockcast Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ally — Consolidated PR Review

Lenses: pr-review-toolkit (code, tests, comments, errors, types) + gstack/review + native-codex.

Important Issues (1)

  • [native-codex] .planning/*.md, .playwright-mcp/*.log, .playwright-mcp/*.yml (23 changed files, 13 of which are these) — this PR adds 13 files that are unrelated to the stated moq-pub-mmtp SSM membership feature: agent planning notes (.planning/blo-6020-next-phase-plan.md and 6 siblings, which reference an entirely different repo, Blockcast/multicast, plus internal engineering discussion — RecvArchitecture flip, MulticastEngineer status, Codex pool outage) and Playwright browser console/DOM captures (.playwright-mcp/console-*.log, .playwright-mcp/page-*.yml) from an unrelated UI debugging session. None of this belongs in moq-rs.gitignore doesn't exclude .planning/ or .playwright-mcp/, so these look like accidental git add -A / workspace-scratch commits carried over from a different task. Recommend dropping these 13 files from the branch (git rm + amend, or a follow-up commit) before merge, and adding .planning/ and .playwright-mcp/ to .gitignore to prevent recurrence.

Suggestions (2)

  • [pr-review-toolkit] moq-pub-mmtp/src/udp.rs (ssm_v6_source_group) — the hand-rolled group_source_req FFI (raw setsockopt with MCAST_JOIN/LEAVE_SOURCE_GROUP, manual sockaddr_in6 construction) is exactly the kind of code that benefits from a real integration test, but the PR's IPv6 SSM coverage is unit-level only (ssm_membership_matches_family_and_rejects_mismatch) — there's no test that actually exercises ssm_v6_source_group against a live socket the way the IPv4 SSM path is exercised in open_udp_socket's multicast tests. Understandable given CI multicast sandboxing constraints (existing tests already tolerate EPERM/ENOMEM gracefully), but worth calling out since this is unsafe, platform-specific code with no CI signal on the actual syscall path.
  • [pr-review-toolkit] No CI checks are configured/reported on this branch (gh pr checks returns "no checks reported") — the PR description's "56 crate tests pass; cargo check --workspace clean" is asserted from local runs only, not verified by an automated gate.

Strengths

  • The demand-driven join/refresh/leave state machine (run_udp_loop) is well-isolated and cleanly guarded: active_subs uses saturating_sub to avoid underflow on unmatched events, join/leave are gated on the 0→1/1→0 transitions (not per-subscribe), and the periodic refresh only fires if joined. The announceon_subscribe wiring in moq-transport correctly pairs each subscribe-start with exactly one subscribe-end (emitted when the corresponding subscribe_tasks future completes), so the counter can't drift.
  • membership_refresh_period fails closed — SSM without an explicit refresh interval is a hard error rather than silently reproducing the age-out bug this PR fixes, and that's covered by dedicated unit tests.
  • All five existing Publisher::announce call sites (moq-clock-ietf, moq-pub, moq-relay-ietf/producer.rs, moq-test-client/scenarios.rs ×4) were updated consistently with None, preserving prior behavior.

Recommended Action

  1. Drop the 13 unrelated .planning//.playwright-mcp/ files before merge (Important).
  2. Suggestions are opportunistic — not blocking.

@kkroo

kkroo commented Jul 8, 2026

Copy link
Copy Markdown
Author

Closing — the premise behind this PR is disproven.

This PR (demand-gated join + periodic SSM re-report) was built to fix a ~260s IGMP snoop-entry age-out on the shred receive fabric. Direct investigation shows that age-out does not exist:

  1. 700s control (SSM (S,G) receiver, ZERO re-reports) forwarded 292,416 pkts monotonically, no flatline — vs an identical refreshing pod (291,950). No age-out.
  2. AMT relay is the querier: linux-amt AMT_INIT_QUERY_INTERVAL=125, amt_gmi(); the relay sends IGMP General Queries every ≤125s.
  3. Live IGMP capture on the fabric: 0.0.0.0 > 224.0.0.1: igmp query v3 + receiver kernels auto-responding with v3 reports; /proc/net/igmp shows a V3 querier on net1.

The kernel already re-reports membership on every querier query, so a userspace periodic re-report is pure duplication. The demand-gating also regresses behavior (bridge idle without a subscriber). The real shred-over-MoQ fix is the source-specific join in #32 (already merged). The intermittent 'flat after ~100k pkts' is an AMT relay/gateway-layer stall class (cf. the source_port/GMI-window reconnect bug), not a receiver-side refresh problem — a linux-amt / amt-verify follow-up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant