sentinel: set tunnel endpoint on multicast publisher create#3583
Merged
sentinel: set tunnel endpoint on multicast publisher create#3583
Conversation
Mirror the CLI's behavior at client/doublezero/src/command/connect.rs:310: pick a tunnel endpoint on the user's device (public_ip or a user_tunnel_endpoint interface IP) that is not already in use by another user at the same client_ip, instead of sending UNSPECIFIED and letting the activator choose.
snormore
force-pushed
the
snor/sentinel-tunnel-endpoint
branch
from
9184543 to
7f8495a
Compare
snormore
requested review from
bgm-malbeclabs,
elitegreg,
juan-malbeclabs and
packethog
packethog
requested changes
Apr 24, 2026
Users created before the tunnel_endpoint field was populated onchain store UNSPECIFIED, but the activator implicitly routes their tunnel through the device's public_ip. The exclude list built for new multicast publisher creates was dropping these users, so the sentinel could select public_ip and collide with an existing legacy tunnel. Resolve UNSPECIFIED users to their device's public_ip when building the exclude list (mirrors client/doublezero/src/command/connect.rs:939).
packethog
approved these changes
Apr 24, 2026
elitegreg
approved these changes
Apr 24, 2026
…ilable When all of a device's tunnel endpoints (public_ip + UTEs) are already in use by the candidate's client_ip, selection returns UNSPECIFIED. Previously the sentinel forwarded that to create_multicast_publisher and the activator rejected the user, burning a create tx for a guaranteed rejection. Skip the candidate with an error log and doublezero_sentinel_multicast_pub_no_endpoint metric; next poll cycle retries if state changes. The admin CLI's create command does the same (writes to stderr).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary of Changes
tunnel_endpointwhen creating a multicast publisher instead of sendingIpv4Addr::UNSPECIFIED. It prefers auser_tunnel_endpointinterface IP on the user's device and falls back to the device'spublic_ip, excluding any IP already used by another user at the sameclient_ip— mirroring the CLI behavior atclient/doublezero/src/command/connect.rs:310.tunnel_endpointmodule (select_tunnel_endpoint,in_use_tunnel_endpoints,select_tunnel_endpoint_for_user) with unit tests.MulticastDzLedgerClientwithfetch_all_device_endpointssopoll_cyclecan resolve endpoints for each candidate; the RPC impl derives them from activated Device accounts.DzUsernow carriestunnel_endpointandDzDeviceInfocarriespublic_ip+user_tunnel_endpoints;build_create_multicast_publisher_instructionstakes an explicittunnel_endpointargument.create-validator-multicast-publisherscommand.Diff Breakdown
Overwhelmingly additive: new selection module + wiring, plus small field additions to existing data types.
Key files (click to expand)
crates/sentinel/src/tunnel_endpoint.rs— new module:select_tunnel_endpoint,in_use_tunnel_endpoints,select_tunnel_endpoint_for_user, and unit testscrates/sentinel/src/multicast_publisher.rs— addsfetch_all_device_endpointsto the trait, implements it inRpcMulticastDzLedgerClient, wires selection intopoll_cycle, updates mocks and adds a test asserting the multicast publisher skips an IBRL-occupied UTEcontrolplane/doublezero-admin/src/cli/sentinel.rs— computes the exclude list fromall_usersfor each candidate and passes the selected endpoint tobuild_create_multicast_publisher_instructionscrates/sentinel/src/dz_ledger_reader.rs— readsuser.tunnel_endpointintoDzUser; populatespublic_ip+user_tunnel_endpointsonDzDeviceInfofrom device interfacescrates/sentinel/src/dz_ledger_writer.rs—build_create_multicast_publisher_instructionstakestunnel_endpoint: Ipv4Addrinstead of hardcodingUNSPECIFIEDTesting Verification
tunnel_endpoint_is_selected_excluding_ibrl_endpointinmulticast_publisher.rs: device exposes two UTE IPs, the IBRL user occupies the first, and the assertedcreate_multicast_publishercall receives the second.select_tunnel_endpointacross all branches (UTE preferred, UTE excluded, public_ip fallback, everything excluded, no endpoints at all) andselect_tunnel_endpoint_for_user(exclude-list derivation, unknown device).cargo test -p doublezero-sentinel -p doublezero-admin— 56 + 8 tests pass.