geoprobe: geoProbe server (doublezero-geoprobe-agent)

[nikw9944]

RFC Reference: rfc16-geolocation-verification.md — Phase 1, item 2 (geoProbe server)
Tracking issue: #2893

Goal

Build a new Go service that acts as the intermediary Probe: reflects TWAMP from DZDs, receives DZD Offsets, measures RTT to targets, generates composite Offsets with DZD references, and delivers them to targets.

Scope

• New binary at controlplane/telemetry/cmd/geoprobe-agent/main.go
• CLI flags: --additional-parents (DZD pubkeys + addresses), --additional-targets (target IPs), -verbose
• TWAMP Reflector: reuse twamplight.Reflector from tools/twamp/pkg/light/ to respond to DZD probes
• UDP Listener (port 8923): receive signed LocationOffset datagrams from DZDs
• Signature Verifier: validate Ed25519 signatures on incoming DZD Offsets using the shared geoprobe library
• Offset Cache: store recent DZD Offsets keyed by DZD pubkey in memory, with configurable TTL (max_offset_age_seconds) and max size
• Target Measurement: measure RTT to targets via TWAMP (for Phase 1 POC per RFC)
• Composite Offset Generation: create new LocationOffset with DZD Offset(s) as references, sign with probe's keypair
• UDP Sender: post composite Offset to targets on port 8923
• Key management: should follow the pattern established in controlplane/telemetry

Composite offset details

• RttNs is the sum of the reference RTT plus measured RTT to target
• lat/lng is copied from the reference (in this POC phase there will only be 1 reference)

Components

• main.go — flag parsing, component wiring, goroutine management (following patterns from existing cmd/telemetry/main.go)
• Offset cache — in-memory map with TTL eviction
• TWAMP pinger for targets — reuse TWAMP implementation in controlplane/telemetry
• Composite offset builder — attaches DZD offset references and computes total RttNs

Files to create

• controlplane/telemetry/cmd/geoprobe-agent/main.go

Estimated scope

Large (~500+ lines)

Dependencies

Depends on #2898 (shared geoprobe library). Independent of Issue 2.

Other

• Only make 1 changelog entry per component
• If you’re working on the RFC, add a review to make sure there’s nothing in the code that’s not strictly required for the rfc
• Run go-fmt before you commit

[ben-dz]

Work Plan (Revision 0)

Estimated Scope: Large

Work Plan: Issue #2900 — geoProbe Server (doublezero-geoprobe-agent)

Summary

Build a new Go binary (doublezero-geoprobe-agent) at controlplane/telemetry/cmd/geoprobe-agent/main.go that acts as the intermediary Probe in the RFC16 geolocation verification chain. The service reflects TWAMP probes from DZDs, receives signed LocationOffset datagrams from DZDs via UDP (port 8923), caches them, measures RTT to target devices via TWAMP, generates composite LocationOffset messages that include DZD offsets as references, and sends them to targets. This is Phase 1 item 2 from RFC16. The implementation heavily reuses the shared geoprobe library added in #2899.

Approach

Architecture

The geoprobe-agent is a standalone binary with these concurrent components:

1. TWAMP Reflector — Reuses twamplight.NewReflector() on port 862 (standard TWAMP port) to respond to DZD probes measuring RTT to this geoProbe.

2. UDP Offset Listener (port 8923) — Listens for signed LocationOffset datagrams from DZDs. Uses geoprobe.NewUDPListener() and geoprobe.ReceiveOffset(). Validates Ed25519 signatures via geoprobe.VerifyOffset(). Stores valid offsets in the offset cache keyed by DZD pubkey.

3. Offset Cache — In-memory map (map[[32]byte]*cachedOffset) storing the most recent DZD offset per DZD pubkey. Entries have a configurable TTL (default 60s via --max-offset-age-seconds). Protected by sync.RWMutex. A background goroutine evicts expired entries periodically.

4. Target TWAMP Pinger — Reuses the geoprobe.Pinger from the shared library to measure RTT to each target via TWAMP. Targets are specified via --additional-targets CLI flag (comma-separated host:port).

5. Composite Offset Builder & Sender — After each measurement cycle:

   • For each target with a successful RTT measurement, pick the best (freshest) DZD offset from the cache.
   • Create a composite LocationOffset where:
     • RttNs = reference DZD's RttNs + measured RTT to target
     • MeasuredRttNs = measured RTT to target
     • Lat/Lng = copied from the reference DZD offset
     • NumReferences = 1, References = [the DZD offset]
     • MeasurementSlot = current slot (from RPC, with cache)
   • Sign with the probe's Ed25519 keypair via geoprobe.NewOffsetSigner()
   • Send to target on port 8923 via geoprobe.SendOffset()

6. Main Loop — Ticker-based at the probe interval (default 10s). Each tick:

   1. Measure RTT to all targets via TWAMP pinger
   2. For each successful measurement, look up a valid DZD offset from cache
   3. Build composite offset with reference
   4. Sign and send to target

CLI Flags

Following the pattern from cmd/telemetry/main.go:

Flag	Default	Description
--additional-parents	""	Comma-separated DZD pubkeys + addresses (pubkey@host:port,...) for accepting offsets
--additional-targets	""	Comma-separated target addresses (host:port,...) to measure and send composite offsets
--keypair	""	Path to probe's Ed25519 keypair file (Solana keygen format)
--env	""	Network environment (devnet, testnet, mainnet-beta)
--ledger-rpc-url	""	Solana RPC URL (ignored if --env set)
--twamp-listen-port	862	Port for TWAMP reflector
--udp-listen-port	8923	Port for receiving DZD offset datagrams
--probe-interval	10s	Interval between measurement cycles
--twamp-sender-timeout	1s	Timeout for TWAMP probes to targets
--max-offset-age-seconds	60	TTL for cached DZD offsets
--verbose	false	Enable debug logging
--version	false	Print version and exit

Key Design Decisions

1. Reuse geoprobe package types extensively — LocationOffset, OffsetSigner, VerifyOffset, SendOffset, ReceiveOffset, NewUDPListener, Pinger, and ProbeAddress are all reused from the shared library in internal/geoprobe/.

2. Parent address format — The --additional-parents flag uses a pubkey@host:port format because the probe needs both the DZD's public key (to validate signatures on incoming offsets) and the network address (to know which DZDs to expect offsets from). For the POC, this is purely for validation — the probe accepts offsets from any source address but verifies the signature against the declared pubkey.

3. Simple offset cache — One entry per DZD pubkey (latest wins). This is sufficient for Phase 1 where there will only be 1 reference per composite offset. The cache enforces TTL to avoid using stale data.

4. Single file implementation — Per the issue spec, the implementation lives in a single main.go file with inline types for the offset cache and measurement loop. This keeps the code simple and avoids premature abstraction.

5. Slot fetching — The probe needs the current slot for MeasurementSlot in composite offsets. Uses Solana RPC with a 5-minute cache (same pattern as publisher.go).

Files to Change

Files to Create

File	Description
controlplane/telemetry/cmd/geoprobe-agent/main.go	Main binary (~500-600 lines). Contains CLI flag parsing, offset cache, UDP listener, measurement loop, composite offset generation, and component wiring.

Files to Modify

File	Change
controlplane/telemetry/Makefile	Add build target for doublezero-geoprobe-agent binary
CHANGELOG.md	Add entry under Unreleased > Changes > Telemetry for the new geoprobe-agent binary

Files NOT Modified

• controlplane/telemetry/internal/geoprobe/* — The shared library from geoprobe: telemetry agent extensions — probe measurement and offset delivery #2899 is used as-is. No modifications needed.
• controlplane/telemetry/cmd/telemetry/main.go — No changes to the existing telemetry agent.

Detailed Component Design

Offset Cache

type offsetCache struct {
    mu         sync.RWMutex
    entries    map[[32]byte]*cachedOffset
    maxAge     time.Duration
}

type cachedOffset struct {
    offset    geoprobe.LocationOffset
    receivedAt time.Time
}

• Put(offset *geoprobe.LocationOffset) — Store/replace offset keyed by offset.Pubkey
• Get(pubkey [32]byte) *geoprobe.LocationOffset — Return offset if not expired, nil otherwise
• GetAll() []geoprobe.LocationOffset — Return all non-expired offsets
• Evict() — Remove expired entries (called periodically)

UDP Listener Loop

for {
    offset, addr := ReceiveOffset(listener)
    verify signature against known parent pubkeys
    if valid: cache.Put(offset)
    if verbose: log receipt details
}

Measurement & Publish Loop

ticker every probe-interval:
    rttData := pinger.MeasureAll(ctx)
    for each target with RTT:
        dzdOffset := cache.GetBestOffset()  // freshest valid offset
        if dzdOffset == nil: skip (no valid reference)
        compositeOffset := buildComposite(dzdOffset, measuredRtt, slot)
        signer.SignOffset(&compositeOffset)
        SendOffset(conn, targetAddr, &compositeOffset)

Risks & Considerations

1. Port 862 requires root/CAP_NET_BIND_SERVICE — The standard TWAMP port is privileged. This is consistent with how the existing telemetry agent runs. The Makefile tests use sudo -E.

2. Single reference in POC — The issue states "in this POC phase there will only be 1 reference." The implementation picks the freshest valid DZD offset. If no valid offset exists (all expired), the probe skips that measurement cycle rather than sending stale data.

3. No onchain integration in Phase 1 — Parents and targets are specified entirely via CLI flags. No ledger queries for probe/target discovery. This simplifies the implementation significantly.

4. UDP is fire-and-forget — No delivery guarantees for offset messages. This is by design per RFC16 — the system is tolerant of dropped packets since measurements repeat every 10 seconds.

5. Parent pubkey validation — The --additional-parents flag specifies which DZD pubkeys are trusted. Offsets from unknown pubkeys are rejected. This prevents arbitrary entities from injecting offsets into the cache.

6. Clock/slot accuracy — The composite offset uses the probe's view of the current slot, which may differ slightly from the DZD's slot. This is acceptable for Phase 1.

Testing Strategy

1. Build verification — Ensure the binary compiles with make build in the telemetry Makefile and go build from repo root.

2. Unit tests for offset cache — Test TTL eviction, put/get, concurrent access, and expired entry handling. File: controlplane/telemetry/cmd/geoprobe-agent/main_test.go (or inline if simple enough).

3. go-fmt — Run make go-fmt before committing per project conventions.

4. Existing tests pass — Run go test ./controlplane/telemetry/... to ensure no regressions from the new code or Makefile changes.

5. Manual integration — Not in scope for this PR, but the binary can be tested end-to-end with the telemetry agent by:

   • Starting the geoprobe-agent with --additional-parents and --additional-targets
   • Starting the telemetry agent with --additional-child-probes pointing to the geoprobe-agent
   • Starting the geoprobe-target (Issue geoprobe: example target listener (TWAMP reflector + UDP offset receiver) #2901) to receive composite offsets

Estimated Scope

Large (~500-600 lines for main.go, plus Makefile and CHANGELOG changes)

[ben-dz]

Approved Work Plan (Revision 1)

Estimated Scope: Large

Work Plan: Issue #2900 — geoProbe Server (doublezero-geoprobe-agent)

Summary

Build a new Go binary (doublezero-geoprobe-agent) at controlplane/telemetry/cmd/geoprobe-agent/main.go that acts as the intermediary Probe in the RFC16 geolocation verification chain. The service reflects TWAMP probes from DZDs, receives signed LocationOffset datagrams from DZDs via UDP (port 8923), caches them, measures RTT to target devices via TWAMP, generates composite LocationOffset messages that include DZD offsets as references, and sends them to targets. This is Phase 1 item 2 from RFC16. The implementation heavily reuses the shared geoprobe library added in #2899.

Approach

Architecture

The geoprobe-agent is a standalone binary with these concurrent components:

1. TWAMP Reflector — Reuses twamplight.NewReflector() on port 862 (standard TWAMP port) to respond to DZD probes measuring RTT to this geoProbe.

2. UDP Offset Listener (port 8923) — Listens for signed LocationOffset datagrams from DZDs. Uses geoprobe.NewUDPListener() and geoprobe.ReceiveOffset(). Validates Ed25519 signatures via geoprobe.VerifyOffset(). Stores valid offsets in the offset cache keyed by DZD pubkey.

3. Offset Cache — In-memory map (map[[32]byte]*cachedOffset) storing the most recent DZD offset per DZD pubkey. Entries have a configurable TTL (default 1 hour via --max-offset-age). Protected by sync.RWMutex. A background goroutine evicts expired entries periodically. When selecting a reference offset for composite messages, the cache returns the entry with the shortest RttNs (i.e., the closest DZD), filtering out expired entries first.

4. Target TWAMP Pinger — Reuses the geoprobe.Pinger from the shared library to measure RTT to each target via TWAMP. Targets are specified via --additional-targets CLI flag (comma-separated host:port).

5. Composite Offset Builder & Sender — After each measurement cycle:

   • For each target with a successful RTT measurement, pick the best DZD offset from the cache (shortest RttNs).
   • Create a composite LocationOffset where:
     • RttNs = reference DZD's RttNs + measured RTT to target
     • MeasuredRttNs = measured RTT to target
     • Lat/Lng = copied from the reference DZD offset
     • NumReferences = 1, References = [the DZD offset]
     • MeasurementSlot = current slot (from RPC, with cache)
   • Sign with the probe's Ed25519 keypair via geoprobe.NewOffsetSigner()
   • Send to target on port 8923 via geoprobe.SendOffset()

6. Main Loop — Two tickers run concurrently:

   Measurement ticker (default 5m, --probe-interval):

   1. Measure RTT to all targets via TWAMP pinger
   2. For each successful measurement, look up the best valid DZD offset from cache (shortest RttNs)
   3. Build composite offset with reference
   4. Sign and send to target

   Target discovery ticker (60s, hardcoded):

   • Stub for future implementation. On each tick, checks for newly added targets (placeholder — currently a no-op with a TODO comment).
   • When target discovery is implemented, any new targets will receive an immediate one-off probe upon discovery. Subsequent probes for those targets happen on the next regular 5m measurement cycle alongside all other targets (no per-target timers).

CLI Flags

Following the pattern from cmd/telemetry/main.go:

Flag	Default	Description
--additional-parents	""	Comma-separated DZD pubkeys + addresses (pubkey@host:port,...) for accepting offsets
--additional-targets	""	Comma-separated target addresses (host:port,...) to measure and send composite offsets
--keypair	""	Path to probe's Ed25519 keypair file (Solana keygen format)
--env	""	Network environment (devnet, testnet, mainnet-beta)
--ledger-rpc-url	""	Solana RPC URL (ignored if --env set)
--twamp-listen-port	862	Port for TWAMP reflector
--udp-listen-port	8923	Port for receiving DZD offset datagrams
--probe-interval	5m	Interval between measurement cycles
--twamp-sender-timeout	1s	Timeout for TWAMP probes to targets
--max-offset-age	1h	TTL for cached DZD offsets
--verbose	false	Enable debug logging
--version	false	Print version and exit

Key Design Decisions

1. Reuse geoprobe package types extensively — LocationOffset, OffsetSigner, VerifyOffset, SendOffset, ReceiveOffset, NewUDPListener, Pinger, and ProbeAddress are all reused from the shared library in internal/geoprobe/.

2. Parent address format — The --additional-parents flag uses a pubkey@host:port format because the probe needs both the DZD's public key (to validate signatures on incoming offsets) and the network address (to know which DZDs to expect offsets from). For the POC, this is purely for validation — the probe accepts offsets from any source address but verifies the signature against the declared pubkey.

3. Best offset = shortest RttNs — When multiple DZD offsets are cached, the one with the shortest RttNs is selected as the reference. This picks the DZD that is geographically closest (lowest accumulated RTT), producing the tightest geolocation bound. Expired entries are filtered out before comparison.

4. Single file implementation — Per the issue spec, the implementation lives in a single main.go file with inline types for the offset cache and measurement loop. This keeps the code simple and avoids premature abstraction.

5. Slot fetching — The probe needs the current slot for MeasurementSlot in composite offsets. Uses Solana RPC with a 5-minute cache (same pattern as publisher.go).

6. Two-ticker main loop — The 5m measurement ticker drives regular probe cycles. A separate 60s ticker is a stub for future target discovery. When discovery is eventually implemented, newly discovered targets get an immediate one-off probe, then join the regular 5m cycle. This avoids per-target timers while still ensuring new targets are measured promptly.

7. 1-hour offset TTL — DZD offsets are cached for up to 1 hour by default. This is generous because DZD locations don't change frequently, and a longer TTL ensures the probe can still function even if a DZD temporarily stops sending offsets.

Files to Change

Files to Create

File	Description
controlplane/telemetry/cmd/geoprobe-agent/main.go	Main binary (~500-600 lines). Contains CLI flag parsing, offset cache, UDP listener, measurement loop with two tickers, composite offset generation, target discovery stub, and component wiring.

Files to Modify

File	Change
controlplane/telemetry/Makefile	Add build target for doublezero-geoprobe-agent binary
CHANGELOG.md	Add entry under Unreleased > Changes > Telemetry for the new geoprobe-agent binary

Files NOT Modified

• controlplane/telemetry/internal/geoprobe/* — The shared library from geoprobe: telemetry agent extensions — probe measurement and offset delivery #2899 is used as-is. No modifications needed.
• controlplane/telemetry/cmd/telemetry/main.go — No changes to the existing telemetry agent.

Detailed Component Design

Offset Cache

type offsetCache struct {
    mu      sync.RWMutex
    entries map[[32]byte]*cachedOffset
    maxAge  time.Duration
}

type cachedOffset struct {
    offset     geoprobe.LocationOffset
    receivedAt time.Time
}

• Put(offset *geoprobe.LocationOffset) — Store/replace offset keyed by offset.Pubkey
• Get(pubkey [32]byte) *geoprobe.LocationOffset — Return offset if not expired, nil otherwise
• GetBest() *geoprobe.LocationOffset — Return the non-expired offset with the shortest RttNs across all DZDs. Returns nil if no valid offsets exist.
• Evict() — Remove expired entries (called periodically)

UDP Listener Loop

for {
    offset, addr := ReceiveOffset(listener)
    verify signature against known parent pubkeys
    if valid: cache.Put(offset)
    if verbose: log receipt details
}

Main Loop (Two Tickers)

measureTicker := time.NewTicker(probeInterval)      // default 5m
discoveryTicker := time.NewTicker(60 * time.Second)  // fixed 60s

for {
    select {
    case <-measureTicker.C:
        // Full measurement cycle for all targets
        rttData := pinger.MeasureAll(ctx)
        for each target with RTT:
            dzdOffset := cache.GetBest()  // shortest RttNs
            if dzdOffset == nil: skip (no valid reference)
            compositeOffset := buildComposite(dzdOffset, measuredRtt, slot)
            signer.SignOffset(&compositeOffset)
            SendOffset(conn, targetAddr, &compositeOffset)

    case <-discoveryTicker.C:
        // TODO: Check for new targets (stub for future implementation).
        // When implemented, new targets will be added to the pinger and
        // receive an immediate one-off probe. Subsequent probes happen
        // on the next regular measurement tick.

    case <-ctx.Done():
        return
    }
}

Immediate Probe for New Targets (Stub Design)

The 60s discovery ticker is a placeholder. When target discovery is implemented, the flow will be:

1. Discovery tick fires → query for new targets (e.g., from onchain state)
2. For each newly discovered target:
   • Add to pinger via pinger.AddProbe()
   • Immediately measure RTT to just that target
   • Build and send composite offset
3. The new target then participates in regular 5m measurement cycles

For now, this is a no-op stub with a TODO comment and a log line at debug level.

Risks & Considerations

1. Port 862 requires root/CAP_NET_BIND_SERVICE — The standard TWAMP port is privileged. This is consistent with how the existing telemetry agent runs. The Makefile tests use sudo -E.

2. Single reference in POC — The issue states "in this POC phase there will only be 1 reference." The implementation picks the DZD offset with the shortest RttNs. If no valid offset exists (all expired), the probe skips that measurement cycle rather than sending stale data.

3. No onchain integration in Phase 1 — Parents and targets are specified entirely via CLI flags. No ledger queries for probe/target discovery. The 60s discovery ticker is stubbed out for future use. This simplifies the implementation significantly.

4. UDP is fire-and-forget — No delivery guarantees for offset messages. This is by design per RFC16 — the system is tolerant of dropped packets since measurements repeat every 5 minutes.

5. Parent pubkey validation — The --additional-parents flag specifies which DZD pubkeys are trusted. Offsets from unknown pubkeys are rejected. This prevents arbitrary entities from injecting offsets into the cache.

6. Clock/slot accuracy — The composite offset uses the probe's view of the current slot, which may differ slightly from the DZD's slot. This is acceptable for Phase 1.

7. 5m probe interval — The longer interval (vs. the telemetry agent's 10s) reflects that geoProbe measurements are less latency-sensitive. Targets still receive timely initial measurements via the one-off probe mechanism (once discovery is implemented).

Testing Strategy

1. Build verification — Ensure the binary compiles with make build in the telemetry Makefile and go build from repo root.

2. Unit tests for offset cache — Test TTL eviction, put/get, concurrent access, expired entry handling, and GetBest selection (shortest RttNs). File: controlplane/telemetry/cmd/geoprobe-agent/main_test.go.

3. go-fmt — Run make go-fmt before committing per project conventions.

4. Existing tests pass — Run go test ./controlplane/telemetry/... to ensure no regressions from the new code or Makefile changes.

5. Manual integration — Not in scope for this PR, but the binary can be tested end-to-end with the telemetry agent by:

   • Starting the geoprobe-agent with --additional-parents and --additional-targets
   • Starting the telemetry agent with --additional-child-probes pointing to the geoprobe-agent
   • Starting the geoprobe-target (Issue geoprobe: example target listener (TWAMP reflector + UDP offset receiver) #2901) to receive composite offsets

Estimated Scope

Large (~500-600 lines for main.go, plus Makefile and CHANGELOG changes)