Skip to content

feat(runtime): combine provider ingress with gossip topology#84

Merged
Lythaeon merged 2 commits into
mainfrom
feat/provider-gossip-control-plane-compat
Mar 29, 2026
Merged

feat(runtime): combine provider ingress with gossip topology#84
Lythaeon merged 2 commits into
mainfrom
feat/provider-gossip-control-plane-compat

Conversation

@Lythaeon

@Lythaeon Lythaeon commented Mar 29, 2026

Copy link
Copy Markdown
Owner

Description

This PR makes the packaged provider-stream runtime compatible with gossip-sourced control-plane topology, so built-in websocket / Yellowstone / LaserStream transaction ingress can run alongside gossip bootstrap without forcing ProviderStreamMode::Generic just to satisfy topology-driven consumers such as sof-tx.

It also prepares the coordinated 0.17.3 release, including the vendored gossip crate bump.

Changes

Detailed list of what changed:

  • crates/sof-observer/src/runtime.rs
    • allow built-in provider ingress modes to coexist with gossip-backed on_cluster_topology
    • keep on_leader_schedule and on_reorg rejected for mixed packaged provider+gossip mode
    • tighten capability validation around mixed ingress/control-plane setups
  • crates/sof-observer/src/app/runtime/**
    • wire gossip control-plane topology into the packaged provider runtime path
    • keep gossip control-plane-only behavior honest and explicitly topology-only
    • fix control-plane-only bootstrap acceptance/stabilization behavior from the audit passes
    • respect explicit min-peer overrides in control-plane-only mode
  • crates/sof-tx/src/adapters/plugin_host.rs
    • add topology-only adapter support so direct routing can use gossip topology without falsely requiring leader schedule
  • crates/sof-observer/src/runtime_env.rs
    • harden env override handling for tests so runtime env-sensitive tests are stable under parallel test runners
  • Docs
    • update SOF / sof-tx README and GitBook docs to describe the mixed provider-ingress + gossip-topology model accurately
    • document that built-in websocket / Yellowstone / LaserStream can pair with gossip topology, but not with packaged leader schedule in this mode
    • update install/version snippets for 0.17.3
  • Release prep
    • bump workspace crates from 0.17.2 to 0.17.3
    • bump sof-solana-gossip from 3.1.8-sof.7 to 3.1.8-sof.8
    • refresh root and fuzz lockfiles

Any architecture/runtime/infra implications:

  • Packaged SOF can now combine built-in processed ingress with gossip-derived topology in one runtime.
  • This remains intentionally narrower than a full custom ProviderStreamMode::Generic integration: topology is supported, leader schedule is not synthesized from gossip alone.

Motivation

Business motivation:

  • Users want to combine built-in websocket / gRPC transaction feeds with gossip-derived topology for sof-tx direct routing without having to drop into custom generic provider mode.
  • This makes the packaged runtime materially more useful for production mixed-ingest setups while keeping the contract honest.

Technical motivation:

  • 0.17.2 rejected a real and valid architecture: provider transaction ingress + gossip control-plane topology.
  • The runtime and docs needed to reflect what gossip can actually provide cleanly: cluster topology, not leader schedule.

Alternative approaches considered:

  • Relax validation only and document a workaround: rejected because the packaged runtime should actually support the mixed topology use case.
  • Pretend gossip also provides leader schedule: rejected because that overclaims what gossip control-plane data can supply.

Scope and impact

  • Affected slices: observer runtime, gossip bootstrap/control-plane path, sof-tx adapter surface, docs/release manifests
  • Data/API changes:
    • packaged mixed mode now supports built-in provider ingress + gossip topology
    • topology-only sof-tx adapter usage is supported/documented
  • Backward compatibility:
    • additive behavior for valid mixed setups
    • existing rejection remains for unsupported control-plane hooks such as leader schedule in packaged mixed mode
  • Performance impact:
    • no intended hot-path regression; changes are capability/control-plane oriented
  • Security impact:
    • no new privileged behavior; no security-sensitive surface expanded beyond documented runtime wiring

Testing

  • Unit tests
  • Integration tests
  • Manual verification
  • Performance checks (if applicable)
  • Security checks (if applicable)

Commands/results:

cargo fmt --all
cargo test -p sof --lib control_plane_only_bootstrap_accepts_peer_discovery_without_packets --features gossip-bootstrap -- --nocapture
cargo test -p sof --lib control_plane_only_stabilization_returns_on_peer_discovery_without_packets --features gossip-bootstrap -- --nocapture
cargo test -p sof --lib full_gossip_bootstrap_still_requires_packets_for_peer_discovery_fallback --features gossip-bootstrap -- --nocapture
cargo test -p sof --lib control_plane_only_bootstrap_respects_explicit_min_peers_override --features gossip-bootstrap -- --nocapture
cargo check -p sof --all-features
cargo check --manifest-path crates/sof-tx/fuzz/Cargo.toml
cargo check --manifest-path crates/sof-gossip-tuning/fuzz/Cargo.toml
TMPDIR=/home/ac/projects/sof/target/tmp-ci cargo make ci

All passed.

Related issues and documentation

  • Fixes: mixed built-in provider ingress + gossip topology packaged runtime compatibility for 0.17.3
  • Related: 0.17.2 gossip control-plane-only release follow-up
  • Architecture docs: docs/architecture/README.md
  • Relevant ARD/ADR:
  • Operations/runbook updates: docs updated in README and GitBook control-plane recipes

Reviewer checklist

  • Code follows project standards and architecture constraints
  • Slice boundaries are respected (docs/architecture/ard/0003-slice-dependency-contracts.md)
  • Tests added/updated and passing
  • Documentation updated (README/docs/operations as needed)
  • No undocumented breaking change
  • Performance trade-offs documented where relevant
  • Security considerations addressed where relevant

Additional notes

Trade-offs, follow-up work, or deviations from standards.

  • This PR deliberately stops at gossip-derived topology support. It does not attempt to derive a full leader schedule from gossip control-plane data alone.
  • If we want low-overhead leader resolution beyond topology, the next step is likely a separate leader-schedule cache/engine layered on top of SOF rather than further stretching the packaged gossip contract.

@Lythaeon Lythaeon merged commit 48d4ed4 into main Mar 29, 2026
2 checks passed
@Lythaeon Lythaeon deleted the feat/provider-gossip-control-plane-compat branch March 29, 2026 23:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant