Add rooted derived-state events and bump 0.18.0

[Lythaeon]

Description

Adds rooted provider-backed derived-state events and finalized epoch-boundary events to SOF, wires them through the runtime and built-in provider adapters, audits the new event surface, and bumps the publishable crate set to 0.18.0.

This matters because the derived-state feed can now replay rooted vote/stake/sysvar/account observations and epoch transitions with explicit feed contracts, while strict capability expectations and built-in account-feed defaults now line up with the actual finalized-delivery semantics.

Changes

• crates/sof-observer/src/framework/derived_state.rs: added RootedAccountObserved and EpochBoundaryObserved feed families, rooted-account classification, control-plane finalized epoch tracking, replay support, and targeted tests.
• crates/sof-observer/src/runtime.rs: routed finalized provider account updates into derived state, extended capability-policy coverage for rooted-authoritative consumers, and added runtime tests for provider-backed derived-state dispatch.
• crates/sof-observer/src/provider_stream/websocket.rs: defaulted account/program subscriptions to finalized unless commitment was explicitly set and added regression tests.
• crates/sof-observer/src/provider_stream/yellowstone.rs: defaulted account feeds to finalized unless commitment was explicitly set and added regression tests.
• crates/sof-observer/src/provider_stream/laserstream.rs: defaulted account feeds to finalized unless commitment was explicitly set and added regression tests.
• crates/sof-observer/tests/derived_state_runtime_restart_e2e.rs: extended restart replay coverage to rooted-account and epoch-boundary events and tightened the assertion to exact replay order.
• crates/sof-observer/src/provider_stream.rs: refreshed the public doctest for the current TransactionEvent shape.
• crates/sof-tx/src/adapters/derived_state.rs and crates/sof-observer/src/framework/mod.rs: re-exported the new derived-state event surface to downstream consumers.
• crates/sof-observer/README.md, docs/gitbook/crates/sof.md, and docs/architecture/derived-state-feed-contract.md: documented the new feed families, ingest surface, and finalized-default behavior for built-in account feeds.
• Release metadata/docs: bumped sof, sof-tx, sof-types, sof-solana-compat, and sof-gossip-tuning to 0.18.0, updated internal dependency constraints, refreshed install snippets, and updated Cargo.lock.

For slice-related changes, include:

• Affected slices: observer runtime / derived-state feed, provider-stream adapters, tx adapters, release/docs surface.
• Cross-slice communication changes (if any) and why: the runtime now forwards finalized AccountUpdate events into derived-state replay, and downstream tx adapters can consume the new feed event types.
• Migration requirements (if any): consumers that relied on default built-in account-stream commitment being processed must now opt back in explicitly with .with_commitment(...Processed); no migration is needed for explicit configs.

Motivation

Business motivation:

• Make the replayable derived-state feed viable for rooted/account-authoritative local services and observer products that need restart recovery without validator-native Geyser dependencies.

Technical motivation:

• Expose finalized provider-backed account observations and epoch transitions as first-class replayable events.
• Align strict capability expectations with real delivery semantics by making built-in account feeds default to finalized when they are used for rooted-authoritative state.
• Strengthen replay correctness with end-to-end ordering checks.

Alternative approaches considered:

• Reject rooted-authoritative consumers in capability policy unless commitment could be proven finalized at runtime. This would have required plumbing config-level commitment knowledge into mode-level validation; defaulting built-in account feeds to finalized kept the current policy shape while making the default contract true.

Scope and impact

• Affected slices: sof derived-state/runtime/provider-stream paths, sof-tx derived-state adapters, release/docs metadata.
• Data/API changes: additive RootedAccountObserved and EpochBoundaryObserved feed families plus public re-exports; built-in account-feed defaults now use finalized commitment unless explicitly overridden.
• Backward compatibility: mostly additive; the only behavior change is the default commitment for built-in account feeds, and explicit processed/confirmed configs continue to work.
• Performance impact: low. The new feed families add lightweight envelope emission on finalized account updates and epoch transitions only.
• Security impact: no new privileged path; provider validation and replay behavior remain within existing trust boundaries.

Testing

• Unit tests
• Integration tests
• Manual verification
• Performance checks (if applicable)
• Security checks (if applicable)

Commands/results:

cargo fmt --all
cargo test -p sof yellowstone_account_stream_defaults_to_finalized_commitment --features provider-grpc --quiet
cargo test -p sof yellowstone_account_stream_preserves_explicit_processed_commitment --features provider-grpc --quiet
cargo test -p sof laserstream_account_stream_defaults_to_finalized_commitment --features provider-grpc --quiet
cargo test -p sof laserstream_account_stream_preserves_explicit_processed_commitment --features provider-grpc --quiet
cargo test -p sof provider_stream_strict_policy_allows_transaction_status_block_meta_and_rooted_account_derived_state --features provider-websocket,provider-grpc --quiet
cargo test -p sof derived_state_runtime_restart_replays_retained_tail_from_disk --features kernel-bypass --quiet
cargo test -p sof-tx --features sof-adapters --quiet

Manual verification also included a local Agave 3.1.10 + Yellowstone 12.2.0+solana.3.1.10 plugin run from staged artifacts under target/, including validator startup, yellowstone-grpc-client ping/get-version, and a successful SOF Yellowstone example connection.

Related issues and documentation

• Fixes: none
• Related: none
• Architecture docs: docs/architecture/README.md
• Relevant ARD/ADR: docs/architecture/derived-state-feed-contract.md
• Operations/runbook updates: none

Reviewer checklist

• Code follows project standards and architecture constraints
• Slice boundaries are respected (docs/architecture/ard/0003-slice-dependency-contracts.md)
• Tests added/updated and passing
• Documentation updated (README/docs/operations as needed)
• No undocumented breaking change
• Performance trade-offs documented where relevant
• Security considerations addressed where relevant

Additional notes