Adaptive Base-Fee Oracle & Network Congestion Policy

[devsimze]

Problem Statement

The outbox dispatcher (src/outbox/dispatcher.ts) reacts to congestion after the fact: computeFeeMultiplier raises the fee only once an op has already failed or gone unconfirmed past submittedTimeoutMs, and it does so with a fixed multiplier compounded per attempt. It never reads the network — it has no idea whether the ledger is currently full, what the recent inclusion fee has been, or whether the surge just started or is ending. The first transaction of a congestion spike always pays the base fee, fails, and eats a full backoff cycle before the multiplier kicks in. This issue adds an adaptive base-fee oracle: a small service that samples recent ledger fee stats, publishes a recommended base fee and a congestion signal, and feeds both the dispatcher (proactive fee setting) and the API (honest "this will cost ~X and take ~Y" estimates), with full telemetry.

Current State

• src/outbox/dispatcher.ts — computeFeeMultiplier(attempts) = feeBumpMultiplier ** min(attempts-1, feeBumpMaxAttempts); applied only on retry. reconcileStuckSubmitted escalates ops unconfirmed past config.outbox.submittedTimeoutMs.
• src/stellar/client.ts — ResilientRpcClient can run any server.* call with failover; nothing currently calls getFeeStats / getLatestLedger.
• src/config/env.ts — config.outbox.* (feeBumpMultiplier, feeBumpMaxAttempts, submittedTimeoutMs, batchSize, concurrency caps).
• src/utils/metrics.ts — recordOutboxFeeBump, recordOutboxLatency, updateOutboxQueueDepth; src/utils/rpc-metrics.ts for RPC health.
• deploy/monitoring/grafana/dashboards/latency.json + dlq.json; docs/OUTBOX.md documents the current fee-bump cap.

Proposed Solution

1. Fee oracle service (src/stellar/feeOracle.ts, new)

• Poll server.getFeeStats() (and getLatestLedger for ledgerCapacityUsage) every FEE_ORACLE_POLL_MS (default 10s) through getResilientClient().
• Maintain a short in-memory (and Redis-mirrored, src/config/redis.ts) history and publish a snapshot:

interface FeeSnapshot {
  recommendedBaseFee: number      // stroops, e.g. p70 of recent inclusion fees, floored at 100
  aggressiveBaseFee: number       // p95, for CRITICAL ops during congestion
  congestionLevel: 'low' | 'elevated' | 'high' | 'severe'
  ledgerCapacityUsage: number     // 0..1
  sampledAt: string
  ttlMs: number
}

• congestionLevel is derived from ledgerCapacityUsage and the spread between min and p95 inclusion fees, with hysteresis so it doesn't flap.
• On poll failure the snapshot goes stale (never silently reused past ttlMs); consumers fall back to a configured safe default and the staleness is a metric + alert.

2. Dispatcher integration

• submitClaimedOp sets the transaction base fee from the oracle before the first attempt:
  • LOW priority (rebalances): recommendedBaseFee, and if congestionLevel >= 'high', defer — leave the op PENDING with a short nextAttemptAt so low-value moves wait out the surge (documented, bounded max defer).
  • NORMAL: recommendedBaseFee.
  • CRITICAL (withdrawals): aggressiveBaseFee during elevated+ congestion — a user's money leaving never waits on a fee surge.
• The per-attempt computeFeeMultiplier still applies on top of the oracle base for retries, but the cap is now expressed as an absolute max fee (OUTBOX_MAX_ABS_FEE) so compounding a high oracle base doesn't overpay without bound.
• reconcileStuckSubmitted uses the current oracle base for the fee-bump resubmission, not a multiple of the original.

3. API estimates

• GET /api/v1/network/conditions — current FeeSnapshot (sans internal stroop math if desired), plus an ETA band per priority derived from recent recordOutboxLatency percentiles.
• Deposit/withdraw responses (and the routing quote from the path-payment issue) include estFee and estConfirmationSeconds from the oracle so the UI can show honest numbers.
• docs/OUTBOX.md gains a "Fee oracle & congestion policy" section; docs/API_REFERENCE.md documents the new endpoint and response fields.

4. Telemetry

• Gauges: recommendedBaseFee, ledgerCapacityUsage, congestionLevel (as a numeric enum), oracle staleness seconds.
• Counters: LOW ops deferred due to congestion, CRITICAL ops that used aggressiveBaseFee, ops that hit OUTBOX_MAX_ABS_FEE.
• A Grafana panel on the latency dashboard; alert rules for "oracle stale > N s" and "severe congestion > M min".

Edge Cases & Failure Modes

• Oracle cold start (no samples yet): consumers use the configured default base fee; deferral is disabled until the first good snapshot.
• RPC returns degenerate fee stats (all equal, or absurdly high): clamp recommendedBaseFee/aggressiveBaseFee to [FEE_ORACLE_MIN, FEE_ORACLE_MAX]; a clamp event is a metric.
• Congestion flapping: hysteresis bands + a minimum dwell time per congestionLevel; unit-test that a single noisy sample doesn't move the level.
• Deferred LOW op starvation: a hard cap on total defer time per op; after it, the op dispatches at recommendedBaseFee regardless of congestion (a rebalance is never cancelled by congestion, only delayed).
• Absolute fee cap hit on a CRITICAL op: submit at the cap anyway and emit a critical alert — a withdrawal is never blocked by the cap, but operators must know fees exceeded policy.
• Clock skew between sampledAt and consumer: TTL is evaluated on the consumer's clock with a small grace; stale-by-skew still falls back to default rather than trusting an old snapshot.
• Redis unavailable: in-memory snapshot still serves this instance; cross-instance consistency is best-effort and its absence is logged, not fatal.

Security & Privacy Considerations

• The oracle reads only public network data; no user data, no auth surface beyond the read-only network/conditions endpoint (rate-limited, unauthenticated-safe or session-gated per platform convention).
• Fee policy constants and the absolute cap are config, changeable only via deploy — not via any API.
• No new signing or key handling; the dispatcher's existing signer lock is untouched.
• The endpoint must not expose internal queue depth or per-user op counts — only aggregate network conditions and generic ETA bands.

Out of Scope

• Predictive/ML congestion forecasting (the oracle is a sampled percentile with hysteresis).
• Fee markets for non-Stellar chains.
• User-selectable "fee speed" tiers beyond the existing priority mapping.
• Rewriting the outbox retry state machine (this issue only changes how the fee is chosen within it).

Suggested Implementation Plan

1. src/stellar/feeOracle.ts — poll getFeeStats/getLatestLedger, percentile + hysteresis logic, staleness handling; unit tests with mocked RPC.
2. Config keys (FEE_ORACLE_*, OUTBOX_MAX_ABS_FEE) in src/config/env.ts.
3. Dispatcher: proactive base-fee selection per priority, congestion deferral for LOW, absolute cap; integration tests for each priority path.
4. reconcileStuckSubmitted uses current oracle base.
5. GET /network/conditions + deposit/withdraw response fields + docs.
6. Metrics, Grafana panel, alert rules.

Acceptance Criteria

• src/stellar/feeOracle.ts publishes a bounded, hysteresis-smoothed FeeSnapshot with a congestionLevel and a hard TTL; stale snapshots are never silently reused
• The dispatcher sets the base fee proactively per priority (CRITICAL uses the aggressive fee during congestion; LOW defers, bounded)
• Retry fee-bump is capped by an absolute max fee; hitting it on a CRITICAL op alerts but still submits
• GET /api/v1/network/conditions returns current conditions + per-priority ETA bands; deposit/withdraw responses include estFee/estConfirmationSeconds
• Metrics for recommended fee, capacity usage, congestion level, oracle staleness, deferrals, and cap hits; Grafana panel + alert rules
• Deterministic fallback to a configured default on oracle cold-start or RPC failure
• docs/OUTBOX.md + docs/API_REFERENCE.md updated; unit + integration tests green

[Niffy03]

@Niffy03 has applied to work on this issue as part of the Stellar Wave Program's 8th wave.

I'd like to work on this.

ℹ️ Repo Maintainers: To accept this application, review their application or assign @Niffy03 to this issue.

[khanavi272-spec]

@khanavi272-spec has applied to work on this issue as part of the Stellar Wave Program's 8th wave.

Hi i have reviewed the issue and i would like to work on this issue

ℹ️ Repo Maintainers: To accept this application, review their application or assign @khanavi272-spec to this issue.

[Mmesolove]

@Mmesolove has applied to work on this issue as part of the Stellar Wave Program's 8th wave.

Hi Maintainer I have read the description and I can resolve this issue please kindly assign to me

ℹ️ Repo Maintainers: To accept this application, review their application or assign @Mmesolove to this issue.

[0xElyte]

@0xElyte has applied to work on this issue as part of the Stellar Wave Program's 8th wave.

I'd love to resolve this issue fast

ℹ️ Repo Maintainers: To accept this application, review their application or assign @0xElyte to this issue.

[JSE19]

@JSE19 has applied to work on this issue as part of the Stellar Wave Program's 8th wave.

Hi Maintainer, I would love to resolve this issue

ℹ️ Repo Maintainers: To accept this application, review their application or assign @JSE19 to this issue.

[drips-wave]

Congratulations, @JSE19! 🎉 Your application was accepted by the repo's maintainers, and the issue is due on August 31, 2026.

🧑‍💻 @JSE19: Please resolve the issue such that the repo's maintainers have enough time to review your contribution before the due date. You'll earn Points for completing the issue on-time, which will make you eligible for a share of the Stellar Wave Program's reward pool.

Warning

When opening a PR, please link it to this issue to ensure it gets tracked accurately. Points are awarded when this issue is marked as completed by the maintainer.

🤠 Repo maintainers: Please keep an eye on the contributor's progress and review their work before the due date. You can manage this issue, including adjusting its complexity and points, here.

🌊 Happy Wave 🌊

[drips-wave]

This issue has been completed by @JSE19 as part of the Stellar Wave Program's 8th Wave 🥳

😎 @JSE19: You earned 200 Points for completing this issue! After the current Wave ends, you'll be eligible for a percentage of the Wave's reward pool based on the percentage of total points you've earned. Learn more here. You can also Leave a review to share your experience working on this issue.

🧑‍💻 Repo maintainers: How'd the contributor do? Leave a review to share your experience working with them.