Skip to content

feat(cli): wire scanner → router → builder → simulator pipeline#42

Merged
obchain merged 4 commits intomainfrom
feat/17-cli-e2e-pipeline
Apr 24, 2026
Merged

feat(cli): wire scanner → router → builder → simulator pipeline#42
obchain merged 4 commits intomainfrom
feat/17-cli-e2e-pipeline

Conversation

@obchain
Copy link
Copy Markdown
Owner

@obchain obchain commented Apr 21, 2026

Closes #16

charon listen now runs the full pipeline every block:

block event
  → adapter.fetch_positions(borrowers)
  → scanner.upsert + bucket counts
  → for each liquidatable position:
      → adapter.get_liquidation_params (vTokens + repay)
      → router.route (Aave V3 quote)
      → calculate_profit (placeholder USD math, see below)
      → tx_builder.encode_calldata     } only if BOT_SIGNER_KEY set
      → simulator.simulate via eth_call }
      → push to OpportunityQueue
  → log pipeline tick with bucket counts + queue depth

End-to-end is read-only: even with a signer present, no transaction is broadcast. Broadcast wiring lands with the MEV / private-RPC submission task.

Wiring details:

  • Single shared Arc<RootProvider<PubSubFrontend>> for adapter, price cache, Aave flash-loan adapter, and tx builder — cuts WS connection count from 4 → 1
  • Tx builder + simulator gracefully degrade: if BOT_SIGNER_KEY is unset, the pipeline still runs and queues opportunities by profit ranking only
  • Simulator::simulate and TxBuilder::build_tx made generic over Provider<T> + Transport to accept the pub-sub provider

Known placeholders (flagged inline):

  • repay_to_usd_cents_placeholder — assumes 1 token = 1 USD with 18 decimals (correct for stablecoin debt, underprices BNB/BTC/ETH). Real per-token decimals + symbol resolution lands when a token registry is added
  • PLACEHOLDER_GAS_USD_CENTS = 50 — fixed estimate until gas oracle wired
  • swap_route.min_amount_out = quote.amount + quote.fee — no DEX optimizer yet; on-chain backstop in CharonLiquidator.sol still catches under-fills

Live soak on BSC: 21 pipeline ticks in 25 s with zero borrowers tracked, zero panics. Only WARN observed: USDC Chainlink feed stale — real on-chain state, cache rejecting it as designed.

Depends on feat/16-executor-builder-and-sim.

#15)

`charon listen` now runs the full pipeline every block:

  block event
    → adapter.fetch_positions(borrowers)
    → scanner.upsert + bucket counts
    → for each liquidatable position:
        → adapter.get_liquidation_params (vTokens + repay)
        → router.route (Aave V3 quote)
        → calculate_profit (placeholder USD math, see below)
        → tx_builder.encode_calldata    } only if BOT_SIGNER_KEY set
        → simulator.simulate via eth_call }
        → push to OpportunityQueue
    → log pipeline tick with bucket counts + queue depth

End-to-end is **read-only**: even with the signer present, no
transaction is broadcast. Broadcast wiring lands with the MEV /
private-RPC submission task (#18).

Wiring details:
- Single shared `Arc<RootProvider<PubSubFrontend>>` for adapter,
  price cache, Aave flash-loan adapter, and tx builder. Cuts WS
  connection count from 4 → 1
- Tx builder + simulator gracefully degrade: if `BOT_SIGNER_KEY` is
  unset, the pipeline still runs and queues opportunities by profit
  ranking only (encoding/sim skipped). Lets dry-runs surface ranked
  candidates without ever needing a private key
- `Simulator::simulate` and `TxBuilder::build_tx` made generic over
  `Provider<T>` + `Transport` so they accept the pub-sub provider
  used elsewhere in the workspace

Known placeholders flagged in code:
- `repay_to_usd_cents_placeholder` — assumes 1 token = 1 USD with 18
  decimals. Correct for stablecoin debt (USDT, USDC, BUSD), badly
  underprices BNB / BTC / ETH. Real per-token decimals + symbol
  resolution lands when a token registry is added to config
- `PLACEHOLDER_GAS_USD_CENTS = 50` — fixed $0.50 estimate. Replaced
  by `eth_estimateGas × gas_price × native_price` once a gas oracle
  is wired
- `swap_route.min_amount_out = quote.amount + quote.fee` — no DEX
  optimizer yet; the on-chain backstop in `CharonLiquidator.sol`
  still catches under-fills

Live soak on BSC: 21 pipeline ticks in 25 s with zero borrowers
tracked, zero panics. Only WARN observed: USDC Chainlink feed stale
(real on-chain state, cache rejecting it as designed).
This was referenced Apr 23, 2026
obchain added 2 commits April 23, 2026 16:05
Move the hot-wallet private key off raw std::env::var and onto the
typed config path. `BotConfig.signer_key` is an
`Option<secrecy::SecretString>` so the secret is redacted from `Debug`
and zeroised on drop. Supplied via `${CHARON_SIGNER_KEY}` in
`config/default.toml`.

Also extend the env-substitution loader with shell-style
`${VAR:-default}` so operator-optional secrets can remain absent
without failing config parse. Scan-only dev runs continue to work out
of the box.

Closes #172.
Rework the per-block pipeline around explicit gates and supervision:

- Simulation gate is now mandatory before enqueue. If no signer is
  configured, `process_opportunity` returns early and nothing lands on
  the queue — enforces the CLAUDE.md safety invariant that every
  queued opportunity has passed `eth_call`. Closes #170.
- `swap_route.min_amount_out` includes a static gas floor plus a
  minimum-profit floor on top of repay + flash fee, so the on-chain
  revert guard rejects zero- or negative-net fills even before the
  gas oracle lands. Closes #171.
- Block listeners are supervised via `tokio::task::JoinSet` and polled
  alongside the event drain; a panic or unexpected exit now triggers
  a controlled shutdown instead of silently orphaning the task.
  Closes #173.
- Each per-block pipeline pass runs under a 2.5s wall-clock deadline;
  a stalled RPC can no longer delay subsequent blocks. Closes #174.
- Within a single tick we count consecutive opportunity-level RPC
  failures and bail at three strikes, letting Docker restart us with
  a fresh provider until the shared provider grows its own reconnect
  loop. Closes #175.
- Startup asserts every Venus market's underlying is on a stablecoin
  allow-list. Refuses to run against BNB/BTC/ETH debt until the
  per-token USD converter (tracking #148) replaces the 1:1 placeholder.
  Closes #178.
- Unit tests cover the four `process_opportunity` branches: happy
  path, sim failure, no signer, below threshold — plus a regression
  guard on the `min_amount_out` floor. Closes #177.

TODO markers in-line flag the profit-calc and flash-loan-params
rebase dependencies from #168/#169; those fixes propagate from the
ancestor branches on rebase.

Refs #168, #169.
obchain added a commit that referenced this pull request Apr 23, 2026
BNB_TESTNET_WS_URL / BNB_TESTNET_HTTP_URL violate the repo-wide
CHARON_* namespace convention enforced on every new env var since
PR #42 and PR #44. The existing mainnet BNB_* vars predate the
convention and stay unrenamed pending a separate repo-wide pass —
only the testnet pair, landed this branch, shifts to the compliant
prefix.

Renames `config/testnet.toml` substitution references and updates
`.env.example` with a short note pointing at #245 so future readers
see the scope. The `config_profiles` integration test exports the
new names.

Closes #245
@obchain obchain changed the base branch from feat/16-executor-builder-and-sim to main April 24, 2026 11:28
# Conflicts:
#	Cargo.lock
#	config/default.toml
#	crates/charon-cli/Cargo.toml
#	crates/charon-cli/src/main.rs
#	crates/charon-core/src/config.rs
#	crates/charon-executor/src/builder.rs
#	crates/charon-executor/src/simulation.rs
@obchain obchain merged commit ee36384 into main Apr 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[executor] End-to-end pipeline wiring: scanner → profit calc → router → tx builder

1 participant