Skip to content

v0.8.0 — Phase 8: dashboard telemetry pipeline

Choose a tag to compare

@ChronoCoders ChronoCoders released this 16 May 23:47
· 37 commits to main since this release

Telemetry pipeline — every currently-empty dashboard panel now gets real data. Pure plumbing: no algorithmic changes to strategy, risk, or execution.

What is in

C++ writer side

  • src/db/pg_reporter.hpp — extended DbPositionSnapshot with 13 new fields (best_bid, best_ask, spread_bps, bid_qty, ask_qty, volatility, gamma, k, T, lat_p50_us, lat_p95_us, lat_p99_us, open_orders). Extended DbTrade with is_maker (postOnly orders are always maker). is_trivially_copyable static_assert preserved.
  • src/db/pg_reporter.cpp — appended idempotent ALTER TABLE ... ADD COLUMN IF NOT EXISTS block to kSchemaSql; extended both the batched-flush and per-event INSERTs. Schema migration verified against the live 3925-row position_snapshots table without disruption.
  • src/market_data/tick_processor.hpp — added current_volatility() accessor so the 1 Hz snap_thread can pull the latest stdev.
  • src/market_data/orderbook.hpp — documented single-writer / multi-reader contract now that snap_thread reads the book concurrently with tick_processor's consumer thread. Same Phase 3 PositionTracker pattern: telemetry-tolerant, no seqlock.
  • src/risk/risk_manager.hpp — exposed open_order_count() accessor (atomic load).
  • src/execution/order_manager.{hpp,cpp} — RDTSC at SUBMITTED, delta on ACKNOWLEDGED, rolling 1000-sample circular buffer, latency_percentiles(p50, p95, p99) via snapshot-then-sort under a per-instance mutex. Lock ordering mu_ → latency_mu_ is one-way; snap_thread only takes latency_mu_, no deadlock. is_maker = true on every fill (inject_fill). submit_cycles lives on OrderSlot so a future async-ACK path (Phase 9 FIX, user-data stream) can compute latency across method boundaries.
  • src/main.cpp snap_thread — populates all 13 new DbPositionSnapshot fields per second: order-book values from processor.book(), volatility from processor.current_volatility(), A-S params from cfg.strategy.{gamma,k,horizon}, latency from order_manager.latency_percentiles(...) (single call, three out-params), open-order count from risk_mgr.open_order_count().

Go backend side

  • dashboard_backend/server.go — extended Snapshot struct with 13 telemetry fields + 4 derived (fill_count_10s, fill_count_60s, maker_ratio, fills_per_10s) + max_inventory_display. latestSnapshot() SQL pulls all 20 columns via sql.NullFloat64/NullInt64 so pre-Phase-8 rows scan as 0 silently. New fillStats() helper does a single COUNT(*) FILTER (WHERE ...) query (Postgres-specific) over the trades table, scoped to the last 60 s. COALESCE(is_maker, TRUE) covers any legacy rows where the column predates the schema migration.
  • /api/v5/status — extended with ws_connected (proxy: any position_snapshots row in the last 5 s), pg_connected (real db.Ping() — agent already wired during scaffolding), last_snapshot_age_ms. New lastSnapshotTs() cheap single-column query avoids the wasteful full latestSnapshot() invocation just for ts_ns.

Config + env

  • config/config.toml — new [dashboard] section (max_open_orders, max_inventory_display). Operator documentation; Go backend reads via SPREADARA_MAX_OPEN_ORDERS / SPREADARA_MAX_INVENTORY_DISPLAY env vars with sensible defaults.

Deviations from spec

  • T column renamed to T_param — T is reserved-feeling enough that some Postgres clients quote it inconsistently; the C++ struct field stays T, only the SQL column differs.
  • DOUBLE PRECISION instead of spec'd NUMERIC(18,8) — matches existing schema types (inventory, mid_price, etc.) and avoids pqxx ↔ numeric conversion friction. Telemetry values are lossy-OK.
  • /api/v5/status falls back to latestSnapshot() if the reader is not *sqlReader — preserves the test-stub interface contract without forcing the stub to implement a new method.

Tests

  • C++: 8/8 pass (test_orderbook, test_spsc, test_strategy, test_risk, test_execution, test_reporter, test_backtest, test_integration) under -O3 -march=native -Wall -Wextra -Werror, zero warnings.
  • Go: 6/6 pass (snapshot handler, 404, trades handler, WS version reject, WS origin reject, WS push interval).
  • Schema migration: verified idempotent — ALTER TABLE ... ADD COLUMN IF NOT EXISTS succeeded against the populated production-style DB. test_reporter::SchemaCreationIdempotent continues to pass (live PG required); zero-initialized DbTrade.is_maker = false from existing tests inserts cleanly into the new BOOLEAN DEFAULT TRUE column.
  • Sample /api/snapshot JSON after a fresh boot showed best_bid=78212.6, gamma=0.1, k=1.5, T_param=1.0, open_orders=0 populated alongside the existing fields.

Review-pass tightenings (all applied before tag)

  • submit_cycles moved from stack-local to OrderSlot::submit_cycles (future-proofs the async-ACK path).
  • OrderBook single-writer / multi-reader contract documented.
  • /api/v5/status lastSnapshotTs() replaces the wasteful latestSnapshot() call (single-column query, ~10× cheaper at scale).

Not in scope

  • Bid/ask still 0 until the trading binary writes real top-of-book values per tick. The snap_thread now pulls from OrderBook but only after the first depth message — pre-Phase-8 dashboard rows remain at 0.
  • Latency percentiles are 0 until the first ACK is captured.
  • Fill counts are 0 until real fills occur on the demo book.
  • Per-stream WS health (instead of the coarse 5 s freshness proxy) is a future enhancement.