Skip to content

v0.2.0 — Relayer Data API

Choose a tag to compare

View all tags
@shanu516516 shanu516516 released this 20 Feb 01:03
· 55 commits to main since this release
v0.2.0
e99c173

Relayer Data API — v0.2.0

The Relayer Data API is a high-throughput, read-only data service for the Twilight BTC derivatives protocol. It consists of two independent binaries — archiver and api — that together ingest, persist, and serve all relayer state to external clients.

Architecture Overview

relayer-core
     │
     ▼  (Kafka topics)
 ┌─────────┐     ┌───────────┐     ┌──────────────────────┐
 │ archiver│────▶│ PostgreSQL│────▶│  api (JSON-RPC)      │
 │  binary │     │  (Diesel) │     │  public  :8987       │
 │         │────▶│   Redis   │────▶│  private :8989       │
 └─────────┘     └───────────┘     └──────────────────────┘

archiver

Consumes events from Kafka, applies batch inserts into PostgreSQL, and maintains a live Redis cache for low-latency reads. Key behaviours:

  • Batch inserts — 2000-item capacity per entity type; flushes automatically every 100 ms or when the batch is full.
  • Redis cache — maintains the live order book (bid/ask sorted sets, recent_orders), the latest risk state (risk_state key), and the current pool share price.
  • Event types handled — trader orders, lend orders, lend pool snapshots, price ticks, funding events, candle updates, risk engine commands, risk parameter updates, account link events, and transaction hashes.

api

A stateless JSON-RPC 2.0 server backed by a PostgreSQL connection pool (r2d2). Exposes two ports:

  • :8987 — public endpoint (no authentication required)
  • :8989 — private endpoint (JWT / API-key authenticated, rate-limited per header)

Public RPC Methods (:8987)

Method Description
lend_pool_info Current lending pool state (TLV, pool shares, sequence)
pool_share_value Latest share price (total_locked_value / total_pool_share)
btc_usd_price Latest BTC/USD price tick
historical_price BTC/USD price series over a time range
apy_chart APY time-series with configurable range, step, and lookback
last_day_apy Single trailing 24-hour APY value
open_limit_orders Current open limit order book
recent_trade_orders Trades closed in the last 24 hours
trader_order_info Single trader order by ID
historical_trader_order_info Paginated trader order history
lend_order_info Single lend order by ID
historical_lend_order_info Paginated lend order history
candle_data OHLCV candle data (1m, 5m, 15m, 30m, 1h, 4h, 8h, 12h, 24h)
open_interest Total long and short BTC exposure
position_size Long, short, and net position sizes
get_funding_rate Current funding rate
historical_funding_rate Funding rate history
get_funding_payment Funding payments for a given address
get_fee_rate Current fee rate
historical_fee_rate Fee rate history
trade_volume Aggregate trade volume over a period
order_history Combined order history for an address
open_orders Currently open orders for an address
unrealized_pnl Unrealized PnL for open positions
last_order_detail Most recent order detail for an address
get_market_stats Market-wide risk statistics derived from the latest RiskState
server_time Server UTC timestamp
account_summary_by_twilight_address Per-address account summary with date-range filtering

Private RPC Methods (:8989)

Method Description
submit_trade_order Submit a signed trade order
submit_lend_order Submit a signed lend order
submit_bulk_order Submit multiple orders in one call
cancel_trader_order Cancel an open trader order
settle_trade_order Settle a trade order on-chain
settle_lend_order Settle a lend order on-chain
transaction_hashes Retrieve transaction hashes for a set of orders

Database Schema

PostgreSQL with Diesel ORM and automated migrations. Key tables:

Table Purpose
trader_order All trader order records
lend_order All lend order records
lend_pool Lending pool snapshots
lend_pool_price_minute Per-minute share price history (source for APY)
btc_usd_price BTC/USD price ticks
candle OHLCV candle data across all resolutions
funding_rate Funding rate history
fee_rate Fee rate history
risk_engine_update Risk engine command audit log with full RiskState snapshot
risk_params_update Risk parameter change history
open_interest Aggregate long/short exposure snapshots
twilight_qq_account_link Twilight address ↔ QQ account mapping
position_size_log Historical position size snapshots

Migrations are embedded into the binary via diesel_migrations and run automatically at startup.

APY Calculation

The apy_series() SQL function computes annualised yield at each time bucket:

APY = (price_at_bucket / price_(lookback)_before_bucket) ^ (365_days_in_seconds / lookback_in_seconds) − 1

The exponent is generalised so any lookback window (24h, 7d, 30d) produces a correct annualised figure. A companion last_day_apy_now() function returns the instantaneous trailing-24h APY.

Risk Engine Integration

The archiver processes RiskEngineCommand events emitted by relayer-core and:

  1. Persists each command and the resulting RiskState snapshot to risk_engine_update.
  2. Caches the latest RiskState in Redis under the key risk_state for O(1) API reads.
  3. Exposes get_market_stats which computes live risk utilisation ratios from the cached state and current pool equity.

Supported commands: AddExposure, RemoveExposure, SetManualHalt, SetManualCloseOnly, SetPauseFunding, SetPausePriceFeed.

Configuration

All configuration is via environment variables (see .env.example):

Variable Description
DATABASE_URL PostgreSQL connection string
REDIS_HOSTNAME Redis connection string
BROKER Kafka broker address(es), comma-separated
CORE_EVENT_LOG Primary Kafka topic for relayer-core events
RUST_LOG Log level (info, debug, warn, error)
RISK_MAX_OI_MULT Max open interest as a multiple of pool equity
RISK_MAX_NET_MULT Max net exposure as a multiple of pool equity
RISK_MAX_POSITION_PCT Max single position as a fraction of pool equity
RISK_MIN_POSITION_BTC Minimum position size in sats
RISK_MAX_LEVERAGE Maximum allowed leverage

Running

# Archiver (event consumer + database writer)
cargo run --release --bin archiver

# API server
cargo run --release --bin api

What's New in v0.2.0

Features

  • APY chart endpoint (apy_chart) with configurable range, step, and lookback intervals
  • Trailing APY (last_day_apy) — single-value 24-hour yield
  • Open interest endpoint (open_interest) — live long/short BTC exposure
  • Market statistics (get_market_stats) — risk utilisation ratios from live RiskState
  • Extended trader order info (trader_order_info_v1) — includes latest close limit and funding applied
  • Account summary (account_summary_by_twilight_address) — per-address history with date-range filter
  • Risk engine archiving — full audit log of all RiskEngineCommand events with state snapshots
  • Risk parameter tracking — risk_params_update table records every parameter change
  • Candle data pipeline — improved OHLCV aggregation across all resolutions
  • Twilight QQ account linking — address mapping persisted and queryable

Bug Fixes

  • Fixed missing third bind parameter in apy_series() call (runtime SQL error)
  • Fixed hardcoded annualisation exponent — non-24h lookbacks now produce correct APY
  • Fixed "1m" token collision in interval parser (was resolving to 30 days instead of 1 minute)
  • Fixed nullable mismatch in last_day_apy_now() Diesel mapping — NULL is now correctly returned as None instead of a deserialisation error
  • Eliminated Box::leak memory allocations in RPC argument resolution

Infrastructure

  • relayer-core dependency updated to risk-param-fix-halt branch
  • Batch insert architecture with configurable capacity and flush timeout
  • Kafka broker configuration supports multiple hosts
  • Dotenv loading made non-fatal — service starts cleanly without a .env file
  • Header metadata extraction centralised in src/rpc/headers.rs
Assets 5
Loading