Skip to content

v0.7.0

Choose a tag to compare

View all tags
@Goodie-Goody Goodie-Goody released this 22 Jun 20:44
· 17 commits to main since this release
v0.7.0
440e4fa

⚠️ Breaking change

Output directory layout changed: every table now nests one level deeper under {exchange}/, and every Parquet table (trades, depth, snapshots) gained a new exchange column.

  • Before: {output_dir}/trades/{asset}/...
  • Now: {output_dir}/trades/{exchange}/{asset}/...

Existing 0.6.x data won't merge automatically with new data — move it under an {exchange}/ subfolder manually if you want one unified tree.

New features

Multi-exchange streaming — run multiple exchanges concurrently in one process/event loop:

LOBStreamer(exchanges=[
    {"exchange": "binance", "assets": ["BTCUSDT"]},
    {"exchange": "kraken", "assets": ["BTC/USD"]},
]).run()

or --exchanges "binance:BTCUSDT;kraken:BTC/USD" on the CLI.

Gap detection (detect_gaps=True, on by default) — flags sequence-id discontinuities for exchanges with real sequence numbers (binance, binance_futures, bybit, bybit_linear, okx, okx_swap), with automatic re-snapshot on a detected gap where a REST endpoint exists. New gaps/ table.

Checksum verification (verify_checksums=True, off by default) — for Kraken, maintains a live order-book mirror and verifies it against Kraken's CRC32 checksum. New checksums/ table.

Futures/perps support — 3 new exchange adapters:

  • binance_futures — Binance USDⓢ-M perpetuals (depth, trades, funding rate)
  • okx_swap — OKX USDT-margined perpetual swaps (depth, trades, funding rate)
  • bybit_linear — Bybit USDT perpetuals (depth, trades, funding rate + mark price)

All three write to a new funding/ table.

Bug fixes (found via live testing, not review)

  • Kraken checksum: local order-book mirror wasn't truncated to the subscribed depth, causing a 100% mismatch rate against live data.
  • Binance Futures: the funding-rate stream was silently dropped — Binance now routes /market channels (like markPrice) separately from /public ones, and the unrouted connection only got the latter. Fixed via a second, /market-routed connection.
  • Bybit (and bybit_linear): gap detection used the wrong counter for continuity (seq, a cross-venue sequence on a different numeric scale, instead of u, the real per-symbol update id), causing constant false-positive gaps. Predates this release — affected plain bybit too, just hadn't been exercised by gap detection until now.

Known limitations

  • okx_swap funding records have mark_price=None (OKX's mark price needs a second, unverified channel — documented rather than guessed)
  • Auto-resync-on-gap only works for REST-snapshot exchanges; OKX/Bybit-family exchanges log the gap but need a reconnect for a fresh snapshot
  • Kraken Futures and Coinbase's institutional perpetuals (Coinbase International Exchange) aren't covered — both are architecturally separate platforms from the spot APIs already integrated, not a simple addition
  • Dated/quarterly futures contracts aren't supported, only perpetuals.
Assets 2
Loading