Source Bybit

Source: Bybit

A data source adapter (see Data sources) for Bybit linear perpetuals.

Why the public dumps

CCXT cannot provide historical L2 or trades. It only wraps Bybit's live APIs, which serve the current book and roughly the last 1000 trades. History comes from Bybit's public dumps.

Files

Data	Source	Notes
Trades	public.bybit.com/trading/{SYM}/{SYM}{YYYY-MM-DD}.csv.gz	Daily, available from 2020. Older files are sorted newest-first.
L2 order book	quote-saver.bycsi.com/orderbook/linear/{SYM}/{date}_{SYM}_ob{depth}.data.zip	Daily. ob500 through 2025-08-20, ob200 from 2025-08-21. Same format for both.
Instrument spec	nautilus BybitHttpClient.request_instruments (REST instruments-info) + REST risk-limit	Today's spec only; delisted symbols aren't served.
Funding rates	nautilus BybitHttpClient.request_funding_rates (REST funding history)	Returns FundingRateUpdates with ts_event = settlement time; the first record of a request can lack interval.
1-minute candles	nautilus BybitHttpClient.request_bars (REST kline)	Last-price klines as Bars of type {SYM}-LINEAR.BYBIT-1-MINUTE-LAST-EXTERNAL, stamped at their close. One UTC day per request, ending 1 ms before midnight (the end is inclusive); a whole month fails with error 10016. Nautilus pages through the day itself.
Mark price	REST mark-price klines, fetched by sbt2	Bybit charges funding on mark price. Nautilus's client only fetches last-price klines. 1-minute klines, each close one MarkPriceUpdate; a day is two requests of 720 (Bybit returns at most 1000).

Mark price is not built from trades: Bybit computes it from the spot index plus a smoothed basis. The klines are only how its history is served.

Raw files go under raw/bybit/linear/{SYM}/ in the root, one per UTC day, dumps with Bybit's own names:

Data	Raw file	Content
Trades	trading/{SYM}{YYYY-MM-DD}.csv.gz	the dump, as downloaded
Instrument spec	instrument/{SYM}{YYYY-MM-DD}.json	nautilus to_dict, named with the day it was taken on; info holds Bybit's fundingInterval (minutes)
Funding rates	funding/{SYM}{YYYY-MM-DD}.json	nautilus to_dict records; the day's first record lacks interval
Mark price	mark_price/{SYM}{YYYY-MM-DD}.json	the two kline responses, as returned
1-minute candles	candles/{SYM}{YYYY-MM-DD}.json	nautilus to_dict bars, stamped at their close

Each data type is one channel in the adapter's registry tuple: its name in known_gaps.toml (trades, funding, mark_price, candles), its raw path, its origin (a dump URL or a REST fetch) and its parser. A new Bybit data type is one new channel.

Instrument ids are {SYM}-LINEAR.BYBIT. REST calls sbt2 makes itself use httpx; proxies come from the environment (HTTPS_PROXY, ALL_PROXY), and nautilus's client gets the same https proxy. Nautilus's own retries are off, since download schedules retries. A symbol Bybit doesn't list (error 10001, symbol invalid) is reported as missing.

Days Bybit doesn't serve (download reports them as missing) can be added to the source's known-gaps list; see Data sources.

Parsing quirks

• Instruments come from nautilus's Bybit HTTP client, which returns a ready CryptoPerpetual and needs no API keys (checked on 2.0.0rc5 and 2.0.0rc6.dev20260927). Its margin is a placeholder (10% initial and maintenance), so the source overwrites it before saving the snapshot with the lowest tier of the public risk-limit endpoint. In nautilus v2 instruments carry no fees: fees are set by the venue profile's fee model (see Spec), which holds Bybit's base rates.
• Nautilus's CryptoPerpetual has no funding interval, so the snapshot also fetches instruments-info and keeps its fundingInterval (minutes) in the instrument's info.
• A parsed instrument gets ts_init = the start (00:00 UTC) of the day its snapshot was taken on.
• A funding record without interval is completed from the snapshot's info["fundingInterval"] by the parser; nautilus only settles funding on records that carry one.
• The funding parser checks that every ts_event is an exact multiple of its interval since the Unix epoch and raises FundingOffGridError otherwise: nautilus silently skips a record off that grid (see Crypto perpetuals).
• Trade timestamps are decimal epoch seconds with 3 or 4 fractional digits (1585180818.434); pyarrow reads them as exact decimals, so nanoseconds are exact. A day's dump is read whole into Arrow columns (up to 8.8M rows for BTC in January 2025) and sorted stably by time, since older dumps run newest-first.
• Aggressor side is the dump's side (Buy/Sell), trade id its trdMatchID, and ts_init = ts_event.
• A mark-price kline is [start, open, high, low, close], newest first. Each close becomes one MarkPriceUpdate stamped 1 ns before the end of its minute (start + 60 s − 1 ns): the close is known at the end of the minute, so a backtest never sees it a minute early, and the day's last close stays inside the day's catalog file bounds [day start, day end − 1 ns] (see Data). Each catalog day then comes from its own raw file alone.
• A candle is rebuilt from its to_dict record and moved 1 ns before its close, for the same reasons as mark prices: the day's last candle, which closes at midnight, stays in its own day's file. Nautilus aggregates an hourly or daily bar from candles by their timestamps, so the candle closing at 01:00 lands in the bar closing at 01:00.
• Prices are built at the instrument's price_precision, not with make_price: make_price rounds to price_increment, and mark prices fall between ticks (BTCUSDT mark 93396.41 on a 0.10 tick).
• L2 is newline-delimited JSON: one snapshot, then delta messages. ts_event = cts (matching-engine time), ts_init = ts (push time).
• Each L2 file ends with the next day's opening snapshot, which is trimmed off.
• Nautilus's own BybitOrderBookDeltaDataLoader is not used. It loads a whole day into pandas (tens of GB for BTC), uses ts instead of cts, and never sets F_LAST.

Decisions

Decision	Why
Public dumps for L2 and trades	CCXT only reaches live APIs.
Custom streaming L2 parser	See above.
Nautilus's Bybit HTTP client for instrument specs and funding history	No hand-written mapping from Bybit's JSON to CryptoPerpetual or FundingRateUpdate; only margin is overwritten, and a missing funding interval is filled from the spec.
Fees on the venue profile, not the instrument	v2 instruments have no fee fields; the fee model belongs to the venue.
1-minute mark-price klines, one raw file per UTC day	The finest interval Bybit serves; per-day files match trades, so download and ingest work per (instrument, data type, day) for every data type.
1-minute candles through nautilus's client, one UTC day per request	The finest interval Bybit serves, and every coarser bar is built from it. Per-day files like every other type; nautilus maps klines to Bars.
Candles 1 ns before their close	As for mark prices: each day's file comes from its own raw file alone.
httpx for sbt2's own REST calls	Nautilus v2 exposes no Python HTTP client; httpx honours proxies from the environment.
Today's instrument spec, fail loudly on mismatch	History isn't served. A loud failure beats v1's guessed specs.
Funding uses mark price	That's what Bybit charges on.
Funding interval kept in the instrument snapshot's info	Nautilus's instrument has no field for it; a missing funding interval needs a source, and the snapshot is the dated spec.
Mark price 1 ns before the kline's close time	Stamping the close at the kline's open would leak a minute of future into a backtest; stamping it exactly at the close puts a day's last mark price in the next day's file.
Trades sorted in memory, not reversed	Robust to any order in the dump; a day fits in memory.

Deferred and open

Asset-class items (funding, liquidation) are on Crypto perpetuals.

• RPI trades: Retail Price Improvement trades are marked in the dump. Currently absent on perps, but present on spot.
• Depth-10 snapshots: sampled top-of-book data for cheaper strategies. Add when a strategy needs it.