TickForge is a compact market-microstructure laboratory built around Nasdaq TotalView-ITCH 5.0. It decodes a verified book-relevant protocol subset, reconstructs deterministic Level-3 books, derives exact L1 features, and runs a latency- and cost-aware aggressive-IOC experiment. The systems path is C++23; an optional pybind11 module exposes the same batch workflows to Python.
The interesting part is the contract between the pieces: wire values stay integer-exact, arrival order is explicit, rejected mutations are transactional, and a replay ends in a stable logical-state hash. That makes results useful for regression testing and research without pretending a synthetic fixture is a live market.
flowchart LR
A["Length-prefixed ITCH bytes"] --> B["Bounds-checked decoder"]
B --> C["Per-locate L3 books"]
C --> D["Exact L1 features"]
D --> E["Latency-aware IOC model"]
C --> F["Canonical state hash"]
B -. "bounded handoff" .-> G["SPSC queue"]
H["Deterministic fixture generator"] --> A
I["Python batch API"] --> B
TickForge needs CMake 3.25 or newer, a C++23 compiler, and a POSIX platform (macOS or Linux). No data or third-party library is needed for the core build.
cmake --preset release -DCMAKE_COMPILE_WARNING_AS_ERROR=ON
cmake --build --preset release-build --parallel
./build/release/tickforge generate /tmp/tickforge.itch
./build/release/tickforge replay /tmp/tickforge.itchThe default seed produces a small, closed book and the following reproducible summary:
generated messages=11 bytes=329
messages=11 applied=8 ignored=3 books=1 open_orders=0 open_quantity=0 state_hash=6550206219837066635
The file is generated test data, not captured Nasdaq data. Increase the corpus
with --cycles, --symbols, and --seed; the exact generator contract is in
docs/synthetic-data.md.
| Area | Design choice |
|---|---|
| Protocol | Explicit big-endian decoding of S A F E C X D U P; exact type lengths and typed offsets on failure |
| Book | Reference-addressable Level-3 state, price priority, FIFO orders, transactional rejection, deterministic depth and hashing |
| Replay | One book per stock_locate, arrival ordinals as the tie-breaker, strict fail-fast processing, optional update capture |
| Research | Exact spread, midpoint, imbalance, microprice, and forward midpoint labels; no internal floating-point conversion |
| Simulation | Deterministic aggressive IOCs with data/order latency, price protection, shared visible liquidity, position limits, fees, equity, and drawdown |
| Concurrency | Bounded SPSC queue with one construction-time allocation, explicit object lifetimes, and acquire/release publication |
| Reliability | Seeded feed faults, parser and replay fuzz harnesses, ASan/UBSan and TSan presets, Linux/macOS CI |
| Measurement | Release replay harness that records raw samples, corpus identity, compiler, platform, and final state hash as JSON |
| Python | Optional generate, replay, features, and simulate batch calls through pybind11 |
Prices use raw Price4 units ($0.0001). Monetary results use signed Money8
units ($0.00000001). Midpoint and microprice cross the Python boundary as
numerator/denominator pairs, and imbalance is integer parts per million.
Given the same bytes and configuration, replay order, feature rows, simulator results, synthetic fixtures, and logical hashes are deterministic across the supported targets. Hashes serialize logical fields in a fixed order; they do not include addresses, padding, or hash-table iteration order. They are regression diagnostics, not cryptographic digests or persistence formats.
ITCH timestamps do not establish a total order, so TickForge assigns each successfully handled message a zero-based arrival ordinal. The sequence monitor is intentionally narrower: ITCH's tracking number can expose a local repeat or discontinuity, but it is not a transport sequence and cannot prove packet loss.
cmake --preset dev -DCMAKE_COMPILE_WARNING_AS_ERROR=ON
cmake --build --preset dev-build --parallel
ctest --preset dev-test --output-on-failure
cmake --preset asan-ubsan -DCMAKE_COMPILE_WARNING_AS_ERROR=ON
cmake --build --preset asan-ubsan-build --parallel
ctest --preset asan-ubsan-test --output-on-failure
cmake --preset tsan -DCMAKE_COMPILE_WARNING_AS_ERROR=ON
cmake --build --preset tsan-build --parallel
ctest --preset tsan-test --output-on-failureThe standalone sanitizer fuzz smoke and optional bindings are documented in docs/verification.md. Benchmark construction and interpretation are deliberately separated from performance claims in docs/benchmarking.md.
python3 -m venv .venv
. .venv/bin/activate
python -m pip install --require-hashes -r python/requirements-test.txt
cmake --preset python -Dpybind11_DIR="$(python -m pybind11 --cmakedir)"
cmake --build --preset python-build --parallel
PYTHONPATH=build/python/python python -m pytest -q python/testsSee python/README.md for the exact-value return contract.
- Architecture and complexity
- Protocol scope
- Order-book invariants
- Replay contract
- Research features and labels
- Simulation model
- SPSC memory ordering
- Fault injection and sequence monitoring
- Verification record
TickForge is a historical research core, not an exchange gateway. It does not implement SoupBinTCP or MoldUDP64, recovery, production risk controls, passive queue reconstruction, hidden liquidity, market impact, venue routing, or live trading. The book uses clear standard-library containers rather than claiming an allocation-free hot path; the simulator uses visible post-event L1 state and should be read as a reproducible convention, not a fill prediction.
Contributions are welcome; start with CONTRIBUTING.md.
MIT. See LICENSE.