A from-scratch limit order book matching engine in C++20.
Zero heap allocation on the hot path. ~20 nanoseconds per match.
I built this project as part of DIY '26, the project series organized by the Finance & Economics Club, IIT Guwahati. NanoMatch is the systems-track problem statement (Quant - Systems, PS #03): build an ultra-low latency order matching engine from scratch in C++, with custom memory management, lock-free concurrency, and rigorous benchmarking against an STL baseline.
The problem statement emphasizes a specific idea: in high-frequency trading, the most brilliant alpha model is worthless if the execution infrastructure is a microsecond too slow. The matching engine is the primary battleground. This project is my attempt to explore that battleground, replacing standard library abstractions with cache-aligned arenas, flat hash tables, and zero-copy I/O to see how far raw hardware sympathy can push throughput on commodity hardware.
Most matching engine implementations you'll find on GitHub use std::map for price levels and std::unordered_map for order lookup. They work. They're correct. But they also scatter memory across the heap, chase pointers through cache misses, and introduce latency spikes from allocator contention. These are the exact things that matter when you're trying to process market data at wire speed.
I wanted to explore a different set of trade-offs: what happens when you eliminate all dynamic memory allocation from the matching path, keep everything in contiguous cache-friendly memory, and push I/O off the critical thread entirely?
The result is a single-threaded matching engine that processes ~49 million events per second, roughly 20 nanoseconds per match, while a background thread handles trade logging through a lock-free queue. That's a 2.87x throughput improvement over an equivalent STL-based engine on the same workload.
Benchmarked on an x86-64 machine (AMD Ryzen, 16 CPUs @ 5.15 GHz, 32 MB L3 Cache), replaying 1,000,000 synthetic trading events (70% limit adds, 25% cancels, 5% market orders).
Measured with Google Benchmark, isolating the matching engine from I/O:
| Engine | Throughput | Latency / Event | Relative |
|---|---|---|---|
| NanoMatch | 48.94M events/sec | 20.4 ns | |
| STL Baseline | 17.07M events/sec | 58.6 ns | |
| Improvement | +186% | -65% | 2.87x faster |
Per-event latency distribution across 1,000,000 operations:
| Engine | Avg | p50 | p90 | p99 | p99.9 | p99.99 |
|---|---|---|---|---|---|---|
| NanoMatch | ~20 ns | ~18 ns | ~24 ns | ~48 ns | ~120 ns | ~400 ns |
| STL Baseline | ~58 ns | ~42 ns | ~80 ns | ~210 ns | ~900 ns | ~3,200 ns |
The tail latency gap widens at higher percentiles. At p99.9, NanoMatch is ~7.5x faster. This is where heap allocation jitter and cache misses in the STL baseline compound.
Full pipeline including mmap ingestion, CSV parsing, matching, and async trade logging to disk:
| Metric | Result |
|---|---|
| Total Events Processed | 1,000,000 |
| Trade Reports Written | 502,243 |
| Wall-Clock Time | 80.9 ms |
| Pipeline Throughput | 12.36M events/sec |
The throughput drop from 49M to 12M events/sec here is expected. It includes file I/O overhead from mmap page faults and the parser scanning raw CSV bytes. The matching engine itself is not the bottleneck in the end-to-end pipeline.
┌──────────────────────────────────┐
│ Startup (one-time cost) │
│ │
│ ┌────────────────────────────┐ │
│ │ MemoryPool<Order> │ │
│ │ 3,000,000 x 64B slots │ │
│ ├────────────────────────────┤ │
│ │ MemoryPool<Limit> │ │
│ │ 100,000 x 64B slots │ │
│ ├────────────────────────────┤ │
│ │ Order* direct-map array │ │
│ │ Limit flat hash table │ │
│ └────────────────────────────┘ │
└──────────────────────────────────┘
┌─────────────────┐ ┌──────────────────────┐ ┌──────────────────────┐
│ Input CSV │ mmap │ Zero-Copy Parser │ events │ Matching Engine │
│ │ ──────── │ │ ──────── │ │
│ synthetic_ │ zero │ • madvise(SEQ) │ A/C/M │ • Price-time FIFO │
│ orders.csv │ copy │ • Integer-only │ stream │ • O(1) order lookup │
│ │ │ price parsing │ │ • Flat hash limits │
└─────────────────┘ └──────────────────────┘ └──────────┬───────────┘
│
TradeReport
│
┌────────────────▼───────────────┐
│ SPSC Lock-Free Ring Buffer │
│ 131,072 slots │
│ release / acquire atomics │
└────────────────┬───────────────┘
│
background
thread
│
┌────────────────▼───────────────┐
│ Async Logger │
│ • Batched flush (per 1000) │
│ • executed_trades.csv │
└────────────────────────────────┘
This section explains why I made each design choice, not just what it does. Every optimization here targets a specific bottleneck I observed in conventional matching engine implementations.
The problem: new/delete on the hot path means system calls, mutex contention inside the allocator, and unpredictable latency spikes when the OS needs to find free pages.
What I did: I pre-allocate two contiguous memory arenas at startup, one for Order objects and one for Limit (price level) objects. A template-based MemoryPool<T> manages each arena as a free-stack: allocation pops a pointer in O(1), deallocation pushes it back. No malloc, no free, no system calls during matching.
// Hot path allocation: compiles to a stack pop + placement new
template <typename... Args>
T* allocate(Args&&... args) {
T* ptr = m_free_stack[--m_free_top]; // O(1)
new (ptr) T(std::forward<Args>(args)...);
return ptr;
}The problem: std::unordered_map adds hashing overhead, bucket chaining, and pointer indirection for every order lookup. Order lookups happen on every cancel and every match.
What I did: Since Order IDs are sequential integers (starting from 1), I use a raw pointer array (Order* m_order_map[]) indexed directly by Order ID. Lookup is a single array dereference. No hashing, no collision handling, no cache misses from chasing bucket pointers.
The problem: std::map gives O(log n) lookup for price levels via a red-black tree. std::unordered_map avoids the log factor but still scatters entries across heap-allocated buckets with pointer chaining.
What I did: I store price levels in a flat, power-of-two-sized array with open addressing and linear probing. The entire table lives in contiguous memory, so sequential probes hit the same or adjacent cache lines. Deletion uses backward-shift rehashing to maintain probe chain integrity without tombstones.
// Lookup: mask + linear scan in contiguous memory
Limit* findLimitNode(Price price) const noexcept {
size_t hash = price & m_limit_map_mask;
while (m_limit_map[hash].price != 0) {
if (m_limit_map[hash].price == price)
return m_limit_map[hash].limit;
hash = (hash + 1) & m_limit_map_mask;
}
return nullptr;
}The problem: When two frequently-accessed objects share a 64-byte cache line, writing to one invalidates the other's cache entry across CPU cores (false sharing). Even on a single core, misaligned objects mean multiple cache reads for a single struct access.
What I did: Both Order and Limit structs are alignas(64), guaranteeing each object occupies exactly one cache line. The SPSC ring buffer's head/tail atomics are also separated by alignas(64) to prevent false sharing between the producer (matching thread) and consumer (logger thread).
The problem: Floating-point arithmetic is not only slower than integer arithmetic on most architectures, it also introduces rounding errors that are unacceptable in financial systems.
What I did: Prices are stored as uint64_t values scaled by 10,000 (4 decimal places). The Parser converts decimal strings directly to this scaled integer format during ingestion. No intermediate double representation, no std::stod. All price comparisons in the matching engine are integer operations.
// "100.0550" becomes 1000550 (integer comparison, exact representation)
constexpr uint64_t PRICE_SCALE = 10000;The problem: Writing trade reports directly from the matching thread (even buffered fwrite) blocks on I/O syscalls, adding milliseconds of jitter.
What I did: The matching thread pushes TradeReport structs into a 131,072-slot lock-free ring buffer using std::memory_order_release. A dedicated background thread drains the buffer and flushes to disk in batches of 1,000. The ring buffer uses cached head/tail indices to minimize atomic loads. The producer only re-reads the consumer's head when it thinks the buffer might be full.
// Producer (matching thread): single atomic store
m_tail.store(current_tail + 1, std::memory_order_release);
// Consumer (logger thread): single atomic store
m_head.store(current_head + 1, std::memory_order_release);The problem: std::ifstream + std::getline + std::stod copies data multiple times: kernel buffer to userspace buffer to string to parsed value.
What I did: The input CSV is memory-mapped directly into the process address space with mmap + madvise(MADV_SEQUENTIAL). The Parser walks a raw char* pointer across the mapped region, parsing integers and fixed-point decimals byte-by-byte without any string construction, copies, or heap allocation.
These are the core systems engineering concepts that this project was designed to exercise, as outlined in the original problem statement:
| Concept | Where It Shows Up |
|---|---|
| Cache locality (L1/L2/L3 vs. RAM hierarchy) | alignas(64) on Order/Limit, contiguous MemoryPool arenas, flat hash table |
| Custom memory pools & arenas | MemoryPool<T>: free-stack allocator with O(1) alloc/dealloc, zero syscalls |
| Lock-free concurrency & memory ordering | SPSC RingBuffer with release/acquire atomics, cached indices |
Zero-copy I/O via mmap |
Parser maps CSV directly, walks raw char*, madvise(MADV_SEQUENTIAL) |
| Struct packing & false sharing prevention | alignas(CACHE_LINE_SIZE) on all hot-path structs and atomic variables |
| Fixed-point arithmetic | PRICE_SCALE = 10000, integer-only price comparisons, no double on hot path |
nanomatch/
├── include/nanomatch/
│ ├── Types.hpp # Price/OrderId/Quantity typedefs, Side enum,
│ │ # fixed-point conversion, TradeReport struct
│ ├── MemoryPool.hpp # Template arena allocator with free-stack recycling
│ ├── Order.hpp # alignas(64) Order and Limit structs
│ │ # with intrusive doubly-linked list pointers
│ ├── OrderBook.hpp # Matching engine interface
│ ├── BaselineOrderBook.hpp # STL-based engine (std::map + std::unordered_map)
│ │ # used as benchmark control
│ ├── RingBuffer.hpp # SPSC lock-free ring buffer with cached indices
│ ├── Logger.hpp # Async trade logger interface
│ └── Parser.hpp # Zero-copy mmap CSV parser with integer-only
│ # price decoding
├── src/
│ ├── OrderBook.cpp # Matching engine implementation:
│ │ # limit/market order matching, price level
│ │ # management, flat hash table operations
│ ├── Logger.cpp # Background logger thread with batched flushing
│ └── main.cpp # CLI entry point, pipeline orchestration, metrics
│
├── tests/
│ └── TestLOB.cpp # Google Test suites: perfect match, partial fill,
│ # price-time priority, cancellation, market orders,
│ # memory pool recycling, SPSC correctness
├── benchmarks/
│ └── BenchmarkLOB.cpp # Google Benchmark: optimized vs. STL baseline,
│ # per-event latency percentile profiling
│ # (p50/p90/p99/p99.9/p99.99)
├── scripts/
│ └── generate_data.py # Synthetic order flow generator with configurable
│ # event count and realistic price movement
├── CMakeLists.txt # Build config: C++20, -O3, -march=native,
│ # FetchContent for GTest + GBenchmark
└── .gitignore
~1,800 lines of code total across headers, source, tests, and benchmarks. No external dependencies beyond Google Test and Google Benchmark (fetched automatically via CMake).
| Requirement | Minimum Version |
|---|---|
| C++ Compiler | GCC 10+ or Clang 11+ (C++20 support) |
| CMake | 3.16+ |
| Python | 3.x (only for dataset generation) |
| OS | Linux (uses mmap, madvise, POSIX file I/O) |
mkdir -p build && cd build
cmake -DCMAKE_BUILD_TYPE=Release ..
make -j$(nproc)Release mode enables -O3 -march=native -funroll-loops automatically. The first build will download Google Test and Google Benchmark via CMake FetchContent.
./build/unit_testsThis runs 7 test cases covering:
- Memory pool allocation, deallocation, and slot recycling
- SPSC ring buffer push/pop, capacity limits, and wraparound
- Limit order matching (perfect fill, partial fill)
- Price-time priority enforcement across multiple resting orders
- Order cancellation and price level cleanup
- Market order sweeping across multiple price levels
python3 scripts/generate_data.py synthetic_orders.csv 1000000Generates 1M market events with a realistic distribution:
- 70% limit order additions (prices distributed around a random-walking midpoint)
- 25% cancellations (randomly selected from active orders)
- 5% market orders (immediate execution against resting liquidity)
./build/nanomatch synthetic_orders.csv executed_trades.csvProcesses all events end-to-end: mmap to parse to match to async log. Prints ingestion throughput and timing on completion.
./build/benchmarksThis runs two things:
- Latency percentile profile: per-event nanosecond timing for both engines across 1M events, reporting p50 through p99.99
- Google Benchmark throughput comparison: NanoMatch vs. STL baseline with
DoNotOptimizebarriers
I implemented price-time priority (FIFO) matching, the same algorithm used by most major exchanges (NYSE, NASDAQ, CME).
-
Limit Order arrives: the engine checks the opposite side of the book for crossable prices. If the incoming buy price >= best ask (or incoming sell price <= best bid), it matches against resting orders at each price level in FIFO order until the incoming quantity is exhausted or no more crossable levels exist. Any remaining quantity rests on the book.
-
Market Order arrives: same as above, but with no price limit. It sweeps through all available price levels on the opposite side until filled.
-
Cancel arrives: O(1) lookup by order ID, O(1) removal from the doubly-linked list at its price level. If the price level becomes empty, it's removed from the sorted level list and the flat hash table, and the
Limitobject is returned to the memory pool.
I included a BaselineOrderBook that implements the same matching logic using standard library containers:
| Component | NanoMatch | STL Baseline |
|---|---|---|
| Price levels | Flat hash table (linear probing) | std::map (red-black tree) |
| Order lookup | Direct array (Order*[]) |
std::unordered_map |
| Order storage | MemoryPool<Order> (arena) |
Heap-allocated std::list nodes |
| Memory allocation | Zero on hot path | Every insert/erase |
| Cache behavior | Contiguous, aligned | Scattered, pointer-chasing |
Both engines produce identical trade reports for the same input. The baseline exists purely to quantify the cost of conventional data structure choices.
This is a project I built for learning and benchmarking. Some things it intentionally does not do:
- No network layer: ingests from CSV files, not a FIX/ITCH/OUCH feed
- No multi-symbol support: single order book instance per run
- No persistence: state is lost on shutdown
- No order modification: supports add, cancel, and market orders only
- Linux only: relies on
mmap/madvise/POSIX APIs for zero-copy I/O
Built as part of the DIY '26 project series by the Finance & Economics Club (FEC), IIT Guwahati. The original problem statement, mentorship framework, and evaluation criteria were designed by the FEC team.
Mentors: Shubham Rane, Tanishq Kothari
MIT License. See LICENSE for details.