Market Data Feed Handler

This repository showcases a low-latency C++20 Market Data Feed Handler simulator (the Market Plant). The motivation behind this project was to build a high-performance middleware service that ingests exchange-style UDP feeds and efficiently scales to one-to-many subscribers via gRPC.

What is a Market Data Feed Handler?

A Market data feed handler connects exchange feeds with internal trading systems, turning raw data into digestible market data and streaming to downstream consumers at scale. They’re a crucial part of market infrastructure and are designed to move high-volume, latency-sensitive data.

Architecture

The diagram below represents the Market Plant system architecture at a high level, showing the data flow between the Exchange simulator, Market Plant server, and subscriber(s).

Market Data Feed Handler

The Market Plant primarily:

• Ingests UDP unicast datagrams from a simulated exchange using Nasdaq’s MoldUDP64 Protocol, including gap detection and retransmission.
• Builds and maintains in-memory L2 price-level order books for configurable instruments from exchange feed data.
• Streams real-time snapshots and deltas to subscribers via server-side gRPC streaming efficiently.

UDP Unicast

For Exchange → Plant communication, the simulator sends the feed over UDP unicast to replicate how market data is delivered. In production, exchange feeds often use multicast for efficient one-to-many distribution, with unicast used for recovery/retransmission. However, for the scope of this project (single Exchange and Market Data Feed Handler), unicast suffices and provides the same advantages as multicast.

moldudp64_client.h implements a MoldUDP64 client state machine that:

• Parses the MoldUDP64 header (session, sequence number, message count) and tracks the active session.
• Enforces in-order processing using sequencing, dropping late/duplicate datagrams.
• Detects sequence gaps and enters recovery state (either cold-start backfill or mid-stream gapfill).
• Retransmits requests starting at the missing sequence number, throttled by a timeout and bounded by MAX_MESSAGE_COUNT.

Below is an example of the message payload utilized (Big-Endian/NBO). As mentioned before, Each MoldUDP64 message is encoded as: msg_len (u16) then msg_len bytes of payload. The offsets below are byte offsets from the start of the UDP datagram buffer.

Payload Offset (bytes)	Field	Size	Type
0–3	instrument_id	4	u32
4	side	1	u8
5	event	1	u8
6–9	price	4	u32
10–13	quantity	4	u32
14–21	exchange_ts	8	u64

Note: for this simulator, each UDP datagram carries a single MoldUDP64 message, while preserving the MoldUDP64 Protocol and sequencing semantics.

gRPC

I decided to use gRPC with server-side streaming because it provides bidirectional communication over a single persistent connection, therefore eliminating the connection overhead associated with repeated HTTP requests. Server-side streaming allows the Market Plant to push updates to subscribers in real-time as market events occur, eliminating the need for clients to poll for data.

The protocol also uses HTTP/2 multiplexing, which allows us to handle multiple concurrent streams over a single TCP connection. Lastly, gRPC's native support for structured data via Protocol Buffers provides type-safe message serialization that can be more efficient than JSON.

Exchange Simulator

The Exchange Simulator produces market movement for testing the Market Plant. It continuously generates randomized L2 price-level deltas (add level, reduce level, remove level) across instruments and sides, serializes each event into MoldUDP64-framed UDP datagrams, and sends them to the Market Plant over UDP unicast.

To support gap recovery, the simulator also keeps a fixed-size in-memory history buffer keyed by sequence number. When it receives retransmission requests (MoldUDP64 header containing a starting sequence number and message count), it re-enqueues the requested events and replays them back to the Market Plant.

Project Structure

• config/config.json Runtime instrument configuration.

• protos/market_plant/market_plant.proto Protobuf definitions for the Market Plant gRPC API.

• src/app/ Top-level applications.

  • exchange/exchange.h Exchange simulator.
  • subscriber/subscriber.h gRPC subscriber client example.

• src/market/ Market Plant main logic.

  • server/market_plant.h Market Plant gRPC server (service implementation).
  • market_core.h Core market components (OrderBook / BookManager).
  • event.h Market event types and shared structures.
  • market_plant_config.h Runtime network/config defaults.
  • cli/market_cli.h CLI parsing.

• src/network/ Networking + wire-format utilities.

  • exchange_feed.h Exchange → Market Plant UDP feed ingestion + event parsing.
  • moldudp/moldudp64.h MoldUDP64 client FSM logic.
  • utils/udp_messenger.h UDP socket send wrapper.
  • utils/endian.h Big-endian (Network Byte Order) read/write helpers.

Usage

Instrument Configuration

The Market Plant Server is configurable via a JSON configuration file that defines the instruments to track:

{
    "instruments": [
        {
            "instrument_id": 1,
            "symbol": "AAPL",
            "specifications": {
                "depth": 10
            }
        },
        {
            "instrument_id": 2,
            "symbol": "META",
            "specifications": {
                "depth": 5
            }
        }
    ]
}

Configuration fields:

• instrument_id: Unique identifier for the instrument
• symbol: Trading symbol (informational)
• depth: Number of price levels to maintain in the order book

Environment Variables

The Market Plant also supports runtime configuration via environment variables:

Variable	Description	Default
GRPC_HOST	gRPC server bind address	0.0.0.0
GRPC_PORT	gRPC server port	50051
MARKET_IP	UDP socket bind address	127.0.0.1
MARKET_PORT	UDP socket bind port	9001
EXCHANGE_IP	Exchange simulator address	127.0.0.1
EXCHANGE_PORT	Exchange simulator port	9000

Running Market Plant

# Use default configuration
./market_plant --config config.json

# Pin exchange feed thread to specified CPU core 
./market_plant --config config.json --cpu 7

# Custom configuration
GRPC_HOST=0.0.0.0 \
GRPC_PORT=8080 \
MARKET_PORT=9002 \
EXCHANGE_IP=10.0.0.1 \
EXCHANGE_PORT=9000 \
./market_plant --config config.json

gRPC API

The Market Plant exposes two main RPC methods for market data streaming and subscription management.

1. StreamUpdates() (Server-side Streaming)

Establishes a streaming connection to receive real-time order book updates.

Request:

message Subscription {
    oneof action {
        InstrumentIds subscribe = 1;    // Initial subscription(s)
    }
}

message InstrumentIds {
    repeated uint32 ids = 1;
}

Response Stream:

message StreamResponse {
    oneof payload {
        SubscriberInitialization init = 1;    // First message
        OrderBookUpdate update = 2;           // Subsequent messages
    }
}

At a high level:

1. The client starts a streaming session by subscribing to one or more instrument IDs.
2. The server acknowledges the session with subscriber_id and session_id.
3. The server publishes an initial snapshot per instrument (top-N depth on bid and ask side).
4. The stream continues with incremental updates reflecting real-time book changes.

2. UpdateSubscriptions() (Subscription Management)

Modifies an existing subscription to add or remove instruments.

Request:

message UpdateSubscriptionRequest {
    uint32 subscriber_id = 1;     // From initialization
    bytes session_id = 2;         // From initialization
    Subscription change = 3;      // Add or remove instruments from client subscription list
}

Response:

google.protobuf.Empty

Basic Authentication:

• Requires subscriber_id and session_id from the initial StreamUpdates connection.
• Invalid credentials return PERMISSION_DENIED error.

Order Book Updates

Snapshot Updates

Sent when a client first subscribes to an instrument. Contains the current state of the order book up to the configured depth.

message SnapshotUpdate {
    repeated OrderBookEventUpdate bids = 1;  // Top-N bid levels
    repeated OrderBookEventUpdate asks = 2;  // Top-N ask levels
}

Incremental Updates

Sent as market data changes occur. Each update modifies a single price level.

message IncrementalUpdate {
    OrderBookEventUpdate update = 1;
}

message OrderBookEventUpdate {
    OrderBookEventType type = 1;  // Delta Type
    Level level = 2;
}

message Level {
    Side side = 1;        // BID or ASK
    uint32 price = 2;     // Price level
    uint32 quantity = 3;  // Quantity to add/remove
}

Event Types:

• ADD_LEVEL: Adds quantity to a price level (creates level if it doesn't exist).
• REDUCE_LEVEL: Removes quantity from a price level (deletes level if quantity reaches zero).

The repository also includes a sample subscriber that displays a live order book:

# Subscribe to instrument 1 (default)
./subscriber

# Custom configuration
GRPC_HOST=192.168.1.100 \
GRPC_PORT=50051 \
INSTRUMENT_IDS=1,2,3,4 \
./subscriber

Styling

The Market Plant primarily utilizes the naming conventions provided below:

• Google C++ Style Guide
• Protobuf Styling