Skip to content

kcherneha/highload_hft_course

Repository files navigation

High-Frequency Trading (HFT) Market Maker Component

Requirements Overview

  1. Low Latency: The system must execute trades and update orders with minimal delay. Order Placement: Rapidly place buy and sell orders based on market conditions. Order Cancellation: Quickly cancel or modify existing orders.
  2. Reliability: Ensure continuous operation and accurate order processing.
  3. Scalability: Handle increasing trade volumes and market data efficiently.
  4. Maintainability: Easy to monitor, manage, and update.

Functional Requirements

  1. Order Management

    • Order Placement: Rapidly place buy and sell orders based on market conditions.
    • Order Cancellation: Quickly cancel or modify existing orders.
  2. Market Data Handling

    • Real-time Data Processing: Consume and process real-time market data feeds with minimal delay.
    • Data Storage: Store historical market data for analysis and backtesting.
  3. Risk Management

    • Position Limits: Enforce limits on positions to manage risk.
    • Exposure Monitoring: Continuously monitor market exposure and adjust positions.
  4. Strategy Execution

    • Algorithmic Trading: Implement and execute trading algorithms based on predefined strategies.
    • Parameter Adjustments: Dynamically adjust strategy parameters based on market conditions.

Non-Functional Requirements

  1. Performance

    • Sub-millisecond latency for order execution.
    • High throughput to handle large volumes of orders and market data.
  2. Scalability

    • Horizontal scaling to handle more trades and data.
    • Vertical scaling for enhanced performance on powerful hardware.
  3. Reliability

    • High availability with minimal downtime.
    • Data integrity and consistency in order processing.
  4. Maintainability

    • Clear and concise logging for troubleshooting.
    • Automated deployment and configuration management.

Limitations

  1. Network Latency

    • The speed of order execution is limited by network latency between the trading system and exchanges.
  2. Algorithm Complexity

    • Highly complex algorithms might introduce processing delays, impacting overall latency.
  3. Data Volume

    • Storing and processing large volumes of historical market data can be challenging and resource-intensive.

Back-of-the-Envelope Calculations

  1. Latency Estimation

    • Order Execution Latency: Assume an execution latency of 200 microseconds for placing or canceling an order.
  2. Throughput Estimation

    • Assume a single instance can handle 10,000 orders per second.
  3. Market Data Processing

    • Data Feed Rate: Assume receiving 1,000 market updates per second per feed.
    • Processing Latency: Each update processed within 50 microseconds.
  4. Memory Usage

    • Order Book Storage: Each order entry is 100 bytes. For 1 million active orders, memory usage is:
      • 1,000,000 orders * 100 bytes = 100 MB
  5. Scalability

    • Horizontal Scaling: Adding more instances to handle additional orders and market data. If each instance can handle 10,000 orders/sec, a cluster of 10 instances can handle 100,000 orders/sec.
  6. Risk Management Overhead

    • Position Monitoring: Assume monitoring 1,000 positions per second with 10 microseconds per position.
    • Total Monitoring Time: 1,000 positions * 10 microseconds = 10 milliseconds
  7. Network Overhead

    • Bandwidth per Order: Assume each order involves 1 KB of data transfer.
    • Bandwidth per Node: For 10,000 orders/sec, the bandwidth requirement is:
      • 10,000 orders/sec * 1 KB = 10 MB/sec

Aggregate the Total Requirements

Given these factors, let's estimate the number of nodes and their RAM:

  1. Order Processing Nodes:
    • To handle 100,000 orders per second, we need 10 nodes (as each node handles 10,000 orders/sec).
    • Latency Requirements: Assume that each order must be processed within 200 microseconds.

Estimating CPU Requirements

  1. Order Processing Load

    • 10,000 orders per second
    • Assume each order processing takes approximately 200 microseconds.
    • CPU cycles needed per order: ( 200 \text{ microseconds} = 0.2 \text{ milliseconds} ).
    • Total CPU time per second: ( 10,000 \text{ orders} \times 0.2 \text{ milliseconds} = 2,000 \text{ milliseconds} = 2 \text{ seconds} ).
    • Since this is per second, a single core running at full capacity can handle this.

CPU Cores and Frequency Calculation

To handle 2,000 milliseconds of CPU time per second, we need 3 CPU cores (2+1 for smooth operation).

CPU Frequency

  1. Latency Requirements:

    • Order execution latency is 200 microseconds.
    • Modern high-performance CPUs (e.g., Intel Xeon or AMD EPYC) can achieve clock speeds around 3 GHz to 4 GHz.
    • At 3 GHz, each clock cycle is approximately ( \frac{1}{3,000} ) microseconds, or about 0.333 nanoseconds.
  2. Memory per Node:

    • Each node requires at least 100 MB for order book storage.
    • Additional memory for processing and overhead, let's assume a buffer of 400 MB.
    • Total Memory per Node: ( 100 \text{ MB} + 400 \text{ MB} = 500 \text{ MB} ).
  3. Network Bandwidth per Node:

    • Each node requires 10 MB/sec bandwidth.

Final Node Specifications

  • CPU: 3 cores per node, 3+ GHz frequency.
  • Number of Nodes: 10
  • RAM per Node: 500 MB
  • Network Bandwidth per Node: 10 MB/sec

Summary

  • Latency: Sub-millisecond latency for order execution, around 200 microseconds.
  • Throughput: A single instance can handle 10,000 orders/sec, scaling to 100,000 orders/sec with a 10-instance cluster.
  • Memory: Order book storage requires around 100 MB for 1 million active orders.
  • Market Data: Processing 1,000 market updates per second with 50 microseconds latency per update.
  • Network: Bandwidth requirement per node is around 10 MB/sec.

Please check system_diagram.md file and simple png diagram for it.

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published