Trading RL Agent

A production-grade hybrid reinforcement learning trading system that combines CNN+LSTM supervised learning with deep RL optimization for algorithmic trading.

🚀 Features

Core Components

• CNN+LSTM Models: Hybrid neural networks for market pattern recognition
• Reinforcement Learning: SAC, TD3, PPO agents for trading decision optimization
• Feature Engineering: 150+ technical indicators with robust implementation
• Data Pipeline: Multi-source data ingestion with parallel processing
• Risk Management: VaR, CVaR, position sizing, and portfolio optimization
• Real-time Processing: Live data feeds and sentiment analysis

Infrastructure

• Configuration Management: YAML-based configuration with validation
• CLI Interface: Unified command-line interface using Typer
• Logging & Monitoring: Structured logging with MLflow/TensorBoard integration
• Testing: Comprehensive test suite with pytest
• Code Quality: Black, isort, ruff, mypy integration
• Docker Support: Containerized deployment ready

Performance Optimizations

• Parallel Data Fetching: Ray-based parallel processing
• Mixed Precision Training: Faster training with memory reduction
• Memory-Mapped Datasets: Memory reduction for large datasets
• Advanced LR Scheduling: Faster convergence
• Gradient Checkpointing: Train larger models with same memory

📦 Installation

Prerequisites

• Python 3.9+ (3.12 recommended)
• Git
• Docker (optional, for containerized deployment)

Quick Setup

1. Clone the repository

   git clone https://github.com/yourusername/trading-rl-agent.git
   cd trade-agent

2. Set up development environment

   # Core dependencies only (fast setup)
   ./setup-env.sh core
   
   # Add ML dependencies
   ./setup-env.sh ml
   
   # Full production setup
   ./setup-env.sh full

3. Configure environment variables

   # Copy the example environment file
   cp .env.example .env
   
   # Edit .env with your actual API keys
   # ⚠️  NEVER commit .env files with real API keys!
   nano .env

   Required environment variables:

   • QWEN_API_KEY: Your Qwen API key from OpenRouter
   • ALPACA_API_KEY: Your Alpaca Markets API key
   • ALPACA_SECRET_KEY: Your Alpaca Markets secret key

4. Install pre-commit hooks

   pre-commit install

5. Verify installation

   python -c "import trading_rl_agent; print('✅ Package imported successfully')"

🎯 Quick Start

1. Use the Unified CLI

# Show version and system info
trade-agent version
trade-agent info

# Download market data
trade-agent data download --symbols "AAPL,GOOGL,MSFT" --start 2023-01-01

# Process and build datasets
trade-agent data prepare --input-path data/raw --output-dir outputs/datasets

# Train CNN+LSTM model
trade-agent train cnn-lstm --epochs 100 --gpu --output models/

# Train RL agent
trade-agent train rl --epochs 50 --output models/

# Evaluate models
trade-agent evaluate models/best_model.pth --data data/test_data.csv

# Run backtesting
trade-agent backtest data/historical_data.csv --model models/agent.zip

# Start live trading
trade-agent live start --paper --symbols "AAPL,GOOGL"

2. Alternative Entry Points

# Using the installed command
trade-agent version
trade-agent data download --symbols "AAPL,GOOGL"

# Using Python module
python -m trade_agent.cli version
python -m trade_agent.cli train cnn-lstm

📚 Documentation

• Project Status: Current development status and roadmap
• Development Roadmap: Detailed task list and priorities
• Contributing Guidelines: How to contribute to the project
• Official Documentation: Full documentation and guides

🏗️ Architecture

┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│   Data Sources  │    │  Feature Eng.   │    │  CNN+LSTM Model │
│                 │    │                 │    │                 │
│ • yfinance      │───▶│ • Technical     │───▶│ • Pattern       │
│ • Alpha Vantage │    │   Indicators    │    │   Recognition   │
│ • Professional  │    │ • Alternative   │    │ • Uncertainty   │
│   Feeds         │    │   Data          │    │   Estimation    │
└─────────────────┘    └─────────────────┘    └─────────────────┘
                                │
                                ▼
                       ┌─────────────────┐    ┌─────────────────┐
                       │  RL Environment │    │  RL Agents      │
                       │                 │    │                 │
                       │ • State Space   │───▶│ • SAC           │
                       │ • Action Space  │    │ • TD3           │
                       │ • Reward Func   │    │ • PPO           │
                       └─────────────────┘    └─────────────────┘
                                │
                                ▼
                       ┌─────────────────┐    ┌─────────────────┐
                       │ Risk Management │    │ Portfolio Mgmt  │
                       │                 │    │                 │
                       │ • VaR/CVaR      │───▶│ • Multi-asset   │
                       │ • Position Size │    │ • Rebalancing   │
                       │ • Monitoring    │    │ • Analytics     │
                       └─────────────────┘    └─────────────────┘

🧪 Testing

# Run all tests
python -m pytest

# Run specific test categories
python -m pytest tests/unit/
python -m pytest tests/integration/
python -m pytest tests/smoke/

# Run with coverage
python -m pytest --cov=trading_rl_agent

🔧 Development

Code Quality

# Format code
black src/ tests/
isort src/ tests/

# Lint code
ruff check src/ tests/
mypy src/

# Run all quality checks
python run_comprehensive_tests.py --quality-only

Adding Features

1. Create a feature branch: git checkout -b feature/your-feature
2. Implement your changes following the contributing guidelines
3. Add tests for new functionality
4. Run the test suite: python -m pytest
5. Submit a pull request

📊 Current Status

• Core Infrastructure: ✅ Complete (63K+ lines of code)
• Data Pipeline: ✅ Complete with parallel processing
• CNN+LSTM Training: ✅ Complete with optimizations
• RL Agents: ✅ Complete (SAC, TD3, PPO with advanced optimization)
• Risk Management: ✅ Complete (VaR, CVaR, Monte Carlo)
• Portfolio Management: ✅ Complete (attribution, transaction costs)
• Production Deployment: ✅ Complete (Docker, Kubernetes ready)
• Monitoring & Logging: ✅ Complete (system health, alerts)
• Testing: 🔄 In Progress (comprehensive test suite)
• Documentation: 🔄 Updated to reflect current state

🚀 Performance Benchmarks

Component	Before Optimization	After Optimization	Improvement
Data Fetching	Sequential	Parallel (Ray)	10-50x faster
Training Speed	Standard	Mixed Precision	2-3x faster
Memory Usage	Standard	Optimized	30-50% less
Dataset Loading	Standard	Memory-mapped	60-80% less memory
Convergence	Standard	Advanced LR	1.5-2x faster

🧪 Testing Status

Current Test Suite: Comprehensive Coverage

Test Results:

• ✅ Core functionality tests passing
• 🔄 Some integration tests need dependency fixes
• 📊 Extensive test coverage across all major components

Well-Tested Components:

• ✅ Core Configuration System
• ✅ Agent Configurations
• ✅ Exception Handling
• ✅ CLI Backtesting
• ✅ Data Caching
• ✅ Risk Management
• ✅ Portfolio Attribution

Needs Attention:

• 🔄 Some dependency issues (structlog missing)
• 🔄 Ray parallel processing compatibility
• 🔄 Integration test environment setup

🤝 Contributing

Please read CONTRIBUTING.md for details on our code of conduct and the process for submitting pull requests.

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

• PyTorch team for the excellent deep learning framework
• Ray team for parallel processing capabilities
• The open-source community for inspiration and contributions