Skip to content

v0.3.0 - Multi-Source Data Retrieval with Backtest Optimization

Latest

Choose a tag to compare

@chuci-qin chuci-qin released this 08 Nov 15:24
· 38 commits to main since this release

πŸŽ‰ What's New in v0.3.0

✨ Major Features

Multi-Source Data Retrieval

  • βœ… DataRetriever - Unified interface for multiple data sources

    • 🏦 Exchange sources: 1024ex (βœ…), Binance, Coinbase, IBKR
    • πŸ“ˆ Finance sources: Yahoo Finance (βœ…), Bloomberg, Alpha Vantage
    • ⛓️ Blockchain sources: Chainlink (framework), The Graph
  • βœ… BacktestDataset - Optimized dataset for backtesting

    • Batch retrieval for multiple symbols
    • Train/test split functionality
    • CSV import/export
    • Data quality validation

Backtest Optimization

  • βœ… Data caching - Framework for caching historical data
  • βœ… Batch retrieval - Get multiple symbols in one call
  • βœ… Timestamp alignment - Critical for multi-asset backtest
  • βœ… Missing value handling - Forward fill for price data
  • βœ… Data validation - Remove duplicates and invalid data
  • βœ… Auto indicators - SMA (20, 50, 200), returns, log returns, volatility
  • βœ… Train/test split - Easy dataset splitting for backtesting
  • βœ… CSV support - Save and load datasets

Adapter Pattern Architecture

  • βœ… BaseAdapter - Abstract interface for all data sources
  • βœ… ExchangeAdapter - Wraps exchange APIs (1024ex implemented)
  • βœ… FinanceAdapter - Wraps finance data APIs (Yahoo implemented)
  • βœ… BlockchainAdapter - Wraps blockchain data sources (framework)

Standardized DataFrame Output
All data sources return the same format:

DataFrame columns:
  - timestamp (datetime): UTC time
  - open, high, low, close (float): OHLC prices
  - volume (float): trading volume
  - source (str): data source name ("1024ex", "yahoo", etc.)
  - provider_type (str): source type ("exchange", "finance", "blockchain")
  - asset_class (str): asset type ("crypto", "stock", "index", "forex")

πŸ§ͺ Testing & Quality

  • βœ… 24 new tests for data retrieval module
  • βœ… 125 total tests (24 new + 101 existing)
  • βœ… 100% pass rate
  • βœ… Integration with Yahoo Finance tested
  • βœ… Backtest scenarios tested
  • βœ… Multi-source compatibility verified

πŸ“š Documentation & Examples

  • βœ… Complete backtest example (examples/backtest_example.py)
  • βœ… 6 usage scenarios demonstrated
  • βœ… Multi-source data comparison examples
  • βœ… Train/test split examples
  • βœ… Comprehensive API documentation

πŸ“¦ Installation

# Basic installation
pip install quant1024==0.3.0

# With Yahoo Finance support
pip install quant1024[yahoo]==0.3.0

# With blockchain support
pip install quant1024[blockchain]==0.3.0

# With all optional features
pip install quant1024[all]==0.3.0

πŸš€ Quick Start

Basic Usage

from quant1024 import DataRetriever

# Get data from 1024ex
data = DataRetriever(source="1024ex", api_key="...", api_secret="...")
btc = data.get_klines("BTC-PERP", interval="1d", days=365)

# Get data from Yahoo Finance (free, no auth required)
data = DataRetriever(source="yahoo")
aapl = data.get_klines("AAPL", interval="1d", days=365)

# Data comes with metadata
print(btc['source'].iloc[0])        # "1024ex"
print(btc['provider_type'].iloc[0]) # "exchange"
print(btc['asset_class'].iloc[0])   # "crypto"

Backtest Dataset

from quant1024 import BacktestDataset

# Create backtest dataset
dataset = BacktestDataset(
    source="yahoo",
    symbols=["BTC-USD", "ETH-USD", "AAPL", "SPY"],
    interval="1d",
    days=365,
    enable_cache=True
)

# Load all data
data_dict = dataset.load(
    fill_missing=True,
    validate_data=True,
    add_indicators=True,
    align_timestamps=True  # Critical for multi-asset backtest
)

# Split for training and testing
train, test = dataset.split(train_ratio=0.8)

# Get summary
summary = dataset.get_summary()
print(summary)

Multi-Source Comparison

from quant1024 import DataRetriever
import pandas as pd

# From 1024ex
data_1024 = DataRetriever(source="1024ex", api_key="...", api_secret="...")
btc_1024 = data_1024.get_klines("BTC-PERP", interval="1h", days=7)

# From Yahoo Finance
data_yahoo = DataRetriever(source="yahoo")
btc_yahoo = data_yahoo.get_klines("BTC-USD", interval="1h", days=7)

# Compare prices
df_all = pd.concat([btc_1024, btc_yahoo])
comparison = df_all.groupby('source')['close'].agg(['mean', 'min', 'max'])
print(comparison)

πŸ”§ Technical Details

New Dependencies

Required:

  • pandas>=2.0.0 - DataFrame operations
  • numpy>=1.24.0 - Numerical computing

Optional:

  • yfinance>=0.2.0 - Yahoo Finance support
  • web3>=6.0.0 - Blockchain support

Supported Data Sources

Source Type Asset Classes Auth Required Free Status
1024ex exchange crypto βœ… βœ… βœ… Implemented
Yahoo Finance finance all ❌ βœ… βœ… Implemented
Binance exchange crypto βœ… βœ… πŸ”„ Planned
Chainlink blockchain crypto ❌ βœ… πŸ”„ Framework
Bloomberg finance all βœ… ❌ πŸ”„ Planned
IBKR broker all βœ… ❌ πŸ”„ Planned

Supported Time Intervals

1m, 5m, 15m, 30m, 1h, 2h, 4h, 6h, 12h, 1d, 1w


πŸ”— Links

πŸ“Š Statistics

  • Files Changed: 13 files
  • Lines Added: 2,170 insertions
  • New Tests: 24 tests
  • Total Tests: 125 tests (100% pass)
  • Code Quality: Fully type-annotated

πŸ”„ Breaking Changes

None. v0.3.0 is fully backward compatible with v0.2.0.

All v0.2.0 features continue to work:

  • βœ… Exchange1024ex - 38 API endpoints
  • βœ… HMAC authentication
  • βœ… Pydantic models
  • βœ… Exception handling

πŸ™ Acknowledgments

Thanks to all contributors and users for feedback and support!

Full Changelog: v0.2.0...v0.3.0