Skip to content

v0.1.20 - Parameter Sweeps & Catalog

Choose a tag to compare

View all tags
@stabilefrisur stabilefrisur released this 20 Dec 20:36
· 22 commits to master since this release
v0.1.20
a84dfc5

Added

  • Parameter Sweep Engine (spec 012-parameter-sweeps)
    • Self-contained sweep configuration files (YAML) defining parameter experiments independently from catalogs
    • Two evaluation modes:
      • Indicator mode: Analyze indicator characteristics (distribution, correlation, stationarity) without running backtests
      • Backtest mode: Run full backtests measuring Sharpe ratio, drawdown, hit rate, total trades, and annualized return
    • Grid mode parameter generation testing all combinations of specified parameter values
    • Runtime parameter override mechanism using dot notation paths (e.g., indicator_transformation.parameters.lookback)
    • Progress tracking with ETA and current parameter display during execution
    • Dry-run mode for previewing combinations before execution
    • Configurable max combinations limit for exploratory sweeps
    • Timestamped output directories (data/sweeps/<name>_<timestamp>/)
    • Parquet results file with metrics for each parameter combination
    • Configuration copy and JSON summary metadata for reproducibility
    • Queryable results by any collected metric
    • CLI command: aponyx sweep <config.yaml> with --dry-run flag
    • Three example configurations: sweep_indicator.yaml, sweep_backtest.yaml, sweep_comprehensive.yaml
    • Complete specification in specs/012-parameter-sweeps/
    • 734 tests for sweep module (evaluators, reports, config, engine)
  • Unified YAML Catalog Management (spec 011-catalog-orchestration)
    • CatalogManager class for unified CRUD operations across all catalog files
    • Two YAML source files: config/catalogs.yaml (signals, strategies, transformations) and config/securities.yaml (securities, instruments)
    • Cross-reference validation between signals, transformations, and securities
    • Bidirectional sync: YAML → JSON for runtime registries (7 JSON files generated)
    • CLI commands: aponyx catalog validate, aponyx catalog sync, aponyx catalog migrate
    • One-time migration tool to bootstrap YAML from existing JSON catalogs
    • Comment preservation during round-trip editing via ruamel.yaml
    • Generation marker (_generated field) in JSON files indicating auto-generation
    • Typed entry classes: SignalEntry, StrategyEntry, IndicatorTransformationEntry, ScoreTransformationEntry, SignalTransformationEntry, SecurityEntry, InstrumentEntry
    • ValidationResult with structured error reporting (category, entry name, message)
    • 77 new tests for catalog module across 6 test files
  • Data Channel Separation (spec 010)
    • DataChannel enum (SPREAD, PRICE, LEVEL) for typed channel access
    • UsagePurpose enum (INDICATOR, PNL, DISPLAY) for context-aware resolution
    • SecuritySpec and SecurityCatalog for multi-channel configuration management
    • ChannelConfig for Bloomberg ticker and field mapping per channel
    • fetch_security_data() for unified channel-aware data fetching
    • resolve_channel_for_purpose() for automatic channel selection
    • list_security_channels() for introspection
    • display_channel parameter in WorkflowConfig for visualization override
    • Multi-ticker support with automatic date alignment (inner join)
    • 861 lines of tests for channel types, catalog validation, and fetch integration
  • Entry Threshold for Asymmetric Entry/Exit
    • entry_threshold parameter in BacktestConfig and StrategyMetadata
    • Separate entry condition (|signal| >= threshold) from exit condition (signal returns to zero)
    • Enables mean-reversion strategies where trades run before closing
    • All strategies in catalog now have entry_threshold values (conservative: 1.8, balanced: 1.5, aggressive: 1.0)
  • Deterministic Test Scenarios
    • test_scenarios module with 18 pre-built scenarios for testing
    • Scenarios include: profitable/unprofitable trades, correlation patterns, stability cases, risk management triggers
    • Each scenario includes expected outcomes dict for test assertions
  • Research Dashboard Integration
    • SignalStep now saves indicator, score, and signal parquet files
    • VisualizationStep generates 5-panel research_dashboard.html
    • Deleted standalone 09_research_dashboard.py (functionality integrated into workflow)
    • Updated 08_visualize_results.py with generate_from_workflow() function

Changed

  • New Dependency
    • Added ruamel.yaml>=0.18.0 for YAML parsing with comment preservation
  • CLI Guide Rationalization
    • Condensed cli_guide.md from 900 to 620 lines (31% reduction)
    • Compact table formats for Product Microstructure, Position Sizing, Troubleshooting
    • Consolidated Common Workflows section
    • Preserved all essential information and command documentation
  • BREAKING: Bloomberg Securities Catalog Structure
    • Updated to multi-channel structure with available_channels and channel_config
    • CDX: spread channel only (spread column, 0-10000 bps validation)
    • ETF: both spread and price channels (spread via OAS, price with 0-1000 validation)
    • VIX: level channel only (level column, 0-200 validation)
    • Legacy single-column format NOT supported (manual conversion required)
  • Channel Resolution Rules
    • INDICATOR: Uses instrument_type defaults (CDX→spread, VIX→level, ETF→spread)
    • PNL: Uses quote_type from security spec (spread or price quoted)
    • DISPLAY: Uses instrument_type defaults, overridable with display_channel
  • Product Microstructure Consolidation
    • Moved dv01_per_million and transaction_cost_bps from strategy catalog to bloomberg_securities.json
    • Added get_product_microstructure() function in bloomberg_config.py
    • Updated WorkflowConfig with microstructure override fields
    • BacktestStep merges product microstructure with runtime overrides
  • Dynamic Transaction Costs
    • Added transaction_cost_pct field for spread-based costs (2.5% default)
    • _calculate_transaction_cost() helper supports static (bps) and dynamic (%) modes
  • ETF Workflow Example
    • Added workflow_etf.yaml demonstrating price-based product backtesting

Fixed

  • Parameter Override Application in signal composition
    • Fixed bug where runtime overrides were not correctly applied to signal computation
    • Enhanced signal_composer.py to properly merge runtime overrides with catalog parameters
    • All sweep evaluations now use correct parameter values from configuration
  • Signal Strength Attribution
    • Fixed pd.qcut error when signal values cluster at boundaries (e.g., bounded signals at ±1.5)
    • Now uses labels=False and dynamic bin count to handle duplicate quantile edges
    • Logs warning when quantiles are reduced due to duplicate values
  • Spread Momentum Signal
    • Changed to trend-following strategy (sign_multiplier=-1, z_score_20d)
    • Corrected signal direction to match strategy intent
  • Deprecation Warnings
    • Added fill_method=None to pct_change() calls to resolve pandas deprecation warning

Validation Enhancements

  • Fail-fast validation in SecuritySpec.post_init
  • Spread-quoted products require positive dv01_per_million
  • quote_type must have corresponding channel (spread→SPREAD, price→PRICE)
  • Clear error messages include security ID, available channels, and guidance
  • VIX exempt from P&L validation (non-tradeable index)

Documentation

  • Updated README.md with parameter sweep section and CLI commands
  • Updated PROJECT_STATUS.md with sweep layer implementation details
  • Updated copilot-instructions.md with sweep usage patterns
  • Complete specification document in specs/012-parameter-sweeps/spec.md
  • Updated copilot-instructions.md with CatalogManager usage patterns
  • Updated cli_guide.md with catalog commands (validate, sync, migrate)
  • Added YAML catalog file locations to See Also section
  • Updated copilot-instructions.md with data channel patterns
  • Updated cli_guide.md with display_channel parameter and product microstructure
  • Updated adding_data_providers.md with channel architecture
  • Added inline examples for purpose-based channel resolution

Breaking Changes

  • Bloomberg securities catalog structure changed (no backward compatibility)
  • FileSource parquet files must include all channel columns defined in catalog
  • fetch_cdx(), fetch_vix(), fetch_etf() deprecated in favor of fetch_security_data()
  • Product microstructure fields moved from strategy catalog to bloomberg_securities.json

Test Coverage

  • 1,711 total tests passing (increased from 977)
  • 734 new sweep tests across evaluators, reports, config validation
  • Catalog module tests: 77 tests across 6 test files
  • Channel separation tests: 861 lines across 3 test modules
  • Zero regression on existing spread-based backtests
Assets 2
Loading