Version 0.1.47

Version 0.1.47 Release

🚀 Major New Features

New CSV Fix Command

• New CLI command: fix-csv to safely remove last N rows from CSV files
• Safety features: Dry-run mode (default), backup support with timestamps
• Flexible targeting: Support for both VN and crypto modes, selective ticker processing
• Comprehensive stats: Detailed operation reporting and error handling

Enhanced Memory Cache Performance

• Minute data in memory: Now caching minute-level data for both VN and crypto modes
• Auto-reload: Added 60s TTL auto-reload for minute data
• Performance boost: Significantly faster minute-level queries

Lock-Free File Coordination

• Removed dependency: Eliminated fs2 file locking for better performance
• Improved concurrency: Lock-free coordination reduces bottlenecks during concurrent access

⚡ Performance Optimizations

Crypto API Client Improvements

• Smart batching: Automatic batching to prevent timeouts (1000 records/batch)
• Pre-check mode: Quick availability checks before full data fetch
• Rate limiting: 500ms delays between batches to avoid overwhelming APIs
• Enhanced retries: Better error handling with exponential backoff

API Enhancements

• Performance monitoring: Special tracking for 1m USDT requests
• Extended timeout: Increased request timeout from 30s to 180s
• Static file caching: Cache control headers for optimized static asset delivery

🛠️ Developer Tools

Debugging Scripts

• scripts/debug_local.sh - Local debugging automation
• scripts/debug_remote.sh - Remote debugging automation
• scripts/quick_crypto_test.sh - Quick crypto testing

Enhanced Testing

• Improved integration tests with better error handling
• Enhanced analysis tests with comprehensive coverage
• Better aggregated interval testing

🐛 Bug Fixes

• Fixed cache sufficiency check for crypto DataStore
• Improved dividend detection and data consistency
• Enhanced CSV parsing and validation
• Better error handling in crypto synchronization

📊 Statistics

• Files changed: 38 files
• Lines added: 6,407
• Lines removed: 841
• Net addition: ~5,566 lines of code and documentation

🔧 Upgrade Notes

• This release includes performance improvements that will benefit all users
• The new fix-csv command provides safe CSV data management
• Enhanced memory caching improves API response times significantly