Skip to content
This repository was archived by the owner on Sep 28, 2025. It is now read-only.

v0.3.0 - Aggregation Testing & GitHub Actions Fix

Choose a tag to compare

@Goldziher Goldziher released this 04 Jul 11:22
· 164 commits to main since this release
bdb3656

v0.3.0 - Aggregation Testing & GitHub Actions Fix

🐛 Bug Fixes

  • Fixed GitHub Actions aggregation error: Resolved "'dict' object has no attribute 'total_runs'" error in reporting pipeline
  • Improved msgspec type loading: All msgspec.json.decode() calls now specify proper type parameters
  • Fixed enum object preservation: Framework and DocumentCategory enums are now correctly maintained during serialization/deserialization

🧪 Testing Improvements

  • Added comprehensive integration tests covering the full aggregation-to-reporting pipeline
  • CLI report generation testing from aggregated files on disk
  • msgspec type loading consistency validation to prevent regression
  • GitHub Actions scenario coverage for the exact failure case

🔄 Files Updated

  • src/cli.py - Fixed aggregated results loading with proper type specification
  • src/aggregate.py - Fixed benchmark results loading with type specification
  • src/quality_assessment.py - Fixed benchmark results loading with type specification
  • src/visualize.py - Fixed aggregated results loading with type specification
  • .github/workflows/deploy-pages.yml - Fixed aggregated results loading with proper msgspec usage
  • tests/aggregation_test.py - New comprehensive test suite (12 tests)
  • tests/conftest.py - Test configuration

🎯 What This Fixes

The aggregation step in GitHub Actions was failing because:

  1. Root Cause: msgspec.json.decode() without type specification returns plain Python dictionaries
  2. Error Chain: Aggregation → Save → Load (as dict) → Report generation (expects AggregatedResults object) → AttributeError
  3. Solution: All decode operations now use type=AggregatedResults or type=list[BenchmarkResult] as appropriate

⚡ Performance Impact

  • No performance degradation - msgspec with type specification is equally fast
  • Better type safety and validation during JSON loading
  • Prevents silent data corruption issues

This release ensures the full benchmark automation pipeline works reliably in GitHub Actions.


🤖 Generated with Claude Code
Co-Authored-By: Claude noreply@anthropic.com