v1.4.4
HERA
HERA is a MATLAB based scientific ranking framework for paired benchmarking. It provides a hierarchical-compensatory logic, advanced effect-size statistics, and automated reporting capabilities.
For more information please refer to the Readme and Project Documentation.
What's New in v1.4.4?
This update introduces Example 5 (LLM Benchmarking on MMLU-Pro), improves rendering stability for multi-page plots, refines bootstrap convergence defaults, and expands data documentation.
⚙️ Enhancements & New Datasets
- LLM Benchmarking Dataset (Example 5): Added Example 5 featuring evaluation data derived from MMLU-Pro to demonstrate HERA's application in evaluating and ranking Large Language Models (LLMs).
- Bootstrap Convergence Defaults: Adjusted percentile bootstrap default settings for thresholds to ensure convergence when effect sizes are very small.
🐛 Bug Fixes
- Multi-Page Plot Rendering: Fixed rendering collapse in multi-page detail comparison plots by increasing figure pause durations during page rendering.
Recent Major Features (from v1.4.0)
✨ Convergence Analysis Enhancements:
- New Scenario: Added a "Sensitivity" scenario with small effect sizes.
- JSON Configuration: All scenarios are fully customizable through a JSON configuration file
(For a full list of features, see the v1.4.0 Release Notes)
🖥️ Platform Support
- MATLAB Toolbox & Python Package: Windows, macOS (Intel & Apple Silicon), Linux
- Standalone Runtime Application: macOS (Apple Silicon ONLY)
📦 Compatibility
- MATLAB: Compatible with R2020a and newer.
- Python: Compatible with Python 3.9 through 3.12 and Matlab Runtime R2025b (25.2).
Full Changelog: v1.4.3...v1.4.4