v0.4.0

[0.4.0] - 2026-07-13

Added

• Four new built-in task batteries — regression, image_quality, audio, and
  retrieval — plus a per-Task update_fn hook for metrics whose update step is
  not (preds, target) (used by retrieval). LPIPS and STOI sit behind the
  optional [image] and [audio] extras. Each battery is exported from mushin.
• Overhauled the documentation: runnable, tested example scripts (MNIST) that the guides embed verbatim, deeper guides with annotated output, and new Tutorial, Core concepts, Custom metrics/predict_fn, and Statistics pages.
• Public task API: Task dataclass plus register_task, get_task, and
  list_tasks make evaluation tasks first-class and reusable. compare(...) and
  Study(...) now accept either a Task object or a registered task name, and the
  built-in batteries (classification_battery, segmentation_battery,
  detection_battery) are exported from mushin.
• compare(task="detection") — compare trained object detectors across seeds over
  the full torchmetrics.detection bounding-box family (mean-average-precision plus
  the IoU/GIoU/CIoU/DIoU variants), reporting every scalar metric with Holm-corrected
  significance. Needs the optional mushin-py[detection] extra.
• mushin.llm.compare_llms — compare LLM systems (callables or hydra-zen configs) across reproducible stochastic seeds with a metric (a plain scorer, a torchmetrics text metric, or the new llm_judge helper), reporting Holm-corrected statistical significance. Includes an on-disk output cache. Provider-agnostic: you bring the systems, data, and judge model.
• tune_batch_size / tune_learning_rate: opt-in, reproducibility-preserving
  auto-tuning. Lightning's batch/LR finder runs once, the result is pinned to a
  sidecar YAML, and later runs reuse it. tune_batch_size pins a hardware-
  independent effective batch, choosing the largest device batch that both fits and
  divides the per-device target exactly, so the effective batch is identical on any
  GPU count with no drift.

Changed

• Modernized the codebase to Python 3.10+ idioms now that 3.9 is no longer
  supported: ruff target-version is py310, and the pyupgrade auto-fixes
  (Optional[X]/Union[X, Y] -> X | None/X | Y) plus explicit zip(..., strict=True)
  have been applied. No behavior change.
• import mushin is now lightweight: the benchmark and llm subsystems load on
  first use instead of at import time, so a bare import no longer pulls the
  battery/eval machinery. Every existing top-level name still resolves. The default
  Hydra config/job name is now mushin_workflow (was rai_workflow), and the new
  mushin.original_cwd() helper anchors relative paths in task() against the
  launch directory rather than Hydra's per-job output directory.

Fixed

• Fixed the docs example scripts surfaced in review: the Study example now trains and evaluates on separate MNIST splits and resolves its checkpoint directory to an absolute path (so it works under Hydra's per-job chdir); the custom-metrics guide no longer implies Study accepts metrics/predict_fn (those are compare-only).
• Hardening from a repo-wide adversarial audit: load_experiment now loads DDP/nested-layout configs (was silently None) and labels each job's own working_dir; the benchmark compare() path masks zero within-group-variance comparisons (no more false-positive significance — now consistent with compare_llms, both via a single compare_methods); MetricsCallback keeps every metric series aligned to the epoch axis (NaN-padding missing metrics, reserving the epoch key); Study labels the seed coordinate with the real seed values and relocates checkpoints across filesystems (shutil.move); multirun overrides are built with Hydra's choice(...) syntax so comma-bearing and single-element values are no longer mis-split; and several legibility/robustness fixes (to_dataset empty-method error, _to_device namedtuple support, '='-bearing override parsing, load_from_dir cache reset, dropped the never-working dict-override path).

Removed

• Dropped support for Python 3.9 (end-of-life October 2025); mushin now requires
  Python >= 3.10. This refreshes the dependency lockfile to patched versions of
  pillow, urllib3, aiohttp, filelock, requests, pytest, and pytorch-lightning,
  clearing the Dependabot security alerts anchored on the old Python-3.9 dependency
  branch. The scipy (>= 1.13) and matplotlib (>= 3.9) floors are raised to their
  first NumPy-2-compatible releases, and the mcp extra no longer needs a Python
  version gate.

Deprecated

• BaseWorkflow and RobustnessCurve are deprecated at the top level and will be
  removed in a future release. Import them from mushin.workflows instead;
  accessing them as mushin.BaseWorkflow / mushin.RobustnessCurve now emits a
  DeprecationWarning.

Misc

• Added a @claude mention bot (GitHub Actions): mention @claude in any issue or PR comment to have Claude answer questions or make changes. PR reviews stay with the Codex connector.