Repository navigation
Releases: autolab-ai/hills
Release list
v0.12.1
Classify LFS attributes without requiring git >= 2.40: materialize now reads the committed .gitattributes via a throwaway index (read-tree + check-attr --cached) instead of check-attr --source. Fixes spec-4 (git-LFS) evaluations failing on nodes with older git (e.g. 2.25).
v0.12.0 — git-LFS datasets
Hills with datasets: spec_version 4 distributes a hill as an ordinary git repo with large files in git-LFS (bytes in object storage), replacing the bundle + lock mechanism for new hills. Spec 1-3 hills keep working unchanged. hills new --template dataset scaffolds one; materialization verifies every object's sha256+size against its pointer. Scoring identity (git tree hash), report envelope, and evaluator contract are unchanged.
v0.11.0
v0.10.1
v0.10.0
v0.9.0
v0.8.0
v0.7.0
Runtime override (--runtime / HILLS_RUNTIME) and singularity support; container hills run the OCI image directly via any runtime. The prebuilt-SIF experiment (spec 4) is removed — per-cluster runtime/storage is platform policy, not the OSS contract.
v0.5.0: container images for hills
Hills can now declare the container image their evaluator runs in, via an optional environment.image block at spec_version: 3. hills eval and hills check run inside that image using docker, podman, or apptainer. Backward compatible: hills without environment (and every spec 1/2 hill) run in the uv environment exactly as before. See docs/SPEC.md section 3.
v0.4.0
Two stacked changes to hill.yaml. No change to reports' ranking, signing, or verification; existing spec-v1 hills keep working after a one-line edit.
spec_version names the manifest contract (v0.3.0). hill.yaml gets a required integer spec_version. Hills now move between machines (hills bundle) and will be shared on a hub, so a hill must be able to say which contract it was written against, and an older hills must say "this hill needs a newer hills" instead of "unknown key" — the version check runs before the unknown-keys check to guarantee exactly that. Existing hills add spec_version: 1 at the top (the error message is the migration). Reports carry hill_spec_version next to tool; report_version stays 1 because the field is additive.
Spec version 2 declares the metrics. A required metrics key states what the evaluator reports — name and direction, in ranking order — so a hub, a platform, or hills describe can show what a hill measures before the first run, and eval.py cannot silently drift from the README. The declaration is a prefix contract: every passing report's metrics must start with exactly the declared entries; extras may follow as informational tie-breaks. hills eval refuses to sign a violating report, and hills check applies the same rule to example reports through the hill's own tests. Reports stay self-contained: per-metric direction stays in them, and any set of reports still ranks with no configuration beyond the reports themselves.
Templates and the bundled examples are now spec v2, which changes the example hills' tree hashes.