Skip to content

Releases: autolab-ai/hills

v0.12.1

Choose a tag to compare

@ottogin ottogin released this 30 Sep 06:16

Classify LFS attributes without requiring git >= 2.40: materialize now reads the committed .gitattributes via a throwaway index (read-tree + check-attr --cached) instead of check-attr --source. Fixes spec-4 (git-LFS) evaluations failing on nodes with older git (e.g. 2.25).

v0.12.0 — git-LFS datasets

Choose a tag to compare

@ottogin ottogin released this 28 Sep 19:58

Hills with datasets: spec_version 4 distributes a hill as an ordinary git repo with large files in git-LFS (bytes in object storage), replacing the bundle + lock mechanism for new hills. Spec 1-3 hills keep working unchanged. hills new --template dataset scaffolds one; materialization verifies every object's sha256+size against its pointer. Scoring identity (git tree hash), report envelope, and evaluator contract are unchanged.

v0.11.0

Choose a tag to compare

@ottogin ottogin released this 11 Sep 05:19

HILLS_SHIM_LAUNCHER: run the stdlib shim via a caller-provided launcher (the k8s shim-adapter — hills stays on the launcher, only the shim enters the pod).

v0.10.1

Choose a tag to compare

@ottogin ottogin released this 11 Sep 02:40

terminate_group waits for the whole process group (grace period) before SIGKILL, not just the leader.

v0.10.0

Choose a tag to compare

@ottogin ottogin released this 11 Sep 02:33

terminate_group now kills the whole process group (SIGTERM, grace, SIGKILL survivors) regardless of leader exit, so a timed-out evaluator can't leave orphans.

v0.9.0

Choose a tag to compare

@ottogin ottogin released this 10 Sep 09:22

Host mode (runtime=host) runs the evaluator under the image's own Python, not the ephemeral interpreter a launcher prepends to PATH, so image hills with Python deps work. Honors HILLS_HOST_PYTHON.

v0.8.0

Choose a tag to compare

@ottogin ottogin released this 10 Sep 08:11

Adds --runtime host / HILLS_RUNTIME=host: run an image hill's evaluator directly in the current environment (no container), for pods booted from the hill's own image — so image hills work where nested containers can't (unprivileged pods).

v0.7.0

Choose a tag to compare

@ottogin ottogin released this 10 Sep 07:59

Runtime override (--runtime / HILLS_RUNTIME) and singularity support; container hills run the OCI image directly via any runtime. The prebuilt-SIF experiment (spec 4) is removed — per-cluster runtime/storage is platform policy, not the OSS contract.

v0.5.0: container images for hills

Choose a tag to compare

@ottogin ottogin released this 09 Sep 18:04

Hills can now declare the container image their evaluator runs in, via an optional environment.image block at spec_version: 3. hills eval and hills check run inside that image using docker, podman, or apptainer. Backward compatible: hills without environment (and every spec 1/2 hill) run in the uv environment exactly as before. See docs/SPEC.md section 3.

v0.4.0

Choose a tag to compare

@ottogin ottogin released this 28 Aug 02:42

Two stacked changes to hill.yaml. No change to reports' ranking, signing, or verification; existing spec-v1 hills keep working after a one-line edit.

spec_version names the manifest contract (v0.3.0). hill.yaml gets a required integer spec_version. Hills now move between machines (hills bundle) and will be shared on a hub, so a hill must be able to say which contract it was written against, and an older hills must say "this hill needs a newer hills" instead of "unknown key" — the version check runs before the unknown-keys check to guarantee exactly that. Existing hills add spec_version: 1 at the top (the error message is the migration). Reports carry hill_spec_version next to tool; report_version stays 1 because the field is additive.

Spec version 2 declares the metrics. A required metrics key states what the evaluator reports — name and direction, in ranking order — so a hub, a platform, or hills describe can show what a hill measures before the first run, and eval.py cannot silently drift from the README. The declaration is a prefix contract: every passing report's metrics must start with exactly the declared entries; extras may follow as informational tie-breaks. hills eval refuses to sign a violating report, and hills check applies the same rule to example reports through the hill's own tests. Reports stay self-contained: per-metric direction stays in them, and any set of reports still ranks with no configuration beyond the reports themselves.

Templates and the bundled examples are now spec v2, which changes the example hills' tree hashes.