v0.2.0

Added

• The package publishes to PyPI as sic-evalkit on every release tag. The command and the import
  package are still evalkit.
• A curl | sh installer (install.sh) that downloads the executable for your platform.
• A documentation site on GitHub Pages with a FAQ, architecture notes, decision records, and a demo.
• Property-based tests, a test that runs the README quickstart, a benchmark gate in CI, a nightly
  deep test run, and weekly mutation testing.
• Standalone evalkit executables for Linux x64, Linux arm64, macOS on Apple silicon, and
  Windows x64. They bundle Python, so you can run evalkit without installing Python.
• Every GitHub Release carries the executables, the wheel and sdist, and a SHA256SUMS file.
• A multi-arch container image on ghcr.io/superintelligenceco/evalkit, signed with cosign.
  Its working directory holds the starter suite, so docker run works without a mount.
• Releases carry SPDX SBOMs and build provenance attestations for the executables, the wheel, the
  sdist, and the image.
• The release workflow moves a major version tag, such as v0, so workflows can use
  superintelligenceco/evalkit@v0.

Changed

• You release by pushing a vX.Y.Z tag. The release-please pull request flow is gone.

Fixed

• The CodeQL workflow now has the actions: read permission it needs in private repositories,
  and keeps the SARIF as a workflow artifact when code scanning isn't available.

Install

pip install sic-evalkit==0.2.0
curl -fsSL https://raw.githubusercontent.com/superintelligenceco/evalkit/main/install.sh | EVALKIT_VERSION=v0.2.0 sh

Container image

docker run --rm ghcr.io/superintelligenceco/evalkit:0.2.0 run evals.yaml

Verify a download with gh attestation verify <file> -R superintelligenceco/evalkit and the image with
gh attestation verify oci://ghcr.io/superintelligenceco/evalkit:0.2.0 -R superintelligenceco/evalkit.