Skivolve 0.4.0

Added

• Added a static, accessible GitHub Pages site with crawlable guides, canonical metadata, social previews, structured data, a sitemap, and robots policy.
• Added a least-privilege, commit-pinned Pages deployment workflow with deterministic site validation before deployment.

Changed

• Renamed the repository, Python distribution, import package, module entry point, console commands, suite identity, release assets, schemas, documentation, and active protocol namespaces to Skivolve.
• Moved the project homepage to https://dhi13man.github.io/skivolve/ and expanded repository metadata for agent skill and instruction evaluation discovery.
• Bumped the package to 0.4.0 because the public import and command surfaces changed.

Migration

• Replace the former harness-evals distribution and harness_evals import with skivolve.
• Replace the former harness-evals command with skivolve, and harness-evals-prepare-holdout with skivolve-prepare-holdout.
• Re-prepare unconsumed holdout plans and do not compare pre-rename artifact evidence as if it used the new canonicalization or source-fingerprint namespaces.

Security

• Advanced source-fingerprint and text/workspace canonicalization namespaces so renamed evidence cannot be confused with pre-rename bytes.
• Kept Pages deployment permissions isolated to the deployment job and pinned every external action to an immutable commit.

Harness Evals 0.3.0

Added

• Added schema v3 with explicit judged and objective-only evaluation modes plus registered or suite-local comparator profile selection.
• Added package-backed comparator profiles, a test-authority plain-language profile, and shared profile-owned semantic contracts for domain-specific judgment without hard-coding evaluator criteria.
• Added schema v4 with explicit per-case bundle source paths and optional contained shared verifier resources.
• Added installed-wheel coverage for external suites whose bundles, cases, and verifier resources use independent layouts.
• Added schema v5 with suite-owned ordered release comparisons and holdout-plan schema v3 with generic per-variant source bindings.
• Added production objective-only holdouts that seal the canonical verifier acceptance policy without constructing a comparator.
• Added schema v6 with reviewed provider adapter IDs, immutable capability declarations, and canonical capability digests.
• Added holdout-plan schema v4 with role-specific generator and comparator authority bindings.
• Added schema v7 with explicit workspace-diff, final-text, and final-JSON artifact contracts.
• Added bounded LF text and RFC 8785 JSON normalization with canonical artifact evidence.
• Added read-only verifier artifact mounts and pristine read-only fixture workspaces for final-output cases.
• Added hash-locked runtime, test, quality, build, and fuzz dependency sets for reproducible CI installs.
• Added bounded pull-request fuzzing and a weekly extended Atheris campaign for final-JSON normalization.
• Added a protected-tag release workflow that rebuilds and inspects distributions, publishes SHA-256 checksums, and records GitHub/Sigstore build provenance.

Changed

• Separated source authority from comparator judgment authority. Judged plans bind profile, release, and certification evidence; objective plans bind verifier-policy evidence.
• Limited holdout-plan schema v2 to the legacy schema-v2 through schema-v4 candidate/original adapter without rewriting historical plan bytes.
• Routed provider construction, scheduling, sandbox selection, billing evidence, and provenance through reviewed adapter capabilities while preserving schema-v2 through schema-v5 compatibility.
• Advanced reviewed provider capabilities to revision 2 for declared final-output capture; unconsumed schema-v4 authority plans from revision 1 must be re-prepared.
• Refreshed the supported Codex app-server runtime contract from 0.144.1 to 0.144.3 without changing the canonical protocol schema.
• Required wheel and sdist package metadata to agree with their packaged harness_evals.__version__ declaration.

Migration

• To adopt explicit evaluation semantics, migrate to schema v3, select evaluation_mode, and either configure a judged comparator_profile with case contracts or remove comparator fields for objective-only diagnostics.
• To adopt independent layouts, migrate to schema v4, declare every case's canonical bundle_source, and set shared_verifier_dir to a contained resource directory or null.
• To adopt generic source authority, migrate an unconsumed suite to schema v5, add holdout.comparison_ids in manifest order, validate the suite, and prepare a new holdout plan.
• To adopt reviewed provider authority, migrate to schema v6, replace legacy provider kinds with registered adapter IDs, validate the suite, and prepare a new schema-v4 holdout plan.
• To adopt declared output artifacts, migrate to schema v7 and add exactly one artifact_contract to every case. Existing schema-v2 through schema-v6 cases retain the workspace-diff compatibility default.
• Re-prepare every unconsumed schema-v4 plan created against provider capability revision 1. Never rewrite a reviewed or consumed plan.

Security

• Added literal Git path handling, bounded source enumeration, Git/worktree snapshot parity, runtime shared-path revalidation, verifier-only source separation, and read-only configured shared mounts.
• Added domain-separated source fingerprints over bundle locators, paths, bytes, executable modes, and context files; equal evaluated arms, source drift, mode confusion, and authority substitution fail before dispatch.
• Bound production provider authority to reviewed capability, configuration, runtime provenance, executable identity, and non-injected provider instances; drift and instance substitution fail before dispatch or release.
• Reject malformed or non-idempotent canonical output before verification or judgment, reject uncalibrated judged artifact kinds before dispatch, and isolate final-output verification from candidate workspace mutations.
• Pin every Python CI dependency by cryptographic hash, validate all release distributions with Twine, and attach signed build provenance to release artifacts.

Harness Evals 0.2.0

Harness Evals 0.2.0 makes the evaluator boundary match its intended scope: reproducible A/B evaluation of user-defined skills and instruction bundles through agent harnesses.

Changed

• Treats the bundled engineering and testing tracks as the first reference corpus, not the product boundary.
• Derives holdout skill counts and aggregate cells from each selected suite instead of two built-in identifiers.
• Uses domain-neutral isolated task instructions so other skill domains are not framed as software-engineering work.
• Documents Claude and Codex as explicitly registered built-in adapters; Codex remains optional and diagnostic.
• Refreshes release locks, package metadata, issue forms, citation metadata, and supported-version policy for 0.2.0.
• Makes test fixture Git operations synchronous to eliminate a Python 3.11 cleanup race.

The bundled blinded comparator remains calibrated for software-change evidence. General comparator profiles, provider capability registration, alternate bundle sources, and installed resource loading are tracked in issue #4 rather than being overclaimed in this release.

Published artifacts come from the successful protected-main CI run for commit 70ddc67bcf52ad1c4262dd7166ce6c28c7ab9e9c.

Harness Evals 0.1.0

Harness Evals 0.1.0 is the first public release of a generic evaluator for software engineering and testing instruction bundles across coding harnesses.

Highlights

• Seventeen calibrated implementation and testing cases with objective, case-specific verifiers and known-good, known-bad, and adversarial variants.
• Isolated provider execution, Git-bound source snapshots, blinded AB/BA comparison, bounded spend accounting, and externally sealed holdout-plan support.
• Neutral engineering and testing reference bundles with no dependency on private skill names or private repository history.
• Installable Python package with harness-evals, python -m harness_evals, and harness-evals-prepare-holdout entry points.
• MIT licensing, contribution and security policies, CodeQL, OpenSSF Scorecard, Dependabot, protected-main rules, and reproducible CI release gates.

Scope

This is an alpha release for expert use on Linux with Python 3.11 or newer, a working systemd --user manager, and unprivileged user/mount namespaces. The public corpus contains train and validation cases, not a private holdout. No live comparator certification is included, and this release makes no claim that one harness or instruction bundle is superior.

See the 0.1.0 changelog for the complete release surface.