Reproducibility and Release Integrity

Reproducibility and Release Integrity

Two supported review modes

Portable reviewer path

Use a fresh Python 3.10+ environment:

python -m venv .venv
. .venv/bin/activate          # Linux/macOS
# .venv\Scripts\Activate.ps1  # Windows PowerShell
python -m pip install --upgrade pip
python -m pip install -r requirements.txt
python -m pip install -e . --no-deps

This path uses normal build isolation and is intended to be practical across supported reviewer environments.

Hash-locked reference path

For the documented CPython 3.13 / Linux x86_64 reference environment:

python -m pip install --require-hashes \
  -r requirements-lock-py313-linux-x86_64.txt
python -m pip install --no-build-isolation -e . --no-deps
make qa-full

The lock includes the required build tooling, so --no-build-isolation is appropriate in this specific path.

Current release tuple

The canonical v2.2.6-public-eval release contains:

permit-receipt-ref-eval-v2_2_6-public-eval.zip
permit-receipt-ref-eval-v2_2_6-public-eval.zip.sha256
permit-receipt-ref-eval-v2_2_6-public-eval.zip.manifest.json
permit-receipt-ref-eval-v2_2_6-public-eval.zip.provenance.json

Treat these four files as one tuple. If any byte changes, publish a new tag and new sidecars rather than replacing assets in place.

Verify downloaded assets

From a checkout containing the verification tool and the four downloaded files:

python tools/verify_release_artifact.py \
  permit-receipt-ref-eval-v2_2_6-public-eval.zip \
  permit-receipt-ref-eval-v2_2_6-public-eval.zip.sha256 \
  --manifest permit-receipt-ref-eval-v2_2_6-public-eval.zip.manifest.json \
  --provenance permit-receipt-ref-eval-v2_2_6-public-eval.zip.provenance.json

Also record the source tag and release page used for the download.

Verify the source tree

python verify_manifest.py
python tools/generate_supply_chain_metadata.py
python tools/make_public_manifest.py
git diff --exit-code -- attestations sbom.cdx.json MANIFEST.json MANIFEST.sha256.json

A non-empty diff means generated supply-chain or manifest material is not current for the checkout.

Rebuild twice and compare exact bytes

rm -rf /tmp/permit-a /tmp/permit-b
python tools/build_release_asset.py --out-dir /tmp/permit-a
python tools/build_release_asset.py --out-dir /tmp/permit-b

for suffix in \
  .zip \
  .zip.sha256 \
  .zip.manifest.json \
  .zip.provenance.json; do
  cmp "/tmp/permit-a/permit-receipt-ref-eval-v2_2_6-public-eval${suffix}" \
      "/tmp/permit-b/permit-receipt-ref-eval-v2_2_6-public-eval${suffix}"
done

Run this in the documented locked environment. Cross-platform ZIP creation may differ even when source behavior is equivalent, so record the environment and distinguish byte reproducibility from functional portability.

Reproducibility questions

• Is the environment fully identified and dependency resolution controlled?
• Are source, generated metadata, release notes, and version taxonomy aligned?
• Can a reviewer reproduce canonical bytes and digests independently of the main verifier?
• Can signatures be verified independently?
• Are all released files bound into the manifest and provenance?
• Are release assets immutable after checksum publication?
• Is every deviation or erratum linked from the release page?

Current erratum

The immutable v2.2.6 reviewer documentation used --no-build-isolation in a portable fresh environment without first installing the pinned build backend. The corrected portable command is python -m pip install -e . --no-deps; reserve the no-isolation command for the locked path or explicitly preinstall the pinned build requirements.

See Current Status and Errata.