Reproducible 202609 chain, step 6: build vocab_labels from archived, pinned TTLs (--ttl-dir), explicit order, manifest

[rdhyee]

🤖 rbotyee (Claude, operated by @rdhyee) — Raymond's intent: continue Part B (reproducible isamples_202609 build). What I did: archived the 10 vocabulary TTLs at the commits pinned on 8/27 (they are still byte-identical to main), taught build_vocab_labels.py to build from that archive with verification, and ran it through three Codex rounds. No published file changes.

Why

build_vocab_labels.py fetched six-plus TTLs from main of four GitHub repos — a moving target, so vocab_labels_* was not reproducible by construction (plan block C). It also had no explicit row order and wrote no provenance. (It happened to be run-to-run stable: two live runs today matched; rows equal to the live vocab_labels_202608 on all shared columns.)

What changed

• --ttl-dir <archive> --ttl-archive ttl_archive.json (must be given together): every expected TTL is read once from <archive>/<repo>/<file>.ttl, its bytes SHA-256-checked against the archive record (which must carry a 40-hex commit), and that same buffer parsed with publicID=<canonical URL> so base-IRI semantics match a live fetch. Missing/altered/extra/duplicate records are fatal even with --allow-partial. source_ttl still records the canonical main URL.
• Deterministic selectors (definition, scheme, prefLabel, dedupe tiebreak); rows sorted by (uri_form, uri, lang, source_ttl), alt_labels sorted; surviving (uri, lang) duplicates are fatal.
• {output}.manifest.json: per-TTL sha256 + commit + row count, failures, ttl_inputs_pinned / archive_verified / complete, script-defined rows (manual overrides) reported separately, script SHA-256, git SHA + dirty flag, rdflib/pandas/pyarrow versions, output SHA-256.

Evidence

Two archived runs → 65bbcdb6… both; equal to a live run the same minute; rows identical to the pre-change output. Downstream step 5 (build_frontend_derived.py) requires this file's broader column — the live vocab_labels_202608 lacks it — so the chain order is 6 → 5 (recorded in the plan). The archive (10 files, SHA256SUMS, ttl_archive.json) lives with the 202609 inputs; hashes are in provenance/inputs_202609_frozen.json (coordination repo).

Review

Codex ×3: R1 changes-required (P1 "pinned" without archive verification; P1 partial inputs misrepresented; P2 publicID; P2 nondeterministic selectors; P2 overclaim) → R2 changes-required (P1 hash≠parsed bytes; P2 commit not enforced; P2 pinned naming; P2 wording) → R3 LGTM-with-nits (applied). Verdict posted below.

🤖 Generated with Claude Code

https://claude.ai/code/session_01LtTxB4jfTZgaTR7CK4zKqy

[rdhyee]

🤖 rbotyee — Codex round 3 (final) verdict, verbatim:

LGTM-with-nits

Verdict: LGTM-with-nits. No remaining P1/P2 findings.

1. Nit — require strings before regex validation. At build_vocab_labels.py:389, str(...) allows a 40-digit JSON number to pass as a commit and remain numeric in the emitted manifest. Prefer isinstance(value, str) and re.fullmatch(...). This does not admit unverified TTL bytes; a numeric SHA cannot equal the computed string digest.

2. Nit — reproducibility wording slightly understates pandas’ role. Pandas performs the final logical row sorting at build_vocab_labels.py:491, while the policy at build_vocab_labels.py:533 associates pandas only with Parquet bytes. The pandas version is recorded, so this is wording rather than missing provenance.

On the central question:

• ttl_inputs_pinned: true cannot occur through the normal execution path with unverified TTL bytes contributing rows. Each file is buffered, hashed, checked, and that same buffer is parsed at build_vocab_labels.py:411 and build_vocab_labels.py:427. Successful entries are counted only after parsing, and the predicate requires all expected inputs with no failures at build_vocab_labels.py:536.
• complete: true alone can intentionally occur for successful live, unpinned fetches at build_vocab_labels.py:432. In that case ttl_inputs_pinned is false. Both cannot be true with unverified TTL input.
• Manual overrides are script-defined rather than TTL-derived and are separately identified at build_vocab_labels.py:539.

Static/import and in-memory exact-buffer parsing checks passed. The full pytest run was unavailable because the review environment provides no writable temporary directory; I also could not independently rebuild the stated output hash without the TTL archive.

[rdhyee]

🤖 rbotyee — staging check before merge: fork Pages run 33205136357 on this branch built green; explorer-smoke 4 passed (16.3s) against https://rdhyee.github.io/isamplesorg.github.io. Merged under RY's standing rule ("keep going as long as you get CC+Codex+LGTM, deployment to rdhyee looks good").