Skip to content

Bind stores and receipts to sources and harden cohort execution - #11

Merged
CraigMyles merged 1 commit into
mainfrom
hardening/store-receipt-cohort
Jul 13, 2026
Merged

Bind stores and receipts to sources and harden cohort execution#11
CraigMyles merged 1 commit into
mainfrom
hardening/store-receipt-cohort

Conversation

@CraigMyles

Copy link
Copy Markdown
Owner

This keeps v0.1 local output names stable, while preventing a same-named slide from silently reusing or appending to a store belonging to another source.

Changes

  • Bind receipt skips to the slide ID, credential-free canonical source, expected output target, and live store provenance.
  • Check the root and every grid before non-force store reuse or append, failing clearly on missing, malformed, inconsistent, or mismatched source provenance.
  • Make --force bypass receipt fast paths and deliberately rebuild the target.
  • Reject duplicate derived output IDs across the complete embed-many cohort before sorting, sharding, or loading models, including repeated identical manifest rows.
  • Propagate parallel worker model-load failures as a non-zero command exit without counting worker failures as failed slides.
  • Replace receipts atomically, treat malformed JSON as incomplete, preserve existing file permissions, and avoid overlong temporary filenames.
  • Add target-aware verify --out-dir handling and update the shipped SLURM array accordingly.

Compatibility notes

  • Ordinary local slide IDs remain basename-based, so existing v0.1 output names do not change.
  • The three-argument is_complete(...) call remains valid but now fails closed when no current source is supplied.
  • Per-output locking across independent processes remains deferred; the cohort preflight prevents duplicate writers within one discovered cohort.

Validation

On Spark:

  • uv run ruff check .
  • Focused regression suite: 116 passed
  • Full non-slow, non-network suite: 407 passed, 10 skipped, 87 deselected

Refs #6

@CraigMyles
CraigMyles marked this pull request as ready for review July 13, 2026 16:44
@CraigMyles
CraigMyles merged commit adb6c34 into main Jul 13, 2026
5 checks passed
@CraigMyles
CraigMyles deleted the hardening/store-receipt-cohort branch July 13, 2026 16:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant