Skip to content

SparkProof v0.3.0 — dataset quality, DPO signal & external seeds

Latest

Choose a tag to compare

@ai-hpc ai-hpc released this 26 Jul 09:52
3104e28

Dataset-quality and training-signal release. generator_version stays 0.3.0 (the new preferences_sha256 manifest field is optional/additive; existing bundles verify unchanged).

Dataset quality — closes the validator-hack pattern found in the merged data

A review of the live sparkproof-mining set surfaced repairs that "passed" by deleting a hardware/SM assertion the model had added (a kernel prompted "Hopper SM90" that then failed on the real Blackwell SM120). Fixed end-to-end:

  • Prevent — generation + repair prompts steer teachers to portable code with no device/SM/architecture asserts and to fix root cause, not delete checks.
  • Root causesparkproof-triton-generate retargets each prompt to the actual validating GPU, so the label↔hardware mismatch can't arise.
  • Detect — a non-blocking repair_guard_removed_rows count in dataset_manifest.json surfaces any guard-removal repair that still slips through.

Training signal

  • DPO correctness preference pairschosen = a hardware-verified kernel, rejected = a validated-failing one, grounded in GPU validation. Now written to the bundle as preferences.jsonl, sha256-pinned in the manifest, secret-scanned, and shipped under proof/ so a validator re-checks them tamper-evidently. sparkproof-export-dpo --pair-type {optimization,correctness,both}.
  • Diff-style repair exportsparkproof-publish-dataset --repair-style diff renders repair turns as unified diffs against the prior attempt (keeps the <think> rationale), cutting repair-turn tokens (~23% on current data, more for small-diff repairs). Export-view only; attested code is unchanged.

Broader corpus

  • External task seedssparkproof-import-external-tasks imports KernelBook / opus / gpt-oss as PyTorch tasks only (origin kernelbook_seed), license-gated (fail-closed), KernelBench blocked + fingerprinted for decontamination, and re-proved from scratch on attested hardware. Never copies external CoT/kernels.

Compatibility

Additive only — the manifest gains an optional, attested preferences_sha256; bundles without it verify exactly as before. No stable_keys change, no generator_version bump. 385 tests pass. Full detail in CHANGELOG.md.

Not in this release

The Kimi K3 teacher swap (PR #50) is held pending a coordinated SparkDistill teacher/providers.py update; the pinned pair remains Fable 5 + GPT 5.6 Sol.