SparkProof v0.3.0 — dataset quality, DPO signal & external seeds

Dataset-quality and training-signal release. generator_version stays 0.3.0 (the new preferences_sha256 manifest field is optional/additive; existing bundles verify unchanged).

Dataset quality — closes the validator-hack pattern found in the merged data

A review of the live sparkproof-mining set surfaced repairs that "passed" by deleting a hardware/SM assertion the model had added (a kernel prompted "Hopper SM90" that then failed on the real Blackwell SM120). Fixed end-to-end:

• Prevent — generation + repair prompts steer teachers to portable code with no device/SM/architecture asserts and to fix root cause, not delete checks.
• Root cause — sparkproof-triton-generate retargets each prompt to the actual validating GPU, so the label↔hardware mismatch can't arise.
• Detect — a non-blocking repair_guard_removed_rows count in dataset_manifest.json surfaces any guard-removal repair that still slips through.

Training signal

• DPO correctness preference pairs — chosen = a hardware-verified kernel, rejected = a validated-failing one, grounded in GPU validation. Now written to the bundle as preferences.jsonl, sha256-pinned in the manifest, secret-scanned, and shipped under proof/ so a validator re-checks them tamper-evidently. sparkproof-export-dpo --pair-type {optimization,correctness,both}.
• Diff-style repair export — sparkproof-publish-dataset --repair-style diff renders repair turns as unified diffs against the prior attempt (keeps the <think> rationale), cutting repair-turn tokens (~23% on current data, more for small-diff repairs). Export-view only; attested code is unchanged.

Broader corpus

• External task seeds — sparkproof-import-external-tasks imports KernelBook / opus / gpt-oss as PyTorch tasks only (origin kernelbook_seed), license-gated (fail-closed), KernelBench blocked + fingerprinted for decontamination, and re-proved from scratch on attested hardware. Never copies external CoT/kernels.

Compatibility

Additive only — the manifest gains an optional, attested preferences_sha256; bundles without it verify exactly as before. No stable_keys change, no generator_version bump. 385 tests pass. Full detail in CHANGELOG.md.

Not in this release

The Kimi K3 teacher swap (PR #50) is held pending a coordinated SparkDistill teacher/providers.py update; the pinned pair remains Fable 5 + GPT 5.6 Sol.

SparkProof v0.2.0 — mandatory attestation & audit security hardening

Security-hardening release from a whole-project audit (8 merged PRs, #40–#48). generator_version stays 0.3.0 (bundle format unchanged).

⚠️ Breaking — publish/release path

run_release_gate / sparkproof-publish-dataset now fail closed unless strict cryptographic attestation passes. Before publishing on v0.2.0:

• install dcap-qvl, and
• provision the SparkProof-enclave MRTD allow-list (SPARKPROOF_TDX_MRTD_ALLOWLIST env or policies/tdx_mrtd_allowlist.json).

Until then the gate rejects all bundles by design. The cheap offline sparkproof-verify / verify_bundle are unchanged — CPU-only verification still works.

Security

• Mandatory attestation for production acceptance (SEC-1/2/3, SEC-TDX-1): strict gate requires an NVIDIA NRAS signature and an Intel-TDX-DCAP-verified quote bound to an allow-listed MRTD.
• Verifier fails closed on malformed/adversarial bundles; row-count inflation (CORR-3) and absent NRAS measurement (SEC-8) rejected in strict mode.
• Secret isolation: untrusted teacher/miner kernel code runs with os.environ scrubbed of API keys / wallet material (SEC-4).
• Anti-cheat catches getattr / functools.partial / local-alias torch-fallback indirection (SEC-6).
• request_sha256 relabeled integrity-only, not proof-of-call (SEC-5).

Robustness & correctness

• Resume no longer drops transient gateway_error tasks; retryable HTTP is transient, not fatal (CORR-1/2).
• Winner checkpointed before the completion marker; self-evolution survives a malformed parent kernel (PIPE-1/2).
• Consistent Blackwell pass-rate floor (CORR-5); optimize_request_sha256 labeled by the actual winner (CORR-6).

Code quality

• One canonical fenced-code extractor, shared teacher-leaf payload, extracted optimize pass — all behind byte-identical golden tests (QUAL-1/3/5).

Docs

• docs/PIPELINE_REDESIGN.md (incremental datagen-factory redesign), docs/SECURITY_FINDINGS.md (findings + status).

354 tests pass, ruff clean. Full detail in CHANGELOG.md.

Known follow-ups (not in this release)

SEC-7 (timing-nonce restructure — needs a GPU node), SEC-6 nn-module instantiate-then-call, and the PIPE-3/4/5 concurrency/budget/resumable-prove phases.

SparkProof v0.1.3 — Sol→Fable CoT recovery

Highlights

• Sol→Fable training-CoT recovery (#31): when GPT 5.6 Sol wins best-of-N with encrypted/empty reasoning, Claude Fable 5 re-solves (hinted by Sol’s verified kernel) or explains Sol’s gold answer so SFT gets plaintext <think> traces.
• Repair-tier novelty fingerprint (#29/#30): fingerprints the mining task prompt, not the self-repair wrapper.
• Package version 0.1.3 (generator_version remains 0.3.0).

Install

uv sync
# or pin: git+https://github.com/gittensor-model-hub/SparkProof.git@v0.1.3

Full notes: see CHANGELOG.

SparkProof v0.1.2 — registry snapshot dedupe for miners

Miner-side accepted-registry snapshot download so novelty checks catch cross-registry duplicates before publish — plus architecture-scoped exact dedupe so the same prompt on Blackwell vs Hopper counts as a fresh row.

generator_version remains 0.3.0 (package release 0.1.2).

Highlights

Registry dedupe prevention ([#25])

• sparkproof-download-registry-snapshot — download accepted_registry_snapshot.jsonl from gittensor-model-hub/sparkproof-mining
• sparkproof-publish-dataset --mining-repo — one-step download + verify + release-gate novelty check
• docs/MINER_GUIDE.md — full dataset-track workflow

Architecture-scoped exact dedupe ([#26])

• Exact fingerprints keyed as {gpu_architecture}:{hash}
• Same prompt on Blackwell vs Hopper is a fresh row, not a duplicate
• Pairs with SparkDistill #133 and canonical mix republish (174→178 rows)

Miner usage

uv sync --extra publish --frozen

sparkproof-publish-dataset --bundle bundles/my-run --repo-id YOU/dataset \
  --release-gate --mining-repo

Full changelog: CHANGELOG.md

SparkProof v0.1.1 — Hopper, per-device NRAS, Intel TDX

Hopper joins Blackwell for dataset generation, per-device NRAS tokens are cryptographically verified, and Intel TDX closes the userland trust gap on the dataset track.

generator_version in new bundles remains 0.3.0 (package release 0.1.1).

Highlights

Hopper H100/H200 dataset generation ([#20])

• sparkproof/gpu/architecture.py accepts Blackwell or Hopper H100/H200 (SM90, memory-size split).
• Prompt text, decontamination fingerprints, and dataset_manifest.json are architecture-aware — Hopper rows genuinely target SM90, not Blackwell SM12x.

Per-device NRAS verification ([#18])

• Every GPU device JWT in REMOTE_GPU_CLAIMS is JWKS-verified; signed hwmodel claims exposed under claims["devices"].
• Pairs with SparkDistill's dataset gate running sparkproof-verify --online.

Intel TDX measured-VM attestation ([#22])

• TDX quote captured via configfs-tsm, report_data bound to dataset nonce.
• Production verify requires gpu_attestation.tdx on new bundles; --online DCAP-verifies via dcap-qvl.
• Legacy bundles without a tdx key are grandfathered.

OpenRouter + legacy fixes ([#19], [#21])

• Trajectories record OpenRouter's actual response model; dated build suffixes accepted in policy/ledger checks.
• Pre-Hopper bundles missing gpu_architecture fall back to "blackwell".

CI

check	result
pytest	pass (PR #24)

Miner setup (TDX guest, once per boot)

sudo chmod 0777 /sys/kernel/config/tsm/report
mkdir /sys/kernel/config/tsm/report/sparkproof
sudo chmod 0666 /sys/kernel/config/tsm/report/sparkproof/inblob
export SPARKPROOF_TSM_REPORT_PATH=/sys/kernel/config/tsm/report/sparkproof

Full details in CHANGELOG.md.

SparkProof v0.1.0

First public release of SparkProof — Blackwell GPU-attested Triton dataset generation for SparkDistill.

Highlights

• sparkproof-2 bundles with GPU CC attestation (NRAS), Merkle root over verified samples, and CPU-verifiable manifests
• Five training sources (A–E): API docs, mutations, Torch→Triton translation, self-evolution, failure-mining
• TritonBench eval quarantine — decontamination blocks eval-origin prompts/AST structures from training data
• Release gate before HF publish (provenance + decontamination + attestation)
• Identity-free diverse sampling (run-seed, stratified prompt selection, novelty gate)
• Reference timing diagnostics (isolated PyTorch baseline; candidate timing is diagnostic-only, not ranking credit)
• HF publish uploads dataset rows + proof/ artifacts for validator re-verification
• Gateways: OpenRouter and yunwu with pinned Fable 5 + GPT 5.6 Sol at xhigh

CI test results (2026-07-11)

check	result
pytest	238 passed, 9 skipped
ruff	All checks passed
Python	3.12.3
pytest	9.0.3
Commit	eeaaa56

Skipped (GPU-gated — require Blackwell + SPARKPROOF_RUN_GPU_TESTS=1)

• tests/test_reference_bench_gpu.py — 3 tests (Sigmoid, Matmul, LayerNorm live reference bench)
• tests/test_reference_kernels_gpu.py — 6 tests (reference kernel integration)

Production validation — Targon SN4 · RTX PRO 6000 Blackwell CC

All production runs below executed on a NVIDIA RTX PRO 6000 Blackwell Server Edition confidential-computing VM provisioned through Targon (Bittensor Subnet 4 / SN4).

environment	value
GPU	NVIDIA RTX PRO 6000 Blackwell Server Edition (SM 12.0, 96 GB)
Driver	595.71.05
Triton	3.7.1
SparkProof generator	0.3.0
Manifest	sparkproof-2
CC attestation	NRAS (passed: true, nonce_verified: true)

Smoke test — gittensor-model-hub/sparkproof-triton-v0

check	result
Verified rows	2
Release gate	passed
GPU attestation	present under proof/
SparkDistill eval.dataset_verify	verified=true

Dataset: gittensor-model-hub/sparkproof-triton-v0

Miner production run — miner-25-20260711 (speedy00)

End-to-end pipeline on the CC VM: generate → compile/execute → attest → release gate → publish → CPU re-verify.

check	result
Bundle	bundles/miner-25-20260711
Prompts processed	26 (proved: 25/26)
Verified rows published	25 (22 silver, 3 repair)
Novel verified rows	17 (8 near-duplicate task IDs deduped at aggregation)
Gateway	yunwu (claude-fable-5, gpt-5.6-sol, xhigh)
Merkle root	8513d35bf173f30439197a438f9906d5eddc11bc8069890edf2503951e4ebc6f
trajectories_sha256	d96cbe4d43ad5e44bdee45633b41b330cb44f90d7aba35ee730d83d4639a80e9
sparkproof-verify (offline)	verified=true, gpu_attested=true, issues=[]
HF publish	speedy00/sparkproof-miner-25-v1

Resume note: the 25th row was added with --resume on the same run_seed, so only one new prompt was regenerated after the initial 24-row run.

SparkDistill registry gate (merged)

Registry PR #22 merged with label dataset:xs.

CI re-verified the HF proof/ bundle, then aggregated into the canonical mining dataset:

check	result
Registry entry	speedy00 → speedy00/sparkproof-miner-25-v1
Canonical mining dataset	gittensor-model-hub/sparkproof-mining
Rows after cross-miner dedupe	17 (mix_manifest.json provenance retained)
Merge blocked unless publish succeeds	yes

First finetune pilot on canonical mining data (same CC VM)

Pilot Axolotl LoRA run on the published mining mix (not a frontier submission — sanity check that verified HF data trains end-to-end):

check	result
Student	Qwen/Qwen3.5-4B (LoRA r=64, bf16)
Training rows	17 (messages format from sparkproof-mining)
Recipe	recipes/qwen3.5-4b-phase1/sft-mining.yaml
Trainable params	84.9M / 4.62B (1.84%)
Initial eval (epoch 0)	eval_loss=0.7767, eval_ppl=2.174
VRAM	~12.1 GiB active

This pilot was running at release time; full 5-epoch checkpoint output lands under SparkDistill/outputs/qwen3.5-4b-mining/.

Quick start

git clone https://github.com/gittensor-model-hub/SparkProof.git
cd SparkProof && cp .env.example .env
scripts/install.sh
scripts/run_triton_pipeline.sh --limit 25 --release-gate --publish <user>/sparkproof-triton-v1

Requirements

• Python >= 3.12
• NVIDIA RTX PRO 6000 Blackwell CC VM for production proving (Targon SN4 or equivalent)
• Sibling SparkDistill repo (for decontamination corpus, registry gate, and SFT formatting)