Skip to content

SparkProof v0.1.2 — registry snapshot dedupe for miners

Choose a tag to compare

View all tags
@ai-hpc ai-hpc released this 15 Jul 12:16
v0.1.2
7e5d348
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Miner-side accepted-registry snapshot download so novelty checks catch cross-registry duplicates before publish — plus architecture-scoped exact dedupe so the same prompt on Blackwell vs Hopper counts as a fresh row.

generator_version remains 0.3.0 (package release 0.1.2).

Highlights

Registry dedupe prevention ([#25])

  • sparkproof-download-registry-snapshot — download accepted_registry_snapshot.jsonl from gittensor-model-hub/sparkproof-mining
  • sparkproof-publish-dataset --mining-repo — one-step download + verify + release-gate novelty check
  • docs/MINER_GUIDE.md — full dataset-track workflow

Architecture-scoped exact dedupe ([#26])

  • Exact fingerprints keyed as {gpu_architecture}:{hash}
  • Same prompt on Blackwell vs Hopper is a fresh row, not a duplicate
  • Pairs with SparkDistill #133 and canonical mix republish (174→178 rows)

Miner usage

uv sync --extra publish --frozen

sparkproof-publish-dataset --bundle bundles/my-run --repo-id YOU/dataset \
  --release-gate --mining-repo

Full changelog: CHANGELOG.md

Assets 2
Loading