v1.0.0: a Wikipedia model you can recompute bit for bit

OpenVerifiableLLM v1.0.0: a Wikipedia model you can recompute bit for bit

This is the first release of OpenVerifiableLLM. It's a 23.1M-parameter GPT trained from a seeded random start on
exactly one pass over the complete eligible English Wikipedia (2026-09-01 dump), followed by a short
conversation phase. Every one of the 37 training segments has been recomputed and matched bit for bit. Anyone
with the same GPU type can recompute any segment and confirm that the published weights are exactly what this
training on these inputs produces.

What is verified is the computation, not the answers. The model is small: it writes fluent encyclopedia-style
text and is often wrong about facts.

Downloads

The model, the training record, every checkpoint and the exact code used for the run are on Hugging Face, pinned
to revision 96dc282a:

Folder	Contents
release/simple-run/	conversation-model.pt (the chat model), base-model.pt (after the Wikipedia pass), all 38 checkpoints, chain.jsonl, recipe.json, anchor.json with two RFC 3161 timestamps, release-hashes.txt, and the exact code in code/
release/unlearning-3i-atlas/	The unlearning fork that removed one article, with its own chain and checkpoints
prepared/8de1d600…	The prepared token streams and tokenizer, rebuildable byte for byte from the raw dump

What's in this release

The model

Architecture	GPT, 23.1M parameters, 512-token context
Training data	7,232,582 Wikipedia articles (7,118,121,139 training targets), each covered exactly once; then 3,481 OpenAssistant conversations (1,278,137 targets)
Training	560,177 Wikipedia updates plus 441 conversation updates, batch 32 × 512 tokens, one RTX 5090, 15.7 hours
Numerics	BF16 autocast with FP32 master weights, deterministic CUDA kernels, no TF32
Software	PyTorch 2.8.0, CUDA 12.8, image runpod/pytorch:1.0.2-cu1281-torch280-ubuntu2404

The verifiable training loop: simple/

• simple/ovl.py, one file: train, verify (--weights, --segment, --random, --all), fork, chat, bench.
• A hash-linked training record. chain.jsonl has 38 entries, one per boundary (every 18,080 updates, plus the
  phase boundaries). Each entry commits to the training position and a SHA-256 over the full model and optimizer
  state, and links to the previous entry. Every boundary checkpoint is published.
• Exact coverage checks. Every training step checks that its targets continue where the previous step stopped,
  and each phase must end with every target covered once.
• Segmented verification. Segment k is checked by loading checkpoint k, recomputing to boundary k+1 and
  comparing the full state digest. Segments are independent, so many people can check them in any order, at the
  same time.
• Timestamps. simple/stamp.py sends each chain entry to two RFC 3161 authorities, FreeTSA and DigiCert. The
  release anchor was stamped at 2026-10-20 14:38:43 UTC.
• A one-click checker. The public RunPod template
  "OpenVerifiableLLM segment check" recomputes a random
  segment on a fresh RTX 5090 and prints one result line (about an hour, about US$1–1.50).
• A data rebuild. simple/rebuild_prep.sh rebuilds the prepared token streams from the raw Wikipedia dump and
  compares them with the published preparation.json.
• Verifiable unlearning. ovl.py fork removes documents the run hadn't reached at the fork point, and records
  the removed range in the fork's chain. simple/loo_eval.py measures the effect.
• Tools: bench_sizes.py (cost of bigger models), hwmap_check.sh (does another GPU reproduce the bits?),
  review_ckpt.py (checkpoint review before publication), follow.sh (mirror and check segments while training
  runs).

The ovllm verify artifact verifier

Hardening since the earlier pipeline:

• verify fails when no checks ran (#207);
• the chain head is checked against the timestamped anchor.json, so a cut-short record fails (#210);
• an unpinned Sigstore signature can't give GREEN (#204);
• a manifest with no expected hashes fails instead of passing on skipped checks (#218);
• verify --weights says when weights match a chain entry that isn't the released one (#213).

Results

Measurement	Result
Segments recomputed on a second RTX 5090	37 / 37 MATCH
Recomputed from public files only, on fresh machines	segments 8 and 21 MATCH
Starting weights regenerated from the seed	MATCH on every device tested, including CPU
Rebuild of the training data from the raw dump	byte-identical preparation.json (sha256 8de1d600…ba68)
Unlearning fork (removed "3I/ATLAS", 10,395 targets)	every fork segment MATCH; loss on the removed text +0.0256, on 50 control articles +0.0002

The method held through three unplanned events: a host failure mid-run (training resumed from checkpoint 10, and
the segment spanning the restart matched), a mid-run change to the CPU-side data loader, and a GPU driver change
(570.195.03 → 595.91.07). All of these produced identical bits.

Check it yourself

No GPU needed for the basic checks:

pip install huggingface_hub torch numpy rfc8785==0.1.4 tokenizers==0.23.2 mwparserfromhell==0.7.2 "safetensors>=0.4.5,<1.0" "pynacl>=1.5,<2.0"
# download release/simple-run/ (without the ckpt-*.pt files) and the tokenizer, then:
sha256sum -c --ignore-missing release-hashes.txt
OVL_DEVICE=cpu python code/simple/ovl.py verify --run . --weights conversation-model.pt
# weights conversation-model.pt: MATCH chain entry 37

To recompute a training segment, use the one-click RunPod check,
or follow Check a training segment.
Every independent check makes the release more credible, so please post your result line in an issue.

Cost

About US$55–60 of rented compute in total: training about US$19, checking all 37 segments about US$12–19, and
rebuilding the data about US$13, plus extra GPU checks and idle pods. A clean repeat (training plus checking every
segment) would cost about US$21. The cost section
has the breakdown and projections for 42M to 110M models.

Known limitations

• Same GPU architecture needed. Bit-exactness holds within one GPU architecture, regardless of driver. RTX 4090,
  H100 and CPU recomputations of the same segment don't match, so checkers need an RTX 5090.
• No independent checker yet. So far every segment check was run by the project, on separate machines.
• The timestamp shows when, not how. The release anchor proves the record existed by 2026-10-20 14:38:43 UTC,
  not when training happened. Runs started after 21 October stamp every boundary as it's written.
• Unlearning is cheap only for unseen documents. A document the run had already seen at the fork point needs
  retraining from before its first appearance.
• A small model. It gives frequent factual errors and only basic conversation.

Repository changes

• simple-production-loop merged into main (#217). The earlier production pipeline (src/ovl_pipeline/) is
  frozen and kept for reference; the simple run still uses its data preparation.
• New docs layout: design
  (method, results, cost, timeline, things to avoid, lessons), project (goals, roadmap, FAQ) and guides (#219, #220, #221).
• CPU-only tests for simple/ovl.py (#208); about 3,250 tests run in CI.
• AOSSIE project files: MAINTAINERS.md, brand/, BestPracticesChecklist.md, a CodeRabbit config and an OpenSSF
  Scorecard workflow (#224).
• 32 pull requests against the frozen pipeline closed as outdated.

Security

No known vulnerabilities in the project's own code. The verifier now fails closed in four more cases (#204, #207,
#210, #218). Report security issues privately, as described in AOSSIE's
security policy.

Thanks

Thanks to the contributors whose fixes are in this release: @AnKiT-GaRG2 (#213), @Haseebx162006 (#204),
@AdityasagarR123 (#207, #218) and @panshularora (#208, #210). Thanks also to our mentor @Archit381, and to AOSSIE.

Full changelog: https://github.com/AOSSIE-Org/OpenVerifiableLLM/commits/v1.0.0