Skip to content

v1.1.0 — flagship checkpoints + matched token-LM baseline

Latest
Latest

Choose a tag to compare

View all tags
@nochinator nochinator released this 19 Jul 22:00
v1.1.0
87b4fc0

Flagship checkpoints for the ThoughtVectors paper (see paper/main.pdf in the repo), plus the paper's matched token-LM baseline (§6.5).

Asset What it is Put it at
FINAL_12H-best.pt Thinker flagship — coherent latent-space small talk (val_cos 0.428) checkpoints/FINAL_12H/best.pt
FINAL2_12H-best.pt Register case-study run (+EmpatheticDialogues +reversal splices) checkpoints/FINAL2_12H/best.pt
m5_frontier-best.pt Codec — byte-perfect 4:1 text compression, prefix-decodable checkpoints/m5_frontier/best.pt
b3_lm_48m_24h-best.pt Matched token-LM baseline — 50.1M decoder-only LM, same data/tokenizer/compute (paper §6.5) checkpoints/b3_lm_48m_24h/best.pt

To chat you need two assets — the thinker checkpoint embeds the codec weights but reads the codec config from the codec file:

mkdir -p checkpoints/FINAL_12H checkpoints/m5_frontier
curl -L -o checkpoints/FINAL_12H/best.pt \
  https://github.com/nochinator/thought-vectors/releases/download/v1.1.0/FINAL_12H-best.pt
curl -L -o checkpoints/m5_frontier/best.pt \
  https://github.com/nochinator/thought-vectors/releases/download/v1.1.0/m5_frontier-best.pt
.venv/bin/tv-chat --ckpt checkpoints/FINAL_12H/best.pt --device cpu

The baseline LM runs standalone: scripted chat probes via scripts/probe_lm_chat.py checkpoints/b3_lm_48m_24h/best.pt, evals via scripts/eval_lm.py and the --lm flag on the shared eval scripts, cost curves via scripts/bench_reply.py. The tokenizer everything depends on ships in the repo (artifacts/tokenizer/spm16k_bpe.model). Setup, evals, and the full reproduction pipeline: docs/REPRODUCE.md. Chat runs on any CPU.

SHA-256

4685ab3dae35e7835ee0fdd469d23d39347b0cca0f71c0598bea30443f0d7672  FINAL_12H-best.pt
bc565782b51d74fe56de1c2d9dd4e44c52b1e6ff192bfcdb758291edeaf5240c  FINAL2_12H-best.pt
77bf080cc216f628fa80c260acc2cfe2ffb03c19bc3d44071047ad4377b0e4ad  m5_frontier-best.pt
86c14f16242859aec9066fc45036326d93eae34d8d95d7724ccb9d3a7d509230  b3_lm_48m_24h-best.pt

The three thinker/codec checkpoints are byte-identical to the v1.0.0 assets; v1.1.0 adds the token-LM baseline and the paper's matched-baseline section (pre-registered comparison, endpoint suite, live-chat round).

Code and checkpoints are MIT-licensed.

🤖 Generated with Claude Code

Assets 6
Loading