Skip to content

v4.2.0 — HF-Reference Fidelity + Reasoning API (7B chapter complete)

Latest

Choose a tag to compare

@thebeastagi thebeastagi released this 06 Jul 23:10

v4.2.0 — Reference Fidelity

The 7B chapter close-out. ATLAS output is now byte-identical to the HuggingFace transformers reference for OLMo-3-7B-Think.

Three silent inference bugs fixed (77fdc34)

Found by differential testing against HF transformers (token-level logit comparison):

  1. YaRN correction range — missing 2π wavelength factor + wrong ramp; attn_factor must be applied squared
  2. Layer-type RoPE split — YaRN only on the 8 full-attention layers; the 24 sliding-window layers use plain θ=500K
  3. QK-norm scope — RMSNorm over the full 4096-dim Q/K vector, not per-head (the per-head variant destroyed retrieval >128 tokens)

Benchmark impact (live A100 endpoint)

Bench Before After
GSM8K (25) 12% 88% (Ai2 ref: 85–91)
MMLU (100, direct) 22% 54%
HumanEval (15, exec-verified) 73.3% pass@1

Serving upgrades (17c4c9c)

  • Context 4K → 16K, max output 3,584
  • Official OLMo chat template + <think> primer + Ai2 system prompt server-side
  • Reasoning API: message.reasoning + streaming delta.reasoning (OpenRouter convention)
  • Sampling defaults per Ai2 reference (rep_penalty 1.15 → 1.0), context-overflow guard
  • Bearer-key auth on /v1/*; production endpoint behind Cloudflare at https://atlas.thebeastagi.com
  • compute-sanitizer memcheck clean; exact CPU/GPU long-sequence parity test added

Also in this release window

  • OpenRouter launch prep: /privacy policy page + provider description knob (3d1a502)
  • GPU kernel-launch failure detection — no more silent garbage (bb8227a, 543edd1)
  • AdamW step_gpu host-pointer no-op fixed with systemic CUDA error propagation (062674d, closes #19)

627 tests · CI green. Next: the 32B chapter (#26), batched prefill (#22), GEMV occupancy (#23), 32K context (#24).