Skip to content

v0.5.1

@bitzic bitzic tagged this 27 May 03:46
bf16 vs fp32 on H100 PCIe (254M params, 262M tokens, seed 1337):
  fp32: loss=3.8173, 16,882 tok/s, 259 min
  bf16: loss=3.8163, 63,361 tok/s,  69 min → 3.8x speedup, same quality

New features:
  - bf16 autocast (use_bf16=True default on CUDA)
  - wandb --wandb flag for live training charts
  - wandb metrics auto-export (wandb_metrics.json in proof bundles)
  - Streamlit dashboard with auto-refresh + live loss curve
  - HuggingFace Hub upload/download for proof bundles
  - Launch tweet drafts
Assets 2
Loading