Skip to content

v0.5.2

@george-connito george-connito tagged this 25 Aug 13:27
feat: resume background eval after a restart (#225)
  A validator killed mid-round now reclaims the round's remaining miners
  instead of discarding them. Measured in production: 101/217 miners
  scored on a resumed round vs 2/212 without.

perf: stop materializing the full state_dict on the hot paths (#228)
  fp8 weights dequantize to fp32 on CPU inside state_dict(), so each call
  materialized ~14.5 GB. Removed from per-miner eval load, the freeze
  snapshot, and the merge hash. Cycle peak 59.9 GB -> 39.1 GB on an
  identical box; optimizer step 110s -> 1s; gradient aggregation 57s -> 13s.
  This is what has been OOM-killing the validator ~4x/day.
Assets 2
Loading