Skip to content

v0.8.3

Choose a tag to compare

View all tags
@warlockee warlockee released this 02 Mar 16:11
· 42 commits to main since this release
v0.8.3
e17bfa3

v0.8.3 — Critical Bug Fixes

8 bugs fixed across PPO, GRPO, checkpoint saving, and environment propagation. All 4 algorithms (SGRPO, GSPO, CISPO, PPO) verified end-to-end with Qwen2.5-0.5B-Instruct. 68/68 unit tests pass.

Critical Fixes

  • PPO gradient leakage: detach hidden_states in value_forward() to prevent value loss gradients corrupting policy parameters
  • PPO mask validation: relax validation to allow legitimate 0→1 transitions at prompt-response boundary

Crash Fixes

  • Ray ObjectRef auto-resolution: use state_dict directly instead of calling ray.get() on already-resolved references
  • Safetensors shared tensors: clone tied embeddings (e.g. Qwen2.5 lm_head/embed_tokens) before saving
  • CUDA_HOME propagation: forward to Ray workers so DeepSpeed compiles correctly
  • VLLM_USE_V1=0 propagation: prevent V1 engine OOM in rollout workers

Other Fixes

  • Save config.json from driver after popping it from state_dict
  • Fix grpo_test.yaml data path

New

  • Test configs for all algorithms (SGRPO, GSPO, CISPO, PPO)
  • GSM8K mini datasets for fast validation runs
Assets 2
Loading