Skip to content

v0.9.0

Choose a tag to compare

View all tags
@mikasenghaas mikasenghaas released this 25 Aug 20:14
· 335 commits to main since this release
v0.9.0
ab5de8f
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

105 commits from 11 contributors since v0.8.0.

Highlights

1. Adaptive Concurrency

The orchestrator and evals process now size in-flight episodes from live vLLM pressure. The controller grows into available capacity, backs off before KV-cache thrash, and cancels the youngest work during overload. Set equal minimum and maximum limits to keep fixed concurrency.

#3285 — Adaptive concurrency for RL
#3309 — Adaptive concurrency for online evals
#3307 — Soft and hard KV-cache caps
#3313 — Staleness-aware cancellation

2. Local Run Dashboard

uv run dashboard shows live metrics, configs, rollout traces, rendered tokens, tool context, and merged logs. Interactive RL and SFT runs start one shared local daemon. Local agents can drive the open dashboard and create reports with verified trace citations.

#3351 — Local run dashboard
#3371 — Auto-started dashboard daemon
#3374 — Dashboard usability fixes
#3377 — Tool context and rendered tokens
#3387 — Agent commands and verified reports
#3268, #3271, #3375 — Run directories, attempt logs, and PRL_OUTPUT_DIR

3. Agentic Evals for SFT

SFT can launch rollout-based eval environments beside training. Single-node runs split trainer and inference GPUs, while multi-node runs use separate trainer and eval jobs. NCCL is the default weight transport. Filesystem transport supports LoRA and external inference.

#3256 — Online evals during SFT
#3323 — NCCL weight transport for SFT evals

4. Full CPU Optimizer Offload

Full offload keeps gradients, FP32 master weights, and optimizer state in host memory. Optimizer chunks run on the CPU while GPU backward continues. AdamW and SignSGD are supported, and SignSGD uses 8 bytes of host memory per parameter.

#3234 — Overlap CPU optimization with backward
#3303 — SignSGD support
#3305 — Fused AdamW defaults
#3383 — Safe cache release before weight broadcast

5. Custom CUDA Kernels

The new prime-kernels submodule and release-wheel pipeline ship architecture-specific CUDA kernels without changing the pure-Python prime-rl wheel. The first opt-in kernel provides fused MoE forward paths for Blackwell with BF16 and MXFP8 variants.

#3154 — Custom vendored kernels
#3321 — Prebuilt release wheels

6. Composable Curricula

Each training source can use a checkpointed task sampler and named admission gates. Built-in curricula provide reward-weighted difficulty pools and advantage-range gates. Rejected groups remain observable while the orchestrator backfills the training batch.

#3261 — Composable curricula
#3297 — Native episode scoring and admission

Dependencies

  • verifiers: 0.3.0 → 0.3.1. #3372
  • renderers: 0.1.9 → 0.1.10. #3367
  • prime-pydantic-config: 0.4.2 → 0.4.3. #3256
  • vllm-router: 0.1.26 → 0.2.0. #3274
  • GPU packages now live in the optional gpu extra. Dashboard dependencies live in the optional dashboard extra. #3370, #3351
  • prime-kernels is available through the optional kernels extra with prebuilt x86_64 and aarch64 wheels. #3154, #3321

Breaking

  • vLLM config: move inference settings to [inference.vllm] for RL and [vllm] for standalone inference. Use native vLLM field names. The old model, parallel, flat vLLM fields, and vllm_extra paths are removed. #3202
  • Concurrency config: replace orchestrator.max_inflight_episodes with [orchestrator.concurrency]. Remove orchestrator.oversampling_factor. Replace eval.max_inflight_episodes with [eval.concurrency]. #3285, #3309
  • Monitor config: use [monitors.wandb], [monitors.file], and [monitors.prime]. Distribution logs, W&B sample tables, usage reporting, and final_summary.json are removed. Read metrics.jsonl for local metrics. #3275
  • SFT deployment config: rename deployment.num_gpus to deployment.num_train_gpus. Rename multi-node deployment.num_nodes to deployment.num_train_nodes. #3256
  • Final weight broadcast: remove weight_broadcast.broadcast_final. The trainer now broadcasts every policy version, including the final one. #3385
  • Run directory and resume config: output now uses output_dir / run.dir. Move ckpt.resume_step to top-level [resume]. Rename clean_output_dir to clean. Resolved component configs move to <run_dir>/configs/resolved/. #3268, #3351
  • SLURM artifacts: generated scripts and completion markers move to <run_dir>/launcher/. SLURM output moves to <run_dir>/launcher/logs/. Update manual sbatch commands and log collection paths. #3324
  • Checkpoints: [ckpt.weights], ckpt.weights_only, and persistent weights/ snapshots are removed. DCP checkpoints are the persistent training artifact. Filesystem broadcasts use sharded safetensors. #3315, #3316
  • Benchmark mode: bench, --bench, and BenchConfig are removed. Run short normal training jobs and read metrics.jsonl. #3272
  • GPU installation: bare uv sync no longer installs the training and inference stack. Use uv sync --all-extras or add --extra gpu. #3370
  • Local run helper: scripts/tmux.sh is removed. Use uv run dashboard. #3351
  • Environment API: only v1 environments are supported. Remove [legacy] blocks. Drop the -v1 suffix from taskset IDs, such as reverse-text-v1 → reverse-text. #3226
  • Multi-tenant training: multi-tenant discovery, eviction, and per-run optimizer settings are removed. Configure one inference endpoint as a string in client.base_url; lists are no longer accepted. #3189
  • Rollout filters: pre_batch_filters and post_batch_filters are removed. Configure a curriculum sampler and admission gates instead. Exact zero-advantage pruning remains at orchestrator.train.filter_zero_advantages. #3261
  • Environment submodule: deps/research-environments moves to deps/prime-envs. Existing clones must sync the new submodule path. #3241
  • Metrics: optim/zero_grad_ratio and its Prometheus gauge are removed. #3249

Changelog

Contributors

eexwhyzee, chethanuk, and 9 other contributors
Assets 4
Loading