v2.9.1: Arena chat sessions & PAT auth, MoE LoRA under ZeRO-3, GRPO deadlock fix
·
62 commits
to main
since this release
Features
- Arena chat sessions: LLM deployments now keep conversation state server-side.
Agent.list_sessions/
get_session/delete_session, an optionalsession_idongenerate()andgenerate_stream(), and a new
arena agent sessions list/get/resume/clear/deletecommand group. The CLI keeps one conversation going per
deployment, with--new-sessionto start fresh and--session-idfor a one-off;resumewith no id opens an
arrow-key picker built on termios and msvcrt, so it adds no dependency (#661). - PAT-authenticated inference:
Agenttakes a single credential, sent asAuthorization: Beareron every
route and falling back toARENA_API_KEY. It is never logged, shown inrepr(), or written to disk (#661). memory_scope:deploy_agentandarena agent deploytake an optional--memory-scope user|organization, omitted from the request unless set so a redeploy keeps the stored scope (#661).
Fixes
- Expert LoRA attaches under ZeRO-3 without all-gathering the packed experts.
get_peft_modeland
upgrade_moe_param_wrappersread only the targeted parameters' shapes, dtype and device, so partitioned params
now get zero-storageds_shapeviews. The gather OOMed at attach time on large MoEs, around 55 GB of experts per
rank for Nemotron-3.5-Lightning-30B on 80 GB A100s (#658). - ZeRO-3 persistent params stay resident while the fetch trace is incomplete. deepspeed 0.19.3 honours
ds_persistinrelease_sub_moduleonly once the trace completes, and leaf-module models never complete one, so
every sub-threshold parameter was re-gathered on each use despite being marked persistent (#658). - Multi-process GRPO no longer deadlocks on advantage filtering. Per-rank sample dropping desynchronized the
data-parallel collective schedule, parking a fully filtered rank at a barrier until the NCCL watchdog fired.
Multi-process runs now keep the full batch and zero the advantages of filtered samples; single-process runs keep
the drop and its compute saving (#660). - A stale deployment URL recovers instead of failing with a bare 404. Redeploying moves a deployment, so
open_inference_agentcatches the 404 at bind time, refetches the binding once, and retries, repairing the cache
for later commands. Narrow by design: only a 404, only when the cached URL was used, and only when the refetched
URL differs (#661). - Family-detection tests use the released
NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16id in place of the dev
preview name. The Hub redirects the old id, so nothing was broken;detect_model_familiesmatches on the
nemotronsubstring and dispatch is unchanged (#663).
Breaking Changes
- Arena inference callers authenticate as themselves with a user PAT. Deployment API keys are gone from the platform, and
the deploymentapi_keyis gone from the binding cache:save_bindingpurges any key an older release left in
~/.arena/inference.json, so one write cleans every stale entry (#661).
Versions
agilerl-arena==0.2.0 released alongside, carrying chat sessions and PAT-based inference auth; the arena extra pins it.
What's Changed
- Attach expert LoRA under ZeRO-3 without gathering the packed experts by @micdoh in #658
- Add PAT-authenticated chat sessions for Arena LLM deployments by @jaimesabalbermudez in #661
- Mask zero-advantage samples instead of dropping them under multi-process GRPO by @micdoh in #660
- Use the released Nemotron 3.5 Lightning model id by @micdoh in #663
- v2.9.1: Arena chat sessions with PAT auth, ZeRO-3 MoE LoRA attach fix by @jaimesabalbermudez in #662
Full Changelog: v2.9.0...v2.9.1