v0.5.19

Release Highlights

What's Changed

• Harden distributed checkpoint finalization (#793)
• Harden checkpoint slot and collective isolation (#792)
• Add topology-portable TrainerRank checkpoints (#790)
• Revert topology-portable trainer checkpoints (#789)
• Fix GPU prewarm on hypervisor nodes (#788)
• Add compact and tensorized trajectory representations (#786)
• Remove deprecated auto trajectory aliases (#785)
• Add topology-portable trainer checkpoints (#784)
• Add Llama 3 Megatron model support (#783)
• Split divergent token histories by default (#782)
• perf: Submit Tinker training requests in the same clock cycle (#781)
• chore: Upgrade Tinker SDK to 0.23 (#780)
• Simplify trajectory capture scopes (#779)
• Refactor trainer rank package API (#777)
• Fix ART runtime compatibility with vLLM 0.25 (#776)
• Add protocol-native trajectory histories and tokenization (#773)
• Expose assistant-turn selection for file SFT (#772)
• Support final-assistant SFT on ServerlessBackend (#771)
• fix: Parse Qwen3.5 tool calls with the model-native grammar (#770)
• Add composable trajectory token flags (#768)
• Stabilize pipeline RL throughput controls (#767)
• Separate pipeline freshness from runtime tuning (#766)
• Lazy-load exchange tokenization dependencies (#765)
• Fix silent loss in legacy trajectory serialization (#764)
• fix: complete TrainerRank model runtime integration (#763)
• Preserve complete trajectories in Parquet (#762)
• ci: select TrainerRank GPU validation by branch (#761)
• ci: allow manual TrainerRank GPU validation (#760)
• Optimize end-to-end Pipeline RL throughput (#758)
• Upgrade ty to 0.0.59 (#757)
• Add protocol-aware trajectory capture and tokenization (#756)
• Support W&B 0.28 and vLLM 0.24 DSV4 LoRA (#754)
• ci: add manual TrainerRank GPU validation (#748)
• Publish TrainerRank checkpoint slots (#747)
• test: run Megatron tree coverage in CI (#746)
• Add DeepSeek V4 Flash support for ART Megatron (#745)
• Add ART Megatron support for GPT OSS (#744)
• Add final-assistant SFT mode (#743)
• Fix Gemma4 MoE Triton flex stage config (#742)
• feat: add TrainerRank API (#740)
• feat: unify shared-prefix Megatron execution (#739)
• fix: Remove prefill/sample costs for megatron (#738)
• fix: prewarm Docker Hub short tag alias (#737)
• Add Gemma 4 Megatron model support (#736)
• fix: retry stale GPU prewarm pods (#735)
• fix: wait for GPU latest tag convergence (#734)
• fix: refresh mutable GPU tag during prewarm (#733)
• fix: keep GPU image fresh for SkyPilot launches (#732)
• Avoid redundant Megatron empty_cache and GC cleanup (#730)
• Use memory-efficient block-mask construction outside CP (#728)
• feat: Add ART GPU image build script (#725)
• Keep routing replay target prep out of Dynamo (#723)
• Fix routing replay split sizes for attention (#721)
• Include Qwen shared experts in MoE LoRA (#720)
• Log SFT metrics during training (#719)
• Improve Megatron SFT runtime startup (#718)
• Remove LocalBackend from root exports (#717)
• Add Megatron FA4 CP/GDN CP, streaming offload, and optimized LoRA publishing (#716)
• chore: Improve Megatron runtime typing (#714)
• fix: honor LangGraph chat model kwargs (#713)
• Avoid pickle in LangGraph rollout logging (#711)
• Fix vLLM V1 unsupported engine args (#710)
• Clarify MoE routing replay cleanup (#709)
• Reduce tau-bench rollout request overhead (#708)
• Move LoRA config to TrainableModel (#707)
• Guard model config against eager Megatron imports (#706)
• feat: Implement tau-bench minimal training script (#705)
• Support tau-bench environment idle TTL (#704)
• Add tau-bench client rollout support (#703)
• PipelineTrainer KL step lag (#702)
• Enable default MoE expert replay for Megatron train/inference parity (#701)
• Support PipelineTrainer with ServerlessBackend (#699)
• Update README serverless model references (#698)
• Release v0.5.18 (#694)
• Fix serverless auth headers for newer OpenAI SDKs (#689)
• Fix Qwen3 MoE identity LoRA export layout (#688)
• Decouple Megatron paths and centralize chat-template defaults (#686)
• Preserve thinking content in Tinker server chat templates (#685)
• feat: Preserve thinking content in chat templates (#684)
• Add pipeline checkpoint retention strategies (#683)
• [codex] Tighten ART training type surface (#682)
• Preserve RNG state in dataset iteration (#681)
• Rework Megatron to Support Adding New Models (#674)
• feat: Upgrade Tinker dependencies (#670)
• Cancel monitor and server tasks on backend shutdown (#669)
• Disable result packing for Unsloth training (#666)
• Drop overlength SFT tensors before backend training (#665)
• Update torchao for Torch 2.10 compatibility (#664)
• Fix Unsloth autocast dtype for bf16 models (#663)
• Fix CI dependency resolution (#659)
• dev: Add checkpoint forking end-to-end tests (#657)
• Use trusted publisher (#655)
• Fix megatron grad norm explosion (#648)
• Fix flex attention compilation failures in Megatron training (#647)
• DeepEP, torch.compile and Fix Megatron Training Bug (#646)
• Add Megatron SFT internal checkpoints (#645)
• Fix #632: "import weave" has an issue (#644)
• [codex] Restore PR 607 KL loss removal (#639)
• feat: Add Qwen3.5 support to Megatron backend (#637)
• feat: Add dedicated merged mode to Megatron backend (#636)

Full Changelog: v0.5.17...v0.5.19