Halo 1.0.0

First public release.

Training methods. SFT and continued pre-training, DPO / KTO / SMPO preference optimization, Bradley-Terry reward modeling and sequence classification, offline / online / environment (multi-turn, tool-using) GRPO, teacher / self / online (SDPG) distillation, and embedding fine-tuning — all on one distributed trainer spine.

Parallelism. Expert (DeepEP V2), Context (Ulysses), Tensor (DTensor), and Expert-Tensor parallelism over FSDP2, composable (EP+CP, EP+TP, EP+ETP) and multi-node aware (NVLink-domain topology, shared and node-local filesystems). Invalid shapes are rejected at config time.

Models. Dense HF models plus 15 MoE families: GPT-OSS, Qwen3, Qwen3.5/3.6, GLM-4, GLM-5 Next, Laguna, Inkling, Command A+ (Cohere2 MoE), Gemma 4, Bailing/Ling, LFM-2, Mistral 4, DeepSeek-V4, Zaya, Step-3.7 Flash — trained natively via transformers (~5.16); checkpoints load with from_pretrained. VLM training where the family ships a vision head.

RL. Fully asynchronous multi-turn RL on a clean Transformers ↔ vLLM/SGLang split: vLLM 0.26.0 or SGLang 0.5.17 rollout servers in separate containers with native NCCL weight transfer, routed-experts capture and replay, rollouts overlapping training via Ray actors and a prefetch queue; multi-turn environments (SWE, code contests, QA/search, MCP, ReAct) with sandboxed tool execution. Both engines take the weight sync for every family their pinned loaders can serve, expert distribution included, and the sync reaches servers on other nodes over AWS EFA at NIC line rate. A reward is a list of weighted terms — the environment's own grade, a generative judge, a served reward model, verifiable graders — read the same way by online and environment GRPO. Off-policy control: a truncated importance-sampling ratio with band, veto and OPSM mask stages, an optional engine-referenced re-score, and a trust-region breaker that skips the update when a step is mostly masked. No Megatron backend, no veRL.

Performance. FlashAttention-4 on Blackwell (FA2+FA3 on Hopper), grouped GEMM, Liger kernels scoped per family, padding-free packing, AdamWBF16 stochastic-rounding optimizer (6 B/param), Muon, FlashAdamW, fp8/fp4 QAT with DeepGEMM and mxfp8/nvfp4 export, per-layer-kind MFU accounting. Up to ~2.8× stock-TRL throughput on 8× B300.

Using it. One halo CLI picks the launcher and takes config overrides as plain flags: halo launch sft config.yaml -n 8 --expert_parallel_size=8, halo run <tool>. human-docs/ is the guide for people — every training method, configs, data, checkpoints, scaling and rollout servers; agent-docs/ is the full reference, and skills/ ships agent skills for Claude Code and Codex.

Images. Prebuilt and credential-free: public.ecr.aws/whitecircle/halo:{blackwell,hopper,vllm-0.26.0,sglang-0.5.17} plus -1.0.0 version pins, built from this commit (371e24e87). All four carry one pinned EFA userspace; compose overlays put the rollout sync on EFA. No latest tag (images are architecture-specific).

Pipeline-parallelism seams (config surface, rank math, stage/loss contracts) ship in this release; the engine is nearly complete and will be enabled for selected models and trainers in an upcoming version after extensive testing.