Skip to content

v1.15.0

Latest
Latest

Choose a tag to compare

View all tags
@qgallouedec qgallouedec released this 08 Oct 19:26
· 17 commits to main since this release
v1.15.0
dae7b6a
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.
Neon-Gripped Industrial Valve

Features

⚡ Fused LM head: up to 6.9× longer sequences on the same GPU

SFT, DPO, KTO, GRPO, RLOO and Distillation now score tokens with a fused LM head: a Triton kernel projects the hidden states through the LM head in tiles and reduces to per-token log-probs and entropy directly, so the [batch, seq, vocab] logits tensor is never built. It is on by default, there is nothing to turn on.

Max trainable sequence length (gemma-3-1b, 262k vocabulary, 79 GiB of GPU memory, bf16, batch size 1, gradient checkpointing, sdpa):

Trainer v1.14.2 v1.15.0
DPO 10,240 59,392 5.80×
KTO (batch size 2) 9,216 63,488 6.89×
GRPO (scoring only) 28,672 114,688 4.00×
RLOO (scoring only) 23,552 100,352 4.26×
SFT (loss_type="nll") 20,480 107,520 5.25×
SFT (default chunked_nll) 107,520 107,520 1.00×

At 8,192 tokens, peak memory drops 52% to 82% and steps are 2.3% to 10.9% faster:

Trainer Peak GiB v1.14.2 → v1.15.0 Tokens/s
DPO 48.97 → 12.00 (-75.5%) 1.035×
KTO 64.91 → 11.98 (-81.5%) 1.045×
GRPO 24.15 → 10.30 (-57.4%) 1.023×
RLOO 29.28 → 14.02 (-52.1%) 1.029×
SFT (nll) 30.33 → 8.87 (-70.7%) 1.029×

The gain is largest for small models with large vocabularies. SFT's previous default (chunked_nll) already avoided full logits, so it is unchanged. Scoring needs Triton on a GPU (Linux with CUDA, ROCm or XPU).

💥 What this changes for you. The full-logits scoring path, the use_liger_kernel chunked path and _forward_redirection are gone from GRPO, RLOO, DPO and KTO:

  • A PEFT adapter on lm_head now raises. Use modules_to_save=["lm_head"] instead.
  • use_liger_kernel=True is deprecated in these four trainers and will be removed in v2.0.0. Liger's layer kernels still apply, but fused_linear_cross_entropy is forced to False (it would replace the fused head) and setting it explicitly raises. Use model_init_kwargs={"use_kernels": True}.
  • In DPO and KTO, compute_metrics and compute_loss(..., return_outputs=True) still receive the full logits, from an extra forward pass taken only when they are used. Both warn that this is deprecated and will be removed in v2.0.0.

Nothing else the trainers logged is lost: the kernel gained log_sum_sq_probs (WPO weighting), mean_logits (logits/chosen, logits/rejected) and is_top1 (DPO's mean_token_accuracy) as opt-in outputs, each checked against the full logits in tests.

Benchmark details

1× B300 with the PyTorch allocator capped at 79 GiB (about an H100 80GB's usable memory), google/gemma-3-1b-pt built from its config with random bf16 weights, lr 1e-6, AdamW, synthetic token ids at exact length. Max length: one fresh process per attempt, 2 training steps, doubling from 8k then bisection to 1024 tokens, arms interleaved round-robin in the same job. Throughput: 12 steps, first 2 discarded, median of 3 A/B/A/B repeats in the same job. Noise floor (A vs A) is 0.1% to 0.8%, below every gap reported. GRPO/RLOO replace generation with fixed random completions, so only scoring is measured.

Not measured: H100 hardware itself, flash-attention (sdpa only), real generation, multi-GPU.

Selective activation checkpointing in SFT

With gradient checkpointing on, the attention output is now saved during the forward instead of being recomputed in the backward, recovering most of the checkpointing slowdown at long context for one extra hidden-state-sized tensor per layer. Same eager SAC approach torchtitan uses under FSDP2, no torch.compile needed.

SFTConfig(
    gradient_checkpointing=True,
    gradient_checkpointing_kwargs={"selective": True},
)

PEFT + DeepSpeed ZeRO-3 is rejected: that combination needs reentrant checkpointing, which SAC cannot use.

by @kashif in #7194

Assistant-only loss on vision datasets in SFT

assistant_only_loss=True now works on vision datasets. SFTTrainer used to refuse it because processors did not return usable assistant masks once image placeholders were expanded; transformers#48793 fixed that, and it ships in transformers 5.18.

SFTConfig(assistant_only_loss=True)  # now fine with an image dataset

The vision collator tokenizes conversational examples through the processor's apply_chat_template and requests the assistant masks to mask non-assistant tokens in the labels. Older transformers still raises, now with the required version in the message.

by @albertvillanova in #7377

Conversations in the completions table

Prompts and completions are logged as raw conversation lists instead of batch_decode'd flat strings, so the table renders turn by turn. Much more readable for multi-turn and tool-calling setups, and for VLMs.

Completions table rendered turn by turn

The rendered system prompt (including tool definitions injected by the chat template) is no longer visible, since the original messages are shown rather than the fully-templated string.

by @qgallouedec in #5309

AsyncGRPO and AsyncDistillation

  • AsyncGRPO: train OpenEnv harnesses from validated token captures by @adithya-s-k in #6947
  • Add PEFT support to AsyncDistillationTrainer by @kashif in #7476
  • Fix duplicate completions in AsyncGRPO groups with data-parallel vLLM by @lewtun in #7549

Other

Breaking changes and removals

  • Drop Python 3.10 support by @qgallouedec in #7497
  • Refuse nn.DataParallel (and drop the multi-GPU slow test job) by @qgallouedec in #7407
  • Remove the experimental MiniLLM trainer by @qgallouedec in #7500
  • use_liger_kernel is deprecated in DPOTrainer, KTOTrainer, GRPOTrainer and RLOOTrainer and will be removed in v2.0.0. These trainers now use the fused LM head, so only Liger's layer kernels apply; use model_init_kwargs={"use_kernels": True} instead. Docs updated by @qgallouedec in #7535
  • Drop vLLM 0.20.0 / 0.20.1 / 0.20.2 support by @qgallouedec in #7435, #7499 and #7568
  • Remove the unreachable prepare_multimodal_messages_vllm by @behroozazarkhalili in #6946

Fixes

Documentation

CI

Chat template coverage, so drift stops recurring, all by @albertvillanova: document every stored chat template and check it in CI (#7502), add a script checking that reference Hub repos still ship a stored chat template (#7523), check it weekly (#7521), and keep the earlier LFM2 revision under test (#7527)

Tiny-model config alignment, continuing the sweep, all by @albertvillanova: DeepSeek-V3 (#7398) and V3-0528 (#7399), GPT-OSS (#7424), Qwen2-VL (#7425) and Qwen2.5-VL (#7426) mrope sections, Nemotron-3-Nano (#7443), Nemotron-3.5-Lightning (#7444), Nemotron-3-Super (#7447), Nemotron-3-Ultra (#7448), LFM2 (#7400), LFM2.5 (#7532)

Test quality, following the "real objects, not mocks" rule, all by @albertvillanova: build real trainers instead of object.__new__ in the GRPO and SDFT tests (#7531), build real configs instead of SimpleNamespace in the GOLD and GKD tests (#7564), build the async rollout loops through __init__ (#7555), run the async GRPO rollout loop tests on the real tokenizer and response parser (#7556), align the DFT loss tests on a real model output (#7481)

Suite hygiene, all by @albertvillanova unless noted: move the vLLM training (#7488), continuous batching (#7487) and SFT activation offloading (#7477) tests to the regular suite, unmark the vLLM client/server tests as slow (#7490), run them on a single xdist worker (#7489), replace the SFT slow tests with an fp16 test (#7550), remove the always-skipped Gemma 3n tests (#7485), drop the no-op slow and low_priority markers (#7486), stop piping the vLLM test server output (#7414), run the VLM test server on the last visible accelerator (#7412), test the vLLM client against the documented vllm serve command (#7378), inject MODEL_REVISIONS into the Auto loaders (#7528), mirror the pad token onto the model config in the callback tests (#7383), remove the DataParallel gather warning filter (#7554), tie the mamba-ssm autocast warning filter removal to transformers 5.6 (#7431), remove the nightly cache cleanup workflow (#7395), scope GITHUB_TOKEN permissions per job by @hf-security-analysis (#7381), bump the doc-builder workflow pin by @paulinebm (#7542), Dependabot group bump (#7411)

Hotfixes: xfail the GRPO continuous batching test under DataParallel (#7422), xfail Qwen3.5-MoE VLM tests against a broken transformers dev build (#7437) and revert once fixed (#7460)

New Contributors

What's Changed

Full Changelog: v1.14.0...v1.15.0

Contributors

kashif, aazizyan, and 19 other contributors
Assets 2
Loading