Skip to content

Release: v5.15.0

Latest
Latest

Choose a tag to compare

View all tags
@LysandreJik LysandreJik released this 10 Aug 10:28
· 7 commits to main since this release
v5.15.0
5eddc12
This commit was signed with the committer’s verified signature.
Cyrilvallez Cyril Vallez
SSH Key Fingerprint: O743lyKbz6cYCDq5uTRdzyLHAo4JjdtgAF9Q3yVhAQA
Verified
Learn about vigilant mode.

Release v5.15.0

New Model additions

Meta Muse Glimmer

Muse Glimmer, released today, is Meta’s new multimodal model, especially designed for agentic use cases. Distilled from Muse to 30B parameters, and released under the Apache 2.0 license, it can be deployed to local setups for privacy-aware applications such as coding, document analysis, personal assistants, Claw- or Hermes-like setups.

Muse Glimmer is a dense 30B parameter model consisting of:

  • 2B ViT-style encoder for vision (Perception Encoder)
  • 28B parameter text decoder

We're covering it in the following blogpost: http://hf.co/blog/muse-glimmer

image

GraniteMoeSWA & GraniteSWA

image

Links: Documentation

Links: Documentation

A.X-K1 & A.X-K2

image

Links: Documentation

Links: Documentation

Cosmos3 Edge

image

Links: Documentation

Breaking changes

Kernels are now opt-in rather than mandatory for linear attention models (Mamba, GDN, Conv-only, etc.), so users who relied on automatic kernel selection must explicitly enable kernels to maintain previous behavior.

  • 🚨 [Kernels] Refactor all linear attn models & native kernels fallback (#47630) by @vasqu

The cache cropping API now only accepts negative values (relative offsets) instead of absolute sizes, so users calling crop methods directly must update their code to pass negative values accordingly.

T5 and its model family (MT5, LongT5, etc.) now support SDPA and other attention backends via ALL_ATTENTION_FUNCTIONS, meaning the default attention implementation may change and users relying on the previous eager-only path should explicitly set attn_implementation="eager" if needed.

  • 🚨 Enable SDPA (and other attention backends) for T5 and propagate to the T5 family (#47014) by @jiqing-feng

Several small private helper functions (e.g., _is_url, _build_image_tokens) have been removed from multimodal processor files, so users or downstream libraries that imported these private functions directly must remove or replace those references.

Attention

This release includes several attention fixes and improvements, including correcting Multi-Head Latent Attention (MLA) cache compression, optimizing Flash Attention max sequence length computation in vision models, and fixing bugs in CTRL flex-attention and SDPA prefill with position bias. Additional changes refactor linear attention models for better maintainability, make Gemma 4's heterogeneous attention config explicit, and improve MPS support via metal-flash-sdpa integration.

Vision

Vision improvements in this release include performance optimizations such as faster image preprocessing for vision-language models (GLM4V, MiniMaxM3-VL, and others) by eliminating redundant tensor copies, and more efficient Flash Attention variable-length paths by precomputing maximum sequence lengths once per forward pass. Several bug fixes were also applied, including correcting dtype alignment in Kosmos2/Kosmos2_5 embedding merges, fixing a position-embedding initialization fallback in Phi4Multimodal, resolving PIL resize parity in Hunyuan-VL, and patching stop-sequence handling in the image-text-to-text pipeline.

Generation

Several generation improvements and bug fixes were made, including enabling batched audio generation for Qwen2.5/3-Omni, allowing sliding window cache layers to work with speculative decoding, and fixing memory overhead from static cache persistence across generate() calls. Multiple model-specific bugs were also resolved, including crashes in KyutaiSpeechToText, MusicgenForCausalLM, CTRL flex-attention, and assisted decoding for EncoderDecoder cache and OlmoHybrid models.

Cache

Several cache-related bugs were fixed, including correcting NemotronH's missing "mlp" layer-type mapping, resolving recurrent-layer padding masks being skipped during chunked prefill and cache continuation for hybrid models, and fixing assisted decoding for models with EncoderDecoderCache and OlmoHybrid. Additional improvements include aligning OlmoHybrid to use a native cache, enabling sliding window layers to support speculative decoding rollback, and stopping the static cache from being stored as a model attribute to reduce unexpected memory overhead.

Kernels

⚠️ The kernels python package will very likely be a required dependency for transformers[torch] in the near future. This will help us deliver maximum performance to all users; kernels will only be downloaded from trusted publishers manually approved by the HF team. Please let us know of any issues you're facing beforehands so that we may solidify our integration.

Improved robustness of the kernels integration by refactoring function handling to use layer repos, fixing CI EROFS fallback patches for kernel downloads via HfApi, resolving a positional argument collision in causal_conv1d_fn, and bumping the FP8 kernels version to prevent NaNs.

Quantization

Quantization support was expanded with FP8 kernels for compressed-tensors models, fixes for FP8 module normalization and format-based compression detection, and a multi-device MXFP4 dequantization race condition fix. GPTQ and MXFP4 tests were also extended to cover Intel XPU devices.

Audio

Batched audio generation is now supported for Qwen2.5/3-Omni, and several bug fixes were applied across audio models, including a dtype mismatch in Gemma4 audio feature merging, a bfloat16 positional embedding error in AudioFlamingo3, and missing backend requirement guards for Voxtral. The VibeVoice ASR processor was also updated to make audio input optional and support multiple audios per prompt.

Parallelization

Expanded FSDP support across 94 ForCausalLM model classes with auto-generated FSDP plans, added end-to-end FSDP tests including distributed checkpoint save/load and generation, and introduced a dedicated FSDP CI job. Additionally, fixed a device mismatch bug in create_bidirectional_sliding_window_mask under model parallelism and resolved a tensor parallel inference issue for models with tied embeddings.

Tokenization

This release adds native support for Mistral's "tekken" tokenizer format via AutoTokenizer, fixes a CodeLlama tokenizer bug where leading whitespace was incorrectly dropped during decode, and patches a potential ReDoS vulnerability caused by unescaped tokenizer filenames being used as regex patterns in from_pretrained.

Serve

Improved the serve chat parsing to unify streaming and non-streaming paths under a single response parser that handles tool calls, reasoning, and content, simplifying the addition of new model support. Additionally, hardened daily CI reporting by fixing GitHub API diagnostic output being captured in Slack payloads and adding rate-limit resilience to prevent report failures when paginating large job matrices.

Bugfixes and improvements

Significant community contributions

The following contributors have made significant changes to the library over the last release:

  • @ydshieh
    • Fix cached_files silently returning stale file on read-only filesystem (EROFS) (#47852)
    • Fix PhimoeIntegrationTest (#46539)
    • Serialize post-mlinter-review after post-link to avoid PR description race (#47832)
    • Use content hash for mlinter review deduplication (#47830)
    • Fix mlinter review job: checkout before artifact download (#47820)
    • Post mlinter findings as inline PR review comments (#47819)
    • Add post-mlinter-review job to post-dashboard-link workflow (#47800)
    • [conftest] Also wrap snapshot_download for EROFS fallback (#47796)
    • [conftest] Fix EROFS fallback for kernel downloads (correct interception point) (#47794)
    • [conftest] Fix EROFS fallback for kernel downloads via HfApi (#47791)
    • PR CI with torch 2.13 (#47767)
    • [CI - Debug] Skip /transformers-dependent steps for CPU runner (#47759)
    • [CI] Add CPU runner support to ssh-runner workflow (#47757)
    • Update daily CI Docker image to torch 2.13.0 / CUDA 13.0 (#47738)
    • Fix missing github_utils.py download in PR CI dashboard workflow (#47668)
    • byebye torch 2.4 (#47609)
    • Use --flake-runs=1 in check_bad_commit.py for PR comment CI (#47522)
    • Run only @slow tests in PR comment CI (#47521)
  • @kaixuanliu
    • make examples under doc device agnostic (#47812)
    • cancel deterministic for XPU in gemma4 tests (#47790)
    • open fla support for XPU to benefit from the acceleration (#47799)
    • Fix DBRX MoE hidden size and expert GLU transposes (#47671)
    • Fix failing tests for axk1 and axk2 (#47727)
    • Fix failing tests for granite_swa and granitemoe_swa (#47723)
    • skip invalid test cases for inkling tests (#47493)
    • Fix failing tests for mimo_v2_flash (#47284)
    • Fix failing tests for zaya (#47268)
    • tipsv2_dpt: fix failing tests for XPU (#47292)
    • Fix some failed test cases related with XPU Expectations (#47173)
    • add paged attention tests support for XPU (#47163)
    • Fix multi-device mxfp4 dequantization race in _convert_moe_packed_tensors (#47423)
    • fix failed test cases for qwen3_omni_moe model (#47449)
    • Fix model tests for tipsv2 (#47356)
  • @vasqu
    • Fix ci style (#47818)
    • 🚨 [Kernels] Refactor all linear attn models & native kernels fallback (#47630)
    • [Kernels] Refactor function handling (#46883)
    • Add AXK2 from SKT (#47528)
    • Add Harry to slow CI (#47454)
    • Update dev version on main (#47366)
    • [FP8] Bump kernels version (#47344)
  • @kmswin1
    • Hotfix axk2 indexer norm (#47810)
    • Fix A.X-K2 fp8 modules_to_not_convert normalization for the gated-norm MLP (#47578)
    • add_axk1 (#46867)
  • @remi-or
    • [Fix] Fix multi-head latent attention (MLA) (#47761)
    • [Fix] Wrong type hint in get_number_of_image_patches (#47788)
    • [Fix] Swapped height and width in KimiK25 (#47786)
    • Deprecate CB config in gen configuration (#47291)
    • [CB] Automatically switch attention implementation to flash (#47330)
  • @juliendenize
    • [Mistral] Add native tekken tokenizer support to AutoTokenizer (#47507)
    • Make tokenization_mistral_common importable without mistral_common installed (#47397)
    • [Mistral] Move MistralConverter into integrations/mistral/ package (#46603)
  • @IMvision12
    • Add support for batched Qwen2.5/3-Omni audio generation (#47186)
    • Fix Hunyuan-VL PIL image resize parity with reference preprocessing (#47233)
  • @jiqing-feng
    • 🚨 Enable SDPA (and other attention backends) for T5 and propagate to the T5 family (#47014)
    • Fix shape mismatch in KyutaiSpeechToText generate() last window (#46952)
    • Fix typo in MusicgenForCausalLM.generate() (#46974)
    • Fix BlockMask crash in CTRL flex-attention generation (#46854)
  • @tarekziade
    • CI: Log GitHub API diagnostics to stderr (#47635)
    • CI: Add serge review relay workflow and review rules (#47610)
    • CI: use a single function for GH calls (#47474)
    • CI: use transformers-ci daily workflow with OTEL (#47360)
    • CI: let's run integration failure cron at 10pm (#47537)
    • fixed the benchmark script with DistributedConfig (#47568)
    • CI: fix torchaudio pinning +proper break in rnnt (#47422)
    • CI: add reproduce mode to serge verify caller (#47492)
    • CI: add serge verify (GPU) caller workflow (#47381)
    • ci: harden daily CI reporting against GitHub API rate limits (#47382)
    • Fix Daily CI reporting issues (#47364)
    • ci: Handle empty GitHub token in CI run lookup (#47362)
  • @daviswer
    • Add Granite-swa and Granitemoe-swa model support (#47179)
  • @ShareLer
    • Optimize flash attention max seqlen computation in vision attention (#47170)
  • @atharvajoshi10
    • Fix Cosmos 3 Edge Patch packing order (#47399)
    • Add Cosmos3 Edge model support (#47181)

Contributors

kashif, tarekziade, and 70 other contributors
Assets 2
Loading