v0.1.6

[Luosuu]

Highlights

• Now we gradually migrate VeOmni modeling to patching the HuggingFace modeling so that users can easily know what is different from original HuggingFace implementation. Currently we migrate dense models first. @Coach257 @FoolPlayer
  • In the following, we will work on generating modeling like HuggingFace, which will happen at the same time with HuggingFace Transformers v5 upgrade. @piyifan123
• Support Qwen3-Omni-MoE by @Crystal-jiang .
• Now VeOmni does not override HuggingFace Transformers ALL_ATTENTION_FUNCTION registry. To ensure backward compatibility, when flash_attention_2/3 is passed to model arguments, they will be replaced by veomni flash attention key names. @Luosuu
• Support padding for packed input when rmpad_with_pos_ids, which will eliminate expensive Triton compiling for varying input size and pave the way for torch.compile integration.
• Support torchcodec -based video preprocessing by @TimYangst
• Many fixes.

What's Changed

• [docs] fix: Optimize document links in Markdown rendering by @Crystal-jiang in [docs] fix: Optimize document links in Markdown rendering #380
• [config] feat: add MFU calculation for qwen3_vl_moe by @ZhuYajun-AI in [config] feat: add MFU calculation for qwen3_vl_moe #385
• [data, model] feat: support Qwen3-VL textual token-based time encoding by @Coach257 in [data, model] feat: support Qwen3-VL textual token-based time encoding #386
• [data,ci] test: enhance video_utils test suite with robust validation and benchmarks by @TimYangst in [data,ci] test: enhance video_utils test suite with robust validation and benchmarks #375
• [data,ci,docs] feat: add torchcodec-based video processing with ffmpeg support and comprehensive testing by @TimYangst in [data,ci,docs] feat: add torchcodec-based video processing with ffmpeg support and comprehensive testing #221
• [perf, dist] feat: add zero2 in fsdp1 and use_orig_params configurable by @zhtao303 in [perf, dist] feat: add zero2 in fsdp1 and use_orig_params configurable #382
• [model] fix: Fused operator fix for qwen3vl by @phdddd in [model] fix: Fused operator fix for qwen3vl #378
• [docs] feat: add async doc in ulysses.md by @zbian99 in [docs] feat: add async doc in ulysses.md #388
• [data] feat: add data collators for embedding classification. by @yiwzhao in [data] feat: add data collators for embedding classification. #376
• [model] chore: add moe split script by @FoolPlayer in [model] chore: add moe split script #390
• [task] feat: train qwen2.5 omni by @Coach257 in [task] feat: train qwen2.5 omni #396
• [model] refactor: change to model patch in qwen3 by @FoolPlayer in [model] refactor: change to model patch in qwen3 #392
• [docker] fix: update npu dockerfile with ffmpeg by @FoolPlayer in [docker] fix: update npu dockerfile with ffmpeg #398
• [config] feat: refactor args to support multi-level config by @FoolPlayer in [config] feat: refactor args to support multi-level config #397
• [docs] fix: update async ulysses document by @zbian99 in [docs] fix: update async ulysses document #394
• [model] fix: add fa3 for Qwen3VL vision attention SP path by @Luosuu in [model] fix: add fa3 for Qwen3VL vision attention SP path #400
• [model] feat: patch qwen3vlmoe; qwen3vl; qwen25vl by @Coach257 in [model] feat: patch qwen3vlmoe; qwen3vl; qwen25vl #399
• [model] fix: qwen25vl_config by @Coach257 in [model] fix: qwen25vl_config #404
• [model] fix: fsdp1 load model weights by @Coach257 in [model] fix: fsdp1 load model weights #403
• [config] fix: remove None option from choice type args by @FoolPlayer in [config] fix: remove None option from choice type args #405
• [misc] feat: add live star history in readme by @Luosuu in [misc] feat: add live star history in readme #406
• [config, model] chore: avoid polluting huggingface transformer attention registry while keeping job config backward compatiable by @Luosuu in [config, model] chore: avoid polluting huggingface transformer attention registry while keeping job config backward compatiable #407
• [docs] fix: correct typos in documentation by @zwhe99 in [docs] fix: correct typos in documentation #411
• [data] feat: decouple rmpad and dyn_bsz by @yangtian6781 in [data] feat: decouple rmpad and dyn_bsz #408
• [model] feat: refactor the rot_pos_emb and fast_pos_embed_interpolate funcs in modeling_qwen_vl by @lipengfei1409 in [model] feat: refactor the rot_pos_emb and fast_pos_embed_interpolate funcs in modeling_qwen_vl #391
• [ci] fix: use wheel URLs to avoid build dependencies in CI by @TimYangst in [ci] fix: use wheel URLs to avoid build dependencies in CI  #417
• [model] fix: remove the check that self.training==True for SP by @A1waysBeenHere in [model] fix: remove the check that self.training==True for SP #421
• [model] feat: Add pure HuggingFace version of Qwen3-Omni-MoE by @Crystal-jiang in [model] feat: Add pure HuggingFace version of Qwen3-Omni-MoE #422
• [data, perf, ops] feat: option to pad packed input to a fixed shape for text-only models by @Luosuu in [data, perf, ops] feat: option to pad packed input to a fixed shape for text-only models #410
• [ops, perf] feat: Liger-Kernel is now available for NPU by @zheliuyu in [ops, perf] feat: Liger-Kernel is now available for NPU #415
• [model] feat: Support qwen3-omni-moe model by @Crystal-jiang in [model] feat: Support qwen3-omni-moe model #409
• [dist] feat: No resharding enabled for accelerated small model training by @yangtian6781 in [dist] feat: No resharding enabled for accelerated small model training #413
• [model] fix: qwen_vl rope index by @Coach257 in [model] fix: qwen_vl rope index #430
• [model] refactor: change dense llm to patch style by @FoolPlayer in [model] refactor: change dense llm to patch style #431
• [perf] fix: The H200 compute power is being recognized as H20 by @HSYZhang in [perf] fix: The H200 compute power is being recognized as H20 #428

New Contributors

• @ZhuYajun-AI made their first contribution in [config] feat: add MFU calculation for qwen3_vl_moe #385
• @zhtao303 made their first contribution in [perf, dist] feat: add zero2 in fsdp1 and use_orig_params configurable #382
• @zwhe99 made their first contribution in [docs] fix: correct typos in documentation #411
• @yangtian6781 made their first contribution in [data] feat: decouple rmpad and dyn_bsz #408
• @lipengfei1409 made their first contribution in [model] feat: refactor the rot_pos_emb and fast_pos_embed_interpolate funcs in modeling_qwen_vl #391
• @A1waysBeenHere made their first contribution in [model] fix: remove the check that self.training==True for SP #421
• @zheliuyu made their first contribution in [ops, perf] feat: Liger-Kernel is now available for NPU #415
• @HSYZhang made their first contribution in [perf] fix: The H200 compute power is being recognized as H20 #428

Full Changelog: v0.1.5...v0.1.6

---

This discussion was created from the release v0.1.6.