Skip to content

v0.1.4

Latest

Choose a tag to compare

@kcz358 kcz358 released this 06 Aug 06:14
· 2 commits to main since this release
25bd616

What's Changed

  • docs: update training launch commands and add hydra override documentation by @kcz358 in #131
  • [feat] NanoVLM Training support by @Purshow in #134
  • [feat] Add total FLOPS, training duration & CO2 emission tracking by @Jinghao-Guo in #137
  • docs: Update raedme.md and scripts of NanoVLM by @Purshow in #138
  • [docs] modify qwen3_vl example config by @KemingWu in #140
  • [docs] Update NanoVLM readme by @Purshow in #141
  • [WIP] feat: Transformers 5.0 compatibility by @kcz358 in #142
  • [feat] support messages column as JSON string in iterable datasets by @mwxely in #147
  • [fix] handle parquet schema mismatch in dataset concatenation by @mwxely in #146
  • [feat] Support EP for transformers >= 5.0 by @ngquangtrung57 in #148
  • doc:fix typo by @Purshow in #149
  • [fix] remove dead code, fix typos, and eliminate unnecessary GPU-CPU syncs by @choiszt in #150
  • fix(cicd): auto-select free port and fix qwen3 FSDP2 attention_type compat by @kcz358 in #151
  • fix(dataset): join data_folder with video path in Qwen3VL iterable dataset by @kcz358 in #152
  • chore: add Dockerfile and .dockerignore by @kcz358 in #153
  • fix(qwen3_vl): align video timestamp token placement with transformers by @kcz358 in #154
  • fix(merger): handle non-sharded tensors in FSDP2 checkpoint merging by @kcz358 in #155
  • fix(merger): copy processor_config.json for custom processors by @kcz358 in #156
  • fix(merger): re-tie weights to avoid duplicating tied parameters by @kcz358 in #157
  • fix(fsdp2): avoid double-dividing tokens_per_gpu by sp_size by @kcz358 in #158
  • fix(qwen3_vl): apply pixel limits via processor size by @kcz358 in #159
  • fix(merger): use DTensor placements to detect replicated vs sharded params by @kcz358 in #160
  • refactor(packing): extract online packing strategies by @kcz358 in #161
  • refactor(packing): streaming BalancedPacking to eliminate worker stalls by @kcz358 in #162
  • feat(kernels): pluggable varlen attention backend (FA2 / SDPA) by @kcz358 in #163
  • fix(packing): rename next_fit to first_fit by @kcz358 in #164
  • feat(datasets): support optional video_start/video_end on video_url by @kcz358 in #165
  • fix(bagel): guard flash_attn imports so models init works without it by @kcz358 in #166
  • feat(qwen3_5): packed linear-attention forward + fix DynamicCache import by @kcz358 in #167
  • feat: ViT frame parallel + Qwen3.5 VL forward by @kcz358 in #168
  • fix(qwen3_vl): patch vision rope with liger + gate text_model.norm rms_norm by @kcz358 in #169
  • [feat] add LLaVA-OneVision2 (8B-Instruct) training support by @kcz358 in #170
  • feat(qwen3_5_moe): modeling + EP support + multi-axis DTensor merger by @kcz358 in #171
  • fix(iterable-dataset): implement stateful protocol for resume by @kcz358 in #172
  • feat(qwen3_5): sequence parallel + ViT frame parallel integration by @kcz358 in #173
  • feat(profiler): CUDA memory snapshot with auto-dump on OOM by @kcz358 in #174
  • fix(qwen3_5): shard ViT frames across CP ranks by @kcz358 in #175
  • fix(profiler): dump memory snapshot on trainer OOM by @kcz358 in #176
  • feat: add Qwen3-VL ViT frame parallel SP support by @kcz358 in #177
  • feat: add Qwen3-VL tensor parallel by @kcz358 in #178
  • feat: add HSDP (Hybrid Sharded Data Parallel) support by @kcz358 in #180
  • feat(profiler): add lightweight CUDA event timing by @choiszt in #179
  • perf(fsdp2): reduce per-step host/comm overhead by @kcz358 in #181
  • docs: fix repo URL in README (LMMs-Lab -> EvolvingLMMs-Lab) by @ngquangtrung57 in #182
  • perf(rope): vectorize qwen3_vl_get_rope_index, drop per-token sync by @kcz358 in #184
  • feat(datasets): lmms_video_utils video backend for LLaVA-OneVision-2 codec stream by @kcz358 in #183
  • fix(fsdp2): broadcast non-persistent buffers under enable_parallel by @kcz358 in #185
  • feat: add Aero Realtime training support by @kcz358 in #187

New Contributors

Full Changelog: v0.1.3...v0.1.4