Repository navigation
Releases
Releases · EvolvingLMMs-Lab/LLaVA-OneVision-2.5
Release v2.0
Release v2.0
This is the first official release of LLaVA-OneVision-2, introducing a new unified visual architecture for image and video understanding, complete training and inference support, scalable data preprocessing, and reproducible checkpoint conversion and validation workflows.
Highlights
- Introduced LLaVA-OneVision-2 model implementations for Hugging Face Transformers and Megatron training.
- Added dense and MoE variants spanning 2B, 4B, 8B, and 30B-A3B configurations.
- Added the OneVision Encoder and codec-aligned image/video processing for stronger spatial and temporal understanding.
- Added alignment, mid-training, instruction tuning, quick-start, and checkpoint conversion workflows.
- Reworked offline packing into a scalable multi-stage pipeline with WebDataset output and distributed execution support.
- Added robust HF/Megatron checkpoint merge, validation, and consistency test suites.
- Added Muon optimizer support and training/checkpointing reliability improvements.
- Launched a dedicated project page with benchmarks, method visualizations, task demos, resources, and community links.
What's Changed
LLaVA-OneVision-2 Models and Training
- Add the LLaVA-OneVision-2 architecture, training entry points, model configurations, and examples by @anxiangsir in #110
- Fix image processor merge-size handling and update documentation by @anxiangsir in #111
- Add Muon optimizer support by @fdcp in #81
- Add training workflows and model variants for Qwen3 2B, 4B, 8B, and 30B-A3B backbones
- Add dense and MoE Hugging Face implementations with image and video processors
Data Processing and Packing
- Introduce a scalable offline packing pipeline covering sample splitting, token-length computation, bin packing, sample packing, and WebDataset conversion
- Add distributed pipeline automation, smoke tests, and practical packing documentation
- Add image validation, dataset inspection, frame extraction, and bounding-box quantization utilities
- Improve JPEG handling, checkpoint configurations, training behavior, and documentation by @anxiangsir in #117
Checkpoint Conversion and Validation
- Add checkpoint conversion scripts for all released model variants
- Add a robust
merge_ov2CLI for dense and MoE checkpoints with explicit validators - Support Hugging Face Hub repository IDs in merge paths by @anxiangsir in #118
- Replace the legacy consistency script with a pytest-based consistency suite by @anxiangsir in #119
Documentation, Evaluation, and Project Page
- Add the LLaVA-OneVision-2 project site and GitHub Pages workflow by @anxiangsir in #121 and #122
- Add interactive image/video task demonstrations and codec visualizations by @didizhu-judy and contributors in #138, #139, #142, and #148
- Add benchmark results, model and dataset resources, technical report links, roadmap, and reproduction links
- Add a streamlined 4B quick start by @anxiangsir in #164, #165, #166, and #167
- Add automatic contributor and community metadata generation by @anxiangsir in #170 and #179
Fixes and Polish
- Improve checkpoint, packing, training, and merge robustness
- Improve mobile layouts, dark-mode figures, citations, resource links, and benchmark presentation
- Add community links for Discord, WeChat, Xiaohongshu, vLLM, and Glint Lab
New Contributors
Thanks to @fdcp, @Feilong607, @didizhu-judy, @YunyaoYan, @tanhuajie, and all contributors who helped build and refine LLaVA-OneVision-2.
Full Changelog: 1.5...2.0
1.5
Release v1.5
This is the first official release of LLaVA-OneVision-1.5, focusing on documentation improvements, data packing & preprocessing reliability, training configuration defaults, evaluation additions, and multiple bug fixes.
Highlights
- Improved documentation and onboarding experience (README, training TODOs, LICENSE updates)
- More robust data packing / WebDataset conversion with support for mixed packing
- Added SFT data preprocessing guidance and materials
- Introduced evaluation-related content
- Improved training defaults (e.g., dtype bfloat16, default 4B merge, auto config updates)
- Numerous bug fixes across packing, filtering, and demo workflows
What’s Changed
Documentation & Project Maintenance
- Update README by @yiyexy in #1
- Add training framework todo list in Readme.md by @VVsssssk in #10
- docs: update LICENSE by @RobitYadda in #21
- offline_sample_packing description by @fdcp in #34
- Update README.md by @wkzhang636 in #77
- Delete run.sh by @anxiangsir in #7
Data Processing / Packing / Filtering
- fix packing bugs by @chengzheng345 in #3
- Update the convert_to_webdataset.py script to support mixed data pack… by @yiyexy in #4
- Fix some bugs by @yiyexy in #5
- add data filter by @chengzheng345 in #12
- fix cfg['filter_with_caption'] bug by @chengzheng345 in #27
- bug fix : processing samples whose names contain some ':' by @fdcp in #49
Training Defaults & Configuration
- set processor.image_processor.max_pixels = 1600*1600 by @yiyexy in #8
- default merge 4b by @yiyexy in #13
- auto update config by @yiyexy in #15
- update default dtype to bfloat16 by @yiyexy in #18
- Remove the hosts Settings for multiple machines and only support sing… by @chengzheng345 in #56
Evaluation
- add Evaluation by @mathCrazyy in #16
SFT Data
Bug Fixes & Robustness Improvements
- fix a bug by @yiyexy in #30
- fix_issue#31 by @killTheHostage in #33
- Remove duplicate dependency
py-cpuinfolibraries. by @Lornatang in #50 - Fixed the issue that the
model_pathname does not correspond to the HF, causing the model to fail to load. by @Lornatang in #51 - refactor(merge_model): Enhanced file implementation robustness by @Lornatang in #52
- fix
Stage 1.5 Mid-Trainingdemo error. by @Lornatang in #57
New Contributors
- @yiyexy made their first contribution in #1
- @chengzheng345 made their first contribution in #3
- @anxiangsir made their first contribution in #7
- @VVsssssk made their first contribution in #10
- @mathCrazyy made their first contribution in #16
- @RobitYadda made their first contribution in #21
- @wideyard made their first contribution in #22
- @killTheHostage made their first contribution in #33
- @fdcp made their first contribution in #34
- @Lornatang made their first contribution in #50
- @wkzhang636 made their first contribution in #77
Full Changelog: https://github.com/EvolvingLMMs-Lab/LLaVA-OneVision-1.5/commits/1.5