Release v2.0

Release v2.0

This is the first official release of LLaVA-OneVision-2, introducing a new unified visual architecture for image and video understanding, complete training and inference support, scalable data preprocessing, and reproducible checkpoint conversion and validation workflows.

Highlights

• Introduced LLaVA-OneVision-2 model implementations for Hugging Face Transformers and Megatron training.
• Added dense and MoE variants spanning 2B, 4B, 8B, and 30B-A3B configurations.
• Added the OneVision Encoder and codec-aligned image/video processing for stronger spatial and temporal understanding.
• Added alignment, mid-training, instruction tuning, quick-start, and checkpoint conversion workflows.
• Reworked offline packing into a scalable multi-stage pipeline with WebDataset output and distributed execution support.
• Added robust HF/Megatron checkpoint merge, validation, and consistency test suites.
• Added Muon optimizer support and training/checkpointing reliability improvements.
• Launched a dedicated project page with benchmarks, method visualizations, task demos, resources, and community links.

What's Changed

LLaVA-OneVision-2 Models and Training

• Add the LLaVA-OneVision-2 architecture, training entry points, model configurations, and examples by @anxiangsir in #110
• Fix image processor merge-size handling and update documentation by @anxiangsir in #111
• Add Muon optimizer support by @fdcp in #81
• Add training workflows and model variants for Qwen3 2B, 4B, 8B, and 30B-A3B backbones
• Add dense and MoE Hugging Face implementations with image and video processors

Data Processing and Packing

• Introduce a scalable offline packing pipeline covering sample splitting, token-length computation, bin packing, sample packing, and WebDataset conversion
• Add distributed pipeline automation, smoke tests, and practical packing documentation
• Add image validation, dataset inspection, frame extraction, and bounding-box quantization utilities
• Improve JPEG handling, checkpoint configurations, training behavior, and documentation by @anxiangsir in #117

Checkpoint Conversion and Validation

• Add checkpoint conversion scripts for all released model variants
• Add a robust merge_ov2 CLI for dense and MoE checkpoints with explicit validators
• Support Hugging Face Hub repository IDs in merge paths by @anxiangsir in #118
• Replace the legacy consistency script with a pytest-based consistency suite by @anxiangsir in #119

Documentation, Evaluation, and Project Page

• Add the LLaVA-OneVision-2 project site and GitHub Pages workflow by @anxiangsir in #121 and #122
• Add interactive image/video task demonstrations and codec visualizations by @didizhu-judy and contributors in #138, #139, #142, and #148
• Add benchmark results, model and dataset resources, technical report links, roadmap, and reproduction links
• Add a streamlined 4B quick start by @anxiangsir in #164, #165, #166, and #167
• Add automatic contributor and community metadata generation by @anxiangsir in #170 and #179

Fixes and Polish

• Improve checkpoint, packing, training, and merge robustness
• Improve mobile layouts, dark-mode figures, citations, resource links, and benchmark presentation
• Add community links for Discord, WeChat, Xiaohongshu, vLLM, and Glint Lab

New Contributors

Thanks to @fdcp, @Feilong607, @didizhu-judy, @YunyaoYan, @tanhuajie, and all contributors who helped build and refine LLaVA-OneVision-2.

Full Changelog: 1.5...2.0