The Cradle framework is a first attempt at General Computer Control (GCC). Cradle supports agents to ace any computer task by enabling strong reasoning abilities, self-improvment, and skill curation, in a standardized general environment with minimal requirements.

[CVPR 2024 🔥] Grounding Large Multimodal Model (GLaMM), the first-of-its-kind model capable of generating natural language responses that are seamlessly integrated with object segmentation masks.

Eagle Family: Exploring Model Designs, Data Recipes and Training Strategies for Frontier-Class Multimodal LLMs

LLaVA-Interactive-Demo

[CVPR'24] HallusionBench: You See What You Think? Or You Think What You See? An Image-Context Reasoning Benchmark Challenging for GPT-4V(ision), LLaVA-1.5, and Other Multi-modality Models

PG-Video-LLaVA: Pixel Grounding in Large Multimodal Video Models

The code for "TokenPacker: Efficient Visual Projector for Multimodal LLM".

Official code for Paper "Mantis: Multi-Image Instruction Tuning" [TMLR2024]

A RLHF Infrastructure for Vision-Language Models

😎 curated list of awesome LMM hallucinations papers, methods & resources.

[ICLR 2025] What do we expect from LMMs as AIGI evaluators and how do they perform?

Official Repo of Graphist

MLLM-Tool: A Multimodal Large Language Model For Tool Agent Learning

🌋👵🏻 Yo'LLaVA: Your Personalized Language and Vision Assistant

🤖 Discord AI assistant with OpenAI, Gemini, Claude & DeepSeek integration, multilingual support, multimodal chat, image generation, web search, and deep thinking | 一个强大的 Discord AI 助手，整合多种顶级 AI 模型，支持多语言、多模态交流、图片生成、联网搜索和深度思考

LLaVA inference with multiple images at once for cross-image analysis.

[CVPR 2025 🔥]A Large Multimodal Model for Pixel-Level Visual Grounding in Videos

[COLING 2025] Idea23D: Collaborative LMM Agents Enable 3D Model Generation from Interleaved Multimodal Inputs

GMAI-MMBench: A Comprehensive Multimodal Evaluation Benchmark Towards General Medical AI.

LMM solved catastrophic forgetting, AAAI2025