Skip to content

LLM Available

Jump to bottom
Leslie Drewery edited this page Sep 18, 2026 · 1 revision

`

The landscape of Vision-Language Models (VLMs) is broadly divided into proprietary API-driven models and open-weight/open-source models that can be self-hosted.

1. Proprietary Frontier Models

Best for state-of-the-art accuracy, massive context windows, complex reasoning, and minimal infrastructure setup.

  • Gemini 2.5 Pro & Gemini 3: Industry leaders for long-context multimodal understanding. Excels at high-resolution image analysis, long-form video, native audio processing, and spatial UI/screen agents.

  • GPT-4o & GPT-5 Vision: High generalist accuracy across chart parsing, document understanding, and real-time visual reasoning with fast inference speeds.

  • Claude 3.5 Sonnet & Claude Opus 4: Strongest models for precise instruction following, multi-page PDF processing, bounding-box detection, and spatial layout extraction.

    Codesota

2. Leading Open-Weight & Self-Hosted Models

Best for full data privacy, low-latency deployment, edge computing, or fine-tuning on custom datasets.

  • Qwen2.5-VL / Qwen3-VL Series:

    • Sizes: 3B, 7B, 32B, 72B

      SiliconFlow

    • Strengths: Widely considered the best open-weight VLM family for Document VQA (invoices, forms), 7-segment display extraction, screen control, and spatial bounding-box predictions.

  • InternVL3 Series (78B):

    • Strengths: Top performance among large open-source models, outperforming many proprietary models on scientific diagrams, complex spatial reasoning, and 3D perception.

      DataCamp

  • Gemma 3 (Google DeepMind):

    • Sizes: 1B, 4B, 12B, 27B

      Dextra Labs

    • Strengths: Lightweight open-weights model fine-tuned for high-speed edge deployment, multilingual OCR, and mobile applications.

  • Llama 3.2 Vision & Llama 4 (Meta):

    • Sizes: 11B, 90B+

      DataCamp

    • Strengths: Broad community support, strong reasoning over mixed text/image inputs, and easily fine-tunable on platforms like Hugging Face.

  • GLM-4.5V / GLM-4.1V-Thinking (Zhipu AI):

    • Strengths: Introduces reasoning-first "thinking" architectures for step-by-step visual problem solving and complex document navigation.

      SiliconFlow

  • Molmo (Allen Institute for AI):

    • Sizes: 1B, 7B, 72B

      Medium

    • Strengths: Features native pixel-pointing capabilities, making it ideal for visual agent control, UI interaction, and object pinpointing.

Quick Selection Guide

Goal Recommended Model
Easiest Integration & API GPT-4o, Claude 3.5 Sonnet, or Gemini 2.5 Pro
Self-Hosted Document OCR / Prices Qwen2.5-VL-7B or Qwen2.5-VL-32B
Edge / Mobile Deployment Gemma 3 (1B/4B) or Phi-4 Multimodal
Video & Audio Analysis Gemini 2.5 Pro
Want to know how to deploy Qwen2.5-VL or Gemma 3 locally using vLLM or Ollama?
Yes
`

Clone this wiki locally