Vision

Let your AI agent see. Vision gives agents like Codex and Claude Code the ability to understand images and PDFs through an external, OpenAI-compatible vision API — even when the active model has no native vision support.

English · 简体中文 · 繁體中文

---

Table of Contents

• Features
• Quick Start
• Installation
• MCP Server
• Configuration
• Usage
• How It Works
• Troubleshooting
• Updating
• Project Structure
• License

Features

• Multi-format — understand images (files, paths, URLs) and PDFs (text-based and scanned).
• No OCR needed — text-based PDFs are extracted locally by pdf-inspector; scanned pages route to the vision API.
• Any OpenAI-compatible endpoint — defaults to Alibaba Cloud DashScope; bring your own VISION_BASE_URL.
• Model fallbacks — tries VISION_MODEL, then each model in VISION_MODEL_FALLBACKS in order.
• Two ways to use it — as a Codex/Claude skill, or as an MCP server with the same tools.
• Utility commands — --models (browse available models) and --check (validate setup).

A skill is a folder of instructions and scripts that extends an AI agent (such as Codex or Claude Code) with a new capability.

Quick Start

Option A — use it as a skill. Send this message to your AI agent:

Install the Vision skill from repo DLeungDL/vision at path skills/vision.

Option B — use it as an MCP server.

git clone https://github.com/DLeungDL/vision
cd vision/mcp
npm install

Then register node /path/to/vision/mcp/server.js in your MCP client. See MCP Server.

Either way, finish by adding your API key — see Configuration.

Installation

The skill lives in the skills/vision folder of this repository. Pick one of the two options below.

Option 1 — Let the AI agent install it (easiest)

Send this message to your AI agent:

Install the Vision skill from repo DLeungDL/vision at path skills/vision.

The agent clones the repository, then copies or links the skills/vision folder into your agent's skills directory (e.g. ~/.codex/skills/vision). The skill becomes available on the next turn — restart the agent if it does not show up.

Option 2 — Install it yourself

macOS / Linux

# 1. Clone this repository
git clone https://github.com/DLeungDL/vision ~/Developer/vision
cd ~/Developer/vision

# 2. Link the skill folder into your agent's skills directory
ln -sfn ~/Developer/vision/skills/vision ~/.codex/skills/vision        # Codex
# ln -sfn ~/Developer/vision/skills/vision ~/.claude/skills/vision     # Claude Code

# 3. Create the .env file with your vision API key
cp skills/vision/scripts/.env.example skills/vision/scripts/.env
$EDITOR skills/vision/scripts/.env        # set VISION_API_KEY=...

Notes:

• Keep the ~/Developer/vision clone — the symlink points to it, so removing it breaks the skill. With a symlink, git pull updates the skill automatically; a copy would need to be re-copied manually.
• The skill becomes available on the next turn; restart the AI agent if it is not picked up.

Windows (PowerShell)

# 1. Clone this repository
git clone https://github.com/DLeungDL/vision "$HOME\Developer\vision"
Set-Location "$HOME\Developer\vision"

# 2. Copy the skill folder into your agent's skills directory
New-Item -ItemType Directory -Force "$HOME\.codex\skills"
Copy-Item -Recurse "$HOME\Developer\vision\skills\vision" "$HOME\.codex\skills\vision"

# 3. Create the .env file with your vision API key
Copy-Item skills\vision\scripts\.env.example skills\vision\scripts\.env
notepad skills\vision\scripts\.env        # set VISION_API_KEY=...

Notes:

• Copying works without admin rights. After pulling updates, re-copy the folder to refresh the skill: Copy-Item -Recurse "$HOME\Developer\vision\skills\vision" "$HOME\.codex\skills\vision".
• Want automatic updates? Use a symbolic link instead (advanced): replace step 2 with New-Item -ItemType SymbolicLink -Path "$HOME\.codex\skills\vision" -Target "$HOME\Developer\vision\skills\vision". This requires administrator PowerShell, or Developer Mode enabled in Windows settings (Settings > Privacy & security > For developers). Keep the clone in place — removing it breaks the skill.

MCP Server

The repository also includes an MCP (Model Context Protocol) server that exposes the same capabilities as tools for MCP-compatible clients (Claude Desktop, Codex, Cursor, etc.):

Tool	Description
describe_image	Describe or analyze an image (file path or URL)
list_vision_models	List models available on the configured endpoint
check_setup	Validate configuration and API connectivity
pdf_to_markdown	Convert a PDF (file path or URL) to Markdown
detect_pdf_type	Classify a PDF as text-based, scanned, image-based, or mixed
pdf_extract_text	Extract the plain text layer of a PDF
describe_pdf_pages	Render scanned PDF pages to images and describe them with the vision API

Setup and register:

cd mcp
npm install

Then add it to your MCP client using the stdio transport (replace <repo> with the absolute path of this repository):

{
  "mcpServers": {
    "vision": {
      "command": "node",
      "args": ["<repo>/mcp/server.js"]
    }
  }
}

Configuration is shared with the skill — set the VISION_* environment variables, or create skills/vision/scripts/.env from .env.example. See mcp/README.md for client-specific examples, the full pipeline, and the test suite.

PDF extraction is powered by pdf-inspector (MIT, © Firecrawl). Third-party licenses: THIRD_PARTY_NOTICES.md.

Configuration

Set these environment variables, or create a .env file next to scripts/vision.js:

Variable	Required	Default	Description
VISION_API_KEY	Yes	-	API key for the vision service
VISION_BASE_URL	No	https://dashscope.aliyuncs.com/compatible-mode/v1	OpenAI-compatible base URL
VISION_MODEL	No	qwen-vl-max	Vision model to call
VISION_MODEL_FALLBACKS	No	(empty)	Comma-separated fallback models tried in order

Example for Alibaba Cloud DashScope:

export VISION_API_KEY="sk-..."
export VISION_BASE_URL="https://dashscope.aliyuncs.com/compatible-mode/v1"
export VISION_MODEL="qwen-vl-max"

⚠️ Do not commit .env — keep your API key out of git.

Usage

Run the script from the skill folder (skills/vision) or pass its full path:

node scripts/vision.js "/path/to/image.png" "Describe this image"
node scripts/vision.js --url "https://example.com/image.jpg" "What is in this photo?"
node scripts/vision.js --models    # list models available on the endpoint
node scripts/vision.js --check     # validate configuration and API connectivity
node scripts/vision.js --mode=ocr "/path/to/screenshot.png"   # extract text from an image
node scripts/vision.js --mode=table "/path/to/table.png"      # table to Markdown
node scripts/vision.js --mode=ui "/path/to/ui.png"            # UI/UX analysis
node scripts/vision.js --mode=json "/path/to/image.png" --schema '{"type":"object"}'   # JSON output
node scripts/vision.js --mode=compare "a.png" "b.png"         # compare two images
node scripts/vision.js --mode=batch "./folder"                # summarize a folder of images

• Local images are base64-encoded; remote URLs are passed through unchanged.
• For multiple images, run the script once per image.
• Modes: describe (default), ocr, table, chart, ui, json, alt, batch, compare — pass --mode=<name> to use a built-in prompt.

How It Works

1. You share an image file, path, or URL.
2. The AI agent runs node scripts/vision.js "<path-or-url>" "[question]".
3. vision.js base64-encodes local images (or passes URLs through unchanged) and calls an OpenAI-compatible chat/completions API.
4. The vision model's text description is returned to the AI agent.

For PDFs, the flow is: classify first, then route — text-based pages are extracted locally, scanned pages are described by the vision API. See the pipeline in mcp/README.md.

Troubleshooting

• VISION_API_KEY missing or authentication fails — create skills/vision/scripts/.env from .env.example, set VISION_API_KEY, then re-run.
• Nothing works after setup — run node scripts/vision.js --check to validate the endpoint and model.
• Wrong model or model failures — set VISION_MODEL (and VISION_MODEL_FALLBACKS as a comma-separated fallback list).
• --check fails with API 404 — some endpoints do not implement GET /models; this is expected and can be ignored.

Updating

• Symlink install (macOS/Linux or Windows Developer Mode): pull the latest version and the skill updates automatically: git -C ~/Developer/vision pull.
• Copy install: re-run the copy command after pulling updates.

Project Structure

vision/
├── skills/vision/                 # The skill (install this folder)
│   ├── SKILL.md                   # Instructions for the AI agent
│   ├── scripts/
│   │   ├── vision.js              # Core script (OpenAI-compatible)
│   │   └── .env.example           # Template for your API configuration
│   ├── references/setup.md        # First-time setup guide
│   └── agents/openai.yaml         # UI metadata for the skill
├── mcp/                           # MCP server (optional)
│   ├── server.js                  # MCP server exposing the same tools
│   ├── package.json
│   ├── README.md                  # MCP docs: setup, pipeline, tests
│   └── test/                      # Test suite (npm test)
├── THIRD_PARTY_NOTICES.md         # Third-party licenses
└── LICENSE                        # MIT

License

Released under the MIT License.