mcp-vision-server

A lightweight MCP server exposing a single tool — describe_image — that forwards a local image file to any OpenAI-compatible vision endpoint and returns the text description.

Most LLMs can't see an image you drop into a chat. This server bridges that gap: the host AI calls describe_image with a file path, your vision model does the actual seeing, and the description flows back into the conversation.

┌──────────┐     MCP (stdio)      ┌─────────────────┐      HTTP POST     ┌──────────────────┐
│  Client  │ ─── describe_image ─▶│  mcp-vision-srv │ ── image+prompt ──▶│  Vision Endpoint │
│  (AI)    │ ◀── text description │   (this repo)   │ ◀──── JSON resp ───│  (vLLM/Ollama/…) │
└──────────┘                      └─────────────────┘                    └──────────────────┘

The server speaks MCP over stdio — it doesn't serve HTTP itself. It reads the file from disk, base64-encodes it, POSTs to your endpoint, and hands the response back as tool output.

Requirements

• Python ≥ 3.10
• uv installed
• A running vision endpoint that speaks the OpenAI chat-completions schema

That last one is not optional. This server has no model of its own — without something listening at VISION_ENDPOINT, every call fails. See Supported Endpoints.

Install

No clone needed — uvx can run it straight from the repo:

uvx --from git+https://github.com/joshsssn/mcp-vision-server mcp-vision-server

Your MCP client will run this for you once configured; the command above is mainly useful to check the server starts.

Claude Code, one line:

claude mcp add vision \
  --env VISION_ENDPOINT=http://localhost:11434/v1/chat/completions \
  --env VISION_MODEL=llama3.2-vision \
  -- uvx --from git+https://github.com/joshsssn/mcp-vision-server mcp-vision-server

VS Code (Copilot Chat) — add to .vscode/mcp.json in your workspace, or to your user settings:

{
  "servers": {
    "mcp-vision-server": {
      "type": "stdio",
      "command": "uvx",
      "args": [
        "--from",
        "git+https://github.com/joshsssn/mcp-vision-server",
        "mcp-vision-server"
      ],
      "env": {
        "VISION_ENDPOINT": "https://api.openai.com/v1/chat/completions",
        "VISION_API_KEY": "sk-your-key-here",
        "VISION_MODEL": "gpt-4o"
      }
    }
  }
}

Claude Desktop — add to claude_desktop_config.json:

{
  "mcpServers": {
    "mcp-vision-server": {
      "command": "uvx",
      "args": [
        "--from",
        "git+https://github.com/joshsssn/mcp-vision-server",
        "mcp-vision-server"
      ],
      "env": {
        "VISION_ENDPOINT": "https://api.openai.com/v1/chat/completions",
        "VISION_API_KEY": "sk-your-key-here",
        "VISION_MODEL": "gpt-4o"
      }
    }
  }
}

Working from a local clone instead? Replace the git+https://… argument with the path to the repo root.

Local development

git clone https://github.com/joshsssn/mcp-vision-server.git
cd mcp-vision-server
cp .env.example .env    # edit with your endpoint URL, API key, and model name
uvx --from . mcp-vision-server

Configuration

All settings come from environment variables, or a .env file in the project root:

Variable	Default	Description
VISION_ENDPOINT	http://localhost:11434/v1/chat/completions	OpenAI-compatible vision API URL
VISION_API_KEY	ollama	Bearer token for the endpoint
VISION_MODEL	llama3.2-vision	Model name the endpoint expects
VISION_TIMEOUT	60	Request timeout in seconds
VISION_MAX_IMAGE_BYTES	20971520 (20 MB)	Reject images larger than this

The defaults target a local Ollama install, so ollama run llama3.2-vision plus a zero-config client entry is enough to get going.

The env block in your client config overrides .env. Use .env for local dev, the client block for anything you share — no keys end up in the repo either way.

See .env.example for a ready-to-copy template.

Tool: describe_image

Parameter	Type	Required	Default
image	string	yes	—
prompt	string	no	Describe this image in detail.

image must be an absolute path to a file on disk — not a URL, not a pasted image. Supported formats: PNG, JPG, JPEG, GIF, WEBP.

C:\Users\photos\cat.jpg
/home/user/images/screenshot.png

prompt is any natural-language instruction for the vision model:

• "Describe this image in detail."
• "Extract all text from this image."
• "What colors dominate this image?"
• "Is there a person in this image? If so, describe them."

Example call:

{
  "tool": "describe_image",
  "arguments": {
    "image": "/home/user/photos/sunset.jpg",
    "prompt": "What colors dominate this image? Is there a person in it?"
  }
}

Supported Endpoints

Anything implementing the OpenAI chat-completions schema with image_url content parts:

Endpoint	Example URL	Notes
Ollama	http://localhost:11434/v1/chat/completions	Easiest local setup
LM Studio	http://localhost:1234/v1/chat/completions	GUI-friendly
vLLM	http://localhost:8000/v1/chat/completions	Great for self-hosted
OpenAI	https://api.openai.com/v1/chat/completions	Cloud, requires API key
Custom / self-hosted	https://your-box/v1/chat/completions	Anything OpenAI-compatible

Troubleshooting

invalid peer certificate: UnknownIssuer on startup. uv uses its own certificate store and doesn't know about TLS-inspecting antivirus software or corporate proxies. Add "UV_NATIVE_TLS": "1" to the env block to make it use the system store instead.

Connection refused / timeout on every call. Nothing is listening at VISION_ENDPOINT. Confirm the endpoint independently before blaming the server:

curl $VISION_ENDPOINT -H "Content-Type: application/json" \
  -d '{"model":"llama3.2-vision","messages":[{"role":"user","content":"hi"}]}'

Tool doesn't appear in the client. Check the MCP logs — in VS Code, the Output panel has a channel per server. A failure to install the package shows up as a non-zero exit before initialize ever completes.

Image rejected for size. Raise VISION_MAX_IMAGE_BYTES, or downscale first. Base64 encoding inflates the payload by roughly a third, so the limit is deliberately conservative.

Project Structure

mcp-vision-server/
├── pyproject.toml
├── .env.example
├── .gitignore
├── README.md
├── LICENCE
└── src/
    └── mcp_vision_server/
        ├── __init__.py
        └── server.py

License

MIT — do whatever.