Model AI

A Docker Desktop extension that turns locally-pulled Docker Hub models into a drop-in replacement for the Azure AI, Google Gemini, and OpenAI APIs — no cloud keys, no rate limits, no data leaving your machine.

---

What it does

Interface	Base URL (external)	Auth header expected
Azure AI	http://localhost:8080/openai/deployments/{model}	api-key: <any>
Gemini	http://localhost:8080/v1beta/models/{model}	x-goog-api-key: <any>
OpenAI	http://localhost:8080/v1	Authorization: Bearer <any>

Point any SDK or application at localhost:8080 with its normal configuration and it will talk to a local Docker Model Runner model instead of the cloud. Auth values are accepted but ignored — the backend does not validate them.

---

Prerequisites

• Docker Desktop 4.40+ with Docker Model Runner enabled
• At least one model pulled: docker model pull ai/smollm2

---

Installation

From source

# Build the extension image and install it into Docker Desktop
docker build -t model-ai:latest .
docker extension install --force model-ai:latest

Or, if make is available:

make install

Update after code changes

docker build -t model-ai:latest .
docker extension update --force model-ai:latest

make update   # on Unix / WSL

---

How to use

1. Pull a model

Before using the extension, pull at least one model from Docker Hub:

docker model pull ai/smollm2
# or a larger model:
docker model pull ai/deepseek-r1-distill-llama

Models appear automatically in the Model dropdown once pulled.

2. Select a provider and model

When the extension opens, Azure AI is selected by default. Click any provider chip — Azure AI, Gemini, or OpenAI — to switch. The entire interface (endpoint URL, request body, and code snippets) updates instantly to match that provider's API format.

Use the Model dropdown to choose which locally-pulled model handles the request.

A GPU / CPU status chip appears to the right of the Model dropdown showing whether Docker Model Runner has hardware GPU acceleration active:

Chip	Meaning
Green GPU	Metal (Mac), CUDA, or DirectML acceleration active
Amber CPU	Running on CPU only — hover for instructions to enable GPU

If you see CPU and want GPU acceleration, go to Docker Desktop → Settings → Resources → GPU and enable GPU support, then restart Docker Desktop. On Mac Apple Silicon, also confirm Settings → General → Use Virtualization framework is enabled.

3. Playground tab — interactive chat

The Playground is an interactive multi-turn chat. Type a message and press Send. Each reply is appended to the thread and the full conversation history is included with every request, giving the model context across turns.

Switch providers or models at any point — the next message will use the newly selected configuration while preserving the chat history on screen.

4. Test tab — API console

The Test tab is a Postman-style console for validating that your application code will work before you write it. It has three sub-tabs:

Request

• Temperature slider and Max tokens field tune the sampling parameters; the JSON body reseeds automatically when they change.
• Stream toggle (next to Max tokens) enables streaming mode: OpenAI and Azure AI bodies get "stream": true; Gemini switches to the :streamGenerateContent endpoint. The Code tab snippets update to use each SDK's streaming iteration API. Note: the in-extension Response tab shows the raw SSE event stream text when streaming is on — use the Code tab snippets to handle streaming properly in your application.
• Headers shows the exact headers for the selected provider.
• Body is the raw JSON sent to the backend — edit it freely before hitting Send.

Response

Shows the HTTP status code, round-trip latency, and the full JSON response. Use the copy button to grab the payload.

Code

Generates copy-ready curl, Python, and JavaScript snippets using each provider's official SDK, already wired to localhost:8080. Switch language tabs with a single click.

5. Sending requests and switching providers mid-flight

Click Send to fire the request. You can switch to a different provider immediately — the Send button becomes available right away rather than waiting for the previous request to complete. When the in-flight response arrives it is automatically added to the history sidebar regardless of which provider is currently active, so no results are lost.

6. History sidebar

The history sidebar (right side of the Test tab) keeps the last 20 calls across all providers. Each entry shows:

• Status chip (green = success, red = error)
• Provider name and model
• Round-trip latency
• Timestamp

Click any entry to reload its request body and response into the console. History is persisted in browser storage — navigating away from the extension (e.g. to the Docker Desktop Models view) and returning preserves the full history.

To remove a single entry click the × on that item. To clear everything click the delete sweep icon at the top of the sidebar.

---

Architecture

Docker Desktop
├── UI (React + MUI + Vite)          ← runs in the extension webview
│   ├── Playground tab               — interactive multi-turn chat
│   └── Test tab                     — request editor, response, code snippets, history
│
└── Backend (Go)                     ← runs in the VM service container
    ├── Unix socket  /run/guest-services/model-ai.sock   (UI ↔ backend)
    ├── TCP          :8080                                (external apps)
    │
    ├── /v1/models                GET   — list pulled models
    ├── /v1/chat/completions      POST  — OpenAI-compatible chat
    ├── /openai/deployments/*     POST  — Azure-compatible chat
    └── /v1beta/models/*          POST  — Gemini-compatible chat

The UI never makes direct HTTP calls; it goes through ddClient.extension.vm.service which Docker Desktop proxies over the Unix socket. External applications use the published TCP port (8080).

---

MCP server

The backend doubles as an MCP stdio server, so AI agents and tools that support MCP can list, chat with, and pull local models directly.

// .mcp.json (Claude Code auto-discovery)
{
  "mcpServers": {
    "model-ai": {
      "type": "stdio",
      "command": "go",
      "args": ["run", ".", "--mcp"],
      "cwd": "${workspaceFolder}/backend"
    }
  }
}

Tool	Description
list_models	Returns the list of locally pulled models
chat	Sends a prompt to a model and returns the reply
pull_model	Runs docker model pull <model>

Run it directly:

make dev-mcp
# or
cd backend && go run . --mcp

---

Development

UI (hot-reload, proxies to Go backend)

make dev-ui
# or
cd ui && npm run dev        # http://localhost:3000

The Vite dev server proxies /v1, /openai, and /v1beta to :8080. Start the backend first.

Backend (local, requires Docker Model Runner)

make dev
# or
cd backend && go run .     # listens on :8080 (TCP) and the Unix socket

Tests

Go is not installed locally; tests run inside a throwaway container with a persistent module cache:

.\test.ps1                  # run all tests
.\test.ps1 -v ./handlers    # verbose, single package

First run: ~15 s (downloads modules). Warm runs: ~2 s (cached via Docker volumes model-ai-gomod and model-ai-gobuild).

---

Project layout

model-ai/
├── Dockerfile               multi-stage: UI → Go binaries → alpine image
├── docker-compose.yaml      VM service definition (used by Docker Desktop at install)
├── metadata.json            extension manifest
├── docker.svg               extension icon
├── Makefile                 dev, build, install, update, uninstall
├── test.ps1                 run Go tests via Docker (Windows)
├── .mcp.json                MCP server declaration for Claude Code
│
├── backend/
│   ├── main.go              HTTP server (Unix socket + TCP) + MCP flag
│   ├── router/              chi router — all three provider route trees
│   ├── handlers/            openai.go · azure.go · gemini.go
│   ├── proxy/               model_runner.go — forwards to Docker Model Runner
│   ├── mcp/                 server.go · tools.go — MCP stdio server
│   └── models/              shared request/response types
│
└── ui/
    ├── src/
    │   ├── App.tsx           top-level layout, tabs, theme
    │   ├── api.ts            ddClient wrappers (fetchModels, sendChat, sendRaw)
    │   ├── testRequests.ts   request builders + code-snippet generators
    │   ├── types.ts          Provider, LocalModel, ChatMessage
    │   └── components/
    │       ├── ModelSelector.tsx    provider chips + model dropdown
    │       ├── ChatPlayground.tsx   interactive multi-turn chat
    │       └── TestConsole.tsx      request editor, response, snippets, history
    └── vite.config.ts

---

API reference

GET /v1/models

Returns the list of models currently available in Docker Model Runner.

{
  "object": "list",
  "data": [
    { "id": "ai/smollm2:latest", "object": "model", "owned_by": "docker" }
  ]
}

POST /v1/chat/completions (OpenAI)

{
  "model": "ai/smollm2:latest",
  "messages": [{ "role": "user", "content": "Hello!" }],
  "temperature": 0.7,
  "max_tokens": 1024,
  "stream": false
}

POST /openai/deployments/{model}/chat/completions?api-version=2024-02-01 (Azure AI)

Same body as OpenAI, minus the model field (the deployment path segment carries it). The api-version parameter is accepted and ignored.

{
  "messages": [{ "role": "user", "content": "Hello!" }],
  "temperature": 0.7,
  "max_tokens": 1024
}

POST /v1beta/models/{model}:generateContent (Gemini)

{
  "contents": [{ "role": "user", "parts": [{ "text": "Hello!" }] }],
  "generationConfig": { "temperature": 0.7, "maxOutputTokens": 1024 }
}

Note on model IDs: Docker model IDs contain slashes and colons (e.g. docker.io/ai/smollm2:latest). All three routes capture the model as a wildcard path segment, so these IDs route correctly.

Streaming

Set "stream": true (OpenAI / Azure AI) or use the :streamGenerateContent endpoint (Gemini). The backend pipes the raw SSE stream from Model Runner to the caller without buffering.

In the extension UI, toggle Stream in the Test tab's Request sub-tab. The Code tab snippets update automatically to use streaming SDK APIs for the selected provider and language.

GPU / CPU detection

Docker Model Runner does not expose engine/runtime metadata over its HTTP API, so the UI reads it from the host CLI instead. On load the extension runs docker model status --json (via ddClient.docker.cli.exec) and inspects the running backend string:

{
  "running": true,
  "backends": {
    "llama.cpp": "Running: llama.cpp latest-cuda (sha256:...) 65ef50a"
  }
}

A backend tag containing an accelerator marker (cuda, metal, vulkan, rocm, …) lights the green GPU chip; otherwise (latest / latest-cpu) it shows the amber CPU chip.

---

License

MIT