vllm-codex-proxy

A lightweight reverse proxy that lets Codex CLI use a locally-hosted vLLM model instead of a cloud API.

How it works

Codex CLI is OpenAI's open-source coding assistant. It supports custom model providers via its ~/.codex/config.toml. With a local-vllm provider configured to use the OpenAI Responses API (wire_api = "responses"), Codex sends all requests to the proxy, which forwards them to vLLM.

Codex CLI  (wire_api = "responses")
    │  primary: your model alias (e.g. "qwen3-coder")
    │  background: Claude model names (title generation, summaries, …)
    ▼
vllm-proxy  :4000          ← rewrites model names, fixes API incompatibilities
    │
    ▼
vLLM  :8080                ← serves your local model (e.g. Qwen3-Coder)

The model alias map must cover both:

• The primary model name you choose (e.g. qwen3-coder)
• Claude model names that Codex uses internally for background tasks (title generation, conversation summaries, etc.), regardless of which provider is configured

The proxy transparently forwards all requests and applies a set of fixes for known incompatibilities between the Codex CLI's use of the OpenAI Responses API and vLLM's implementation.

Why this exists

When Codex CLI (configured for the Anthropic provider) is pointed at vLLM, several issues arise:

Problem	Fix applied
text.format = {type: "json_schema"} sent without required schema field → vLLM 400	Rewrites to {type: "text"}
Model name sent by Codex (e.g. claude-sonnet-4-6) not known to vLLM	Configurable alias map
phase field on assistant messages rejected by vLLM schema	Stripped from history before forwarding
output_text content type in history messages rejected by some vLLM versions	Flattened to plain string
Extra fields (id, status) on function_call history items → vLLM 400	Stripped to allowed fields only
Double-encoded JSON arguments from Qwen3-Coder (arrays/numbers as JSON strings)	Decoded before forwarding
Concatenated JSON objects in arguments string → vLLM "Extra data" error	Trimmed to first valid object
Fake reasoning items sent back from Codex history → vLLM rejects (no decryption key)	Stripped
phase: null in output_item.done → Codex status bar disappears	Patched to "commentary"
Empty content: [] in output_item.done → model re-answers previous question	Accumulated delta text injected
vLLM skips output_item.done for text-only responses → Codex loses history	Synthesized from streamed text
No reasoning tokens from local models → no "Working…" status in Codex	One fake reasoning_summary_text.delta injected
wait_agent timeout too short (30 s default)	Bumped to 1 hour
read_file / write_file tool calls not supported by Codex executor	Emulated directly by the proxy

Requirements

• Python 3.10+
• aiohttp (pip install aiohttp)
• A running vLLM instance serving the OpenAI-compatible API

Installation

git clone https://github.com/florath/vllm-codex-proxy.git
cd vllm-codex-proxy
pip install -r requirements.txt

Quick start

1. Start the proxy

python3 vllm-proxy.py \
    --vllm-base http://localhost:8080 \
    --model-map models.json \
    --log-level 1

See models.example.json for the model map format.

2. Configure Codex CLI

Two separate files are involved — do not confuse them:

models.json (this repo) — used by the proxy to rewrite model names before forwarding to vLLM. Format: { "alias": "vllm-model-id" }. See models.example.json.

Codex model catalog — used by Codex CLI to learn the model's capabilities (context window size, tool support, reasoning levels, etc.). This is a Codex-specific JSON schema unrelated to the proxy's model map. Example:

{
  "models": [{
    "slug": "qwen3-coder",
    "display_name": "Qwen3-Coder (local vLLM)",
    "context_window": 98304,
    "auto_compact_token_limit": 80000,
    "tool_output_token_limit": 10000,
    "shell_type": "shell_command",
    "visibility": "list",
    "supported_in_api": true,
    "supports_reasoning_summaries": true,
    "supports_parallel_tool_calls": true,
    "input_modalities": ["text"],
    "default_reasoning_level": "medium",
    "supported_reasoning_levels": [
      {"effort": "low",    "description": "Fast responses"},
      {"effort": "medium", "description": "Balanced"},
      {"effort": "high",   "description": "Deep reasoning"}
    ]
  }]
}

Save this as e.g. ~/.codex/model-catalogs/qwen3-coder.json.

Then add a custom provider and profile to ~/.codex/config.toml:

[model_providers.local-vllm]
name     = "local-vllm"
base_url = "http://localhost:4000/v1"
wire_api = "responses"

[profiles.local]
model_provider     = "local-vllm"
model              = "qwen3-coder"
model_catalog_json = "/home/YOU/.codex/model-catalogs/qwen3-coder.json"

The wire_api = "responses" tells Codex to use the OpenAI Responses API, which is what vLLM exposes (and what this proxy speaks). The slug in the model catalog must match the model field in the profile.

3. Start Codex

codex --profile local --dangerously-bypass-approvals-and-sandbox

• --profile local — activates the profile from step 2, which points Codex at the proxy and selects your local model
• --dangerously-bypass-approvals-and-sandbox — skips the per-command approval prompts and sandbox restrictions; only use this if you trust the model and your environment

Configuration

All options can be set via CLI arguments. Environment variables are used as defaults when the corresponding CLI argument is not provided.

CLI argument	Env variable	Default	Description
--vllm-base URL	VLLM_BASE	http://localhost:8080	vLLM base URL
--port PORT	PORT	4000	Proxy listen port
--log-level N	LOG_LEVEL	0	Verbosity level (0–3)
--fake-thinking	FAKE_THINKING=1	on	Inject fake thinking events
--no-fake-thinking	FAKE_THINKING=0	—	Disable thinking injection
--model ALIAS=MODEL	—	—	Add a model alias (repeatable)
--model-map FILE	—	—	JSON file with alias → model mapping

Log levels

Level	Output
0	Silent (errors only)
1	One line per request: method, path, status, adaptations applied
2	+ request/response JSON bodies (non-streaming)
3	+ full SSE stream events (received and injected)

Model aliases

Two categories of model names arrive at the proxy:

• Primary model — whatever model is set to in the Codex profile (e.g. qwen3-coder)
• Internal Codex models — Claude model names that Codex uses for background tasks (title generation, summaries, etc.) regardless of your provider configuration

Every name Codex may send needs an entry in the model map.

Via CLI flags:

python3 vllm-proxy.py \
    --vllm-base http://localhost:8080 \
    --model claude-sonnet-4-6=Qwen/Qwen3-Coder-480B-A35B-Instruct \
    --model claude-haiku-4-5-20251001=Qwen/Qwen3-Coder-480B-A35B-Instruct \
    --model claude-opus-4-6=Qwen/Qwen3-Coder-480B-A35B-Instruct

Via a JSON file:

Copy models.example.json to models.json, edit as needed, then:

python3 vllm-proxy.py --vllm-base http://localhost:8080 --model-map models.json

models.json is in .gitignore so your local config is not accidentally committed.

The --model flags and --model-map can be combined; individual --model entries take precedence over the file.

Running as a systemd service

A template unit file is provided:

# Install
cp vllm-proxy.service ~/.config/systemd/user/vllm-proxy.service

# Edit the ExecStart line to match your setup
# (adjust --vllm-base, --port, --model-map path as needed)
$EDITOR ~/.config/systemd/user/vllm-proxy.service

# Enable and start
systemctl --user daemon-reload
systemctl --user enable --now vllm-proxy.service
systemctl --user status vllm-proxy.service

Tested with

• Codex CLI (openai/codex)
• vLLM ≥ 0.8 with --tool-call-parser qwen3_coder
• Qwen3-Coder models (AWQ quantized and full precision)

Author

Andreas Florath andreas@florath.net

License

MIT — see LICENSE.