feat: OpenAI-compatible /v1/chat/completions endpoint

[just-cameron]

Goal

Add an OpenAI-compatible POST /v1/chat/completions endpoint (plus GET /v1/models) to lettabot's existing HTTP API server. This enables:

• Web chat UIs (Open WebUI, Chatbot UI, LibreChat) to connect directly
• Developers to use standard OpenAI client libraries (openai Python/JS SDK, LangChain, etc.)
• Any tool that speaks the OpenAI API format to interact with lettabot agents

Scope

In scope

• POST /v1/chat/completions -- sync and streaming (SSE)
• GET /v1/models -- list available agents as "models"
• Authorization: Bearer <key> auth (OpenAI convention), alongside existing X-Api-Key
• Stream tool calls and reasoning as OpenAI-format deltas (full visibility)
• Response shapes that pass validation by the openai Python SDK

Out of scope (future work)

• Token usage counting (no access to token counts from the SDK; return null for now)
• tools / tool_choice request parameters (lettabot's tools are agent-defined, not caller-defined)
• temperature, max_tokens, etc. (model params are agent-configured, not per-request)
• Multi-turn messages history (lettabot has its own conversation memory; we extract the last user message)
• Image/audio content parts in messages
• /v1/completions (legacy, non-chat)

Design

Routing

Two new routes in src/api/server.ts:

POST /v1/chat/completions   -- main chat endpoint
GET  /v1/models              -- list agents as models

These live alongside the existing /api/v1/chat endpoint (which remains for backward compat).

Authentication

Accept both:

• Authorization: Bearer <key> (OpenAI convention -- what all clients send)
• X-Api-Key: <key> (existing lettabot convention)

Same LETTABOT_API_KEY, same timing-safe comparison.

Request handling (POST /v1/chat/completions)

Request body (minimum viable):

{
  "model": "lettabot",
  "messages": [
    { "role": "user", "content": "Hello" }
  ],
  "stream": false
}

Behavior:

1. Validate auth (Bearer or X-Api-Key)
2. Parse JSON body
3. Extract last user role message from messages[] as the prompt
4. Resolve agent name from model field (map to agent names via getAgentNames())
5. If stream: true -> use streamToAgent(), emit SSE chunks in OpenAI format
6. If stream: false (default) -> use sendToAgent(), return full completion object

Sync response:

{
  "id": "chatcmpl-<uuid>",
  "object": "chat.completion",
  "created": 1707850000,
  "model": "lettabot",
  "choices": [{
    "index": 0,
    "message": {
      "role": "assistant",
      "content": "Hello! How can I help?"
    },
    "finish_reason": "stop"
  }],
  "usage": null
}

Streaming response (SSE):

Each chunk follows OpenAI's chat.completion.chunk format:

data: {"id":"chatcmpl-<uuid>","object":"chat.completion.chunk","created":1707850000,"model":"lettabot","choices":[{"index":0,"delta":{"role":"assistant"},"finish_reason":null}]}

data: {"id":"chatcmpl-<uuid>","object":"chat.completion.chunk","created":1707850000,"model":"lettabot","choices":[{"index":0,"delta":{"content":"Hello"},"finish_reason":null}]}

data: {"id":"chatcmpl-<uuid>","object":"chat.completion.chunk","created":1707850000,"model":"lettabot","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: [DONE]

Tool call streaming:

When the agent calls tools, emit them as OpenAI tool_call deltas:

data: {"choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"id":"call_xxx","type":"function","function":{"name":"web_search","arguments":""}}]},"finish_reason":null}]}

data: {"choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"{\"query\":\"...\"}"}}]},"finish_reason":null}]}

Tool results are internal to the agent -- skip them in the stream and just continue with the final assistant response text.

Reasoning: Skip in v1 (no standard OpenAI field). Can revisit based on UI needs.

Models endpoint (GET /v1/models)

{
  "object": "list",
  "data": [
    {
      "id": "lettabot",
      "object": "model",
      "created": 1707850000,
      "owned_by": "lettabot"
    }
  ]
}

Each configured agent appears as a separate "model". The id is the agent name from config.

SDK stream message -> OpenAI chunk mapping

SDK message type	OpenAI mapping
assistant (streaming text)	delta.content chunks
tool_call	delta.tool_calls[] with function name and arguments
tool_result	Skip (internal)
reasoning	Skip in v1 (no standard OpenAI field)
result	Final chunk with finish_reason: "stop", then data: [DONE]

File structure

• src/api/server.ts -- add routes (or extract to a router pattern)
• src/api/types.ts -- add OpenAI-format type definitions
• src/api/openai-compat.ts (new) -- mapping functions: SDK messages -> OpenAI chunks, request parsing, response building
• src/api/auth.ts -- add Bearer token extraction alongside X-Api-Key

Given server.ts is already getting large, consider extracting the OpenAI compat routes into their own handler module.

Configuration

No new config needed. Uses existing:

• LETTABOT_API_KEY / server.api.key for auth
• Agent names from config for model list
• Existing server.api.port and server.api.host

Testing

• Unit tests for request parsing (extract last user message from messages array)
• Unit tests for response building (SDK message -> OpenAI chunk mapping)
• Unit tests for Bearer token auth extraction
• Manual test: curl with OpenAI format
• Manual test: openai Python SDK pointing at lettabot
• Manual test: Open WebUI connected to lettabot

Implementation order

1. Add Bearer token support to auth
2. Create src/api/openai-compat.ts with type definitions and mapping functions
3. Add GET /v1/models route
4. Add POST /v1/chat/completions sync route
5. Add streaming support to the chat completions route
6. Add tool call streaming mapping
7. Tests

Critical files

• src/api/server.ts -- existing routes and patterns
• src/api/auth.ts -- existing auth
• src/api/types.ts -- existing API types
• src/core/interfaces.ts -- AgentRouter interface (sendToAgent, streamToAgent, getAgentNames)
• src/core/gateway.ts -- how streamToAgent yields SDK messages

Written by Cameron and Letta Code

"Make it work, make it right, make it fast." -- Kent Beck