local mlx responsesAPI server.stub

---

id: local-mlx-responsesAPI-server name: MLX Local Inference Server repo_path: /sessions/charming-sleepy-ritchie/repos/local-mlx-responsesAPI-server remote_url: https://github.com/sw30labs/local-mlx-responsesAPI-server.git primary_language: Python framework: FastAPI + MLX ingested: 2026-04-11 last_commit_date: 2026-04-11

MLX Local Inference Server

Purpose

OpenAI-compatible local inference server for Apple Silicon. Drop-in replacement emulating OpenAI Chat Completions, OpenAI Responses API, Azure OpenAI Chat/Responses, and Anthropic Messages API. Full tool/function calling support.

API Surfaces Supported

1. OpenAI Chat Completions
2. OpenAI Responses API
3. Azure OpenAI Chat Completions
4. Azure OpenAI Responses API
5. Anthropic Messages API

All with full tool/function calling support.

Architecture

• FastAPI Application: mlx_server/ directory
• Model Holder: Singleton for model state management
• Request/Response Models: Pydantic schemas matching each API
• Tool Call Extraction: Multi-format support (JSON, tagged)
• Inference Lock: GPU state protection for concurrent requests
• Conversation Store: In-memory TTL-based conversation history

Key Technologies

• Framework: FastAPI, Uvicorn
• Inference: MLX, mlx-lm
• API Compatibility: OpenAI, Azure, Anthropic formats

Dependencies

mlx, mlx-lm, fastapi, uvicorn, pydantic

Tags

python, mlx, apple-silicon, local-inference, fastapi, openai, azure, anthropic, llm

Cross-references

• deepagent-azure-cli — uses as local backend
• tars-ai — shared MLX inference