L2dLLM

An OpenAI-compatible LLM backend with a switchable persona library and LangGraph-powered tool calling. Designed to be plugged into any OpenAI client (Open WebUI, LobeChat, the official openai SDK, …) as the model provider.

Architecture

For a system overview - component map, data flow, and scope boundaries - see ARCHITECTURE.md. Per-module design rationale lives in specs/DESIGN.md.

Installation

Requires uv.

git clone git@github.com:Me9uru/L2dLLM.git
cd L2dLLM
uv sync

Configuration

cp .env.example .env
# Edit .env: set L2DLLM_API_KEY (required), L2DLLM_MODEL, L2DLLM_BASE_URL, ...

Every setting is a L2DLLM_* environment variable loaded from .env. Real environment variables override .env, so you can also just export L2DLLM_API_KEY=... (or OPENAI_API_KEY-style global keys are not read — set L2DLLM_API_KEY). Only L2DLLM_API_KEY is required.

Running the server

uv run l2dllm                          # 127.0.0.1:8000
uv run l2dllm --host 0.0.0.0 --port 9000
uv run l2dllm --env /path/to/.env       # alternate .env file
uv run l2dllm --persona cat            # promote 'cat' to be the default model

The server exposes the OpenAI Chat Completions surface at /v1:

• GET /v1/models — lists default plus one entry per persona.
• POST /v1/chat/completions — streaming (SSE) and non-streaming. With tts=true (and TTS configured), streaming responses also emit delta.audio chunks — one base64 WAV per synthesized speech segment.
• POST /v1/audio/speech — direct Chinese text -> Japanese WAV (TTS smoke-test endpoint; 501 if TTS is unconfigured).
• POST /v1/live2d/expressions — pick Live2D expression cues matching the assistant's last reply (503 without an API key, 502 on LLM failure).

Each persona becomes a model id; selecting the model in your UI is how you switch character.

Personas

Drop markdown files under ./personas/. Each is a YAML-frontmatter + markdown-body card:

---
name: cat
description: A playful cat-girl persona.
---

你是一只爱卖萌的猫娘，名叫小喵…

name must match [A-Za-z0-9_-]+. The body becomes the system prompt for that model id. Sample personas (cat, assistant, flandre) ship in ./personas/.

Skills

Drop markdown files under ./skills/ to expose them as on-demand tools the model can call. Skills follow the same frontmatter shape as personas; calling one returns its body as a tool result the model reads next turn.

TTS and Live2D (optional)

TTS (voice-clone). Set L2DLLM_TTS_HOST / L2DLLM_TTS_PORT in .env to a Qwen3 voice-clone service that accepts Chinese and returns Japanese speech. Streaming chat completions with tts=true then interleave delta.audio chunks (one base64 WAV per segment) with the text. Leave the host empty to disable. See docs/design-docs/tts-design.md.

Live2D (avatar). Rendering is frontend-only (pixi-live2d-display + pixi.js@6). The model assets and live2dcubismcore.min.js are gitignored - place them under web/public/ locally to render avatars. Expression control runs server-side at POST /v1/live2d/expressions; set L2DLLM_EXPRESSION_MODEL to a small/cheap model (empty -> reuses L2DLLM_MODEL) for fast per-turn expression picks. See docs/design-docs/live2d-design.md.

Connecting Open WebUI

In Open WebUI: Admin → Settings → Connections → Add Connection (OpenAI type), set:

• Base URL: http://localhost:8000/v1 (use http://host.docker.internal:8000/v1 if Open WebUI runs in Docker).
• API Key: any string — the server doesn't validate it.

Hit the refresh icon and default, cat, assistant, … should appear in the model dropdown.

Quick smoke test

# list models
curl -s http://localhost:8000/v1/models | python -m json.tool

# non-streaming
curl -s http://localhost:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"cat","messages":[{"role":"user","content":"你好"}]}' \
  | python -m json.tool

# streaming
curl --no-buffer -N http://localhost:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"cat","messages":[{"role":"user","content":"你好"}],"stream":true}'

Security note

The server has no built-in auth and defaults to binding on 127.0.0.1. For remote exposure, put it behind a reverse proxy (nginx, Caddy) with a token check.

Web frontend (dev)

A minimal Vite + Vue 3 chat UI lives in web/. It talks to this backend via the OpenAI-compatible API — same protocol used for any other client.

# In a separate terminal, keep the backend running (uv run l2dllm).
cd web
npm install
npm run dev          # http://localhost:5173

Vite proxies /v1/* to http://localhost:8000, so the page picks up models and streams responses without any CORS dance. The Live2D model assets and live2dcubismcore.min.js are gitignored (not in VCS) - place them under web/public/ locally (model folders in web/public/live2D/, SDK at web/public/live2dcubismcore.min.js) to render avatars.

Development

uv sync --all-extras
uv run ruff check src/l2dllm/
uv run pytest