giq 0.5.0 — recipes, instances, and the Responses API

giq 0.5.0 gives the domain proper names, adds the OpenAI Responses API and structured output, and fixes how giq serves from more than one card. It is a breaking release for the control API: read Upgrading before you deploy.

One word per thing

giq now names what it serves in six terms, one meaning each (ADR-003):

• Weights are the files: a GGUF, a checkpoint directory, an HF repository.
• An engine runs them: llama.cpp, vllm, sd.cpp, and the Python runtimes.
• A recipe is weights + engine + parameters, with its VRAM figure. Its name is what a client sends as model, and it serves one or more modalities (llm, text2image, ocr …).
• An instance is a recipe running on a card. Residency decides whether it stays: keep warm, on demand, or off.

What that changes in practice:

• Recipes are keyed by name. A name or alias is unique across modalities, so model alone says what to run.
• flux_klein is one recipe for text-to-image and image edit. Alternating renders and edits no longer reloads it: measured on an RTX 5090, a 12 s render followed by a 2.8 s edit on the same process.
• The speech-to-text recipes are named after their weights (faster-whisper-tiny … faster-whisper-large-v3). The bare sizes (tiny, large-v3 …) still work as aliases.
• Weights are an inventory. GET /weights lists every checkpoint once, however many recipes load it, with its format, source, revision, licence and size. DELETE /weights/{id} removes one, and is refused while a recipe using it is loaded. Two recipes that disagree about what one checkpoint is are a load error, not a silent pick.
• Instances are visible. GET /instances lists what runs on which card: residency, state, port, pid, lanes, VRAM.
• /recipes replaces the model control routes. GET /recipes, PUT/DELETE /recipes/{name}/residency and PUT /recipes/{name}/card take a recipe by name or alias, and keep the 409 guards: two LLMs pinned to one card, a pinned set that cannot fit its card, a recipe bound to a card it can never fit.

Dashboard

• Recipes (was Models): one card per recipe with its residency, GPU binding, engine and fit, filterable by modality, engine and residency.
• Inventory (new): the weights table with delete, the engines and their builds, and the disks.
• Overview: the GPU cards and the running-instances table are built from /instances.
• Every string uses the new terms, in English and German.

OpenAI Responses API and structured output

Contributed by @pauldkling in #1.

• POST /v1/responses, streaming included: tool calls, the developer role, thinking controls, tool-output parts. Thinking streams as reasoning_text, one output item at a time.
• Structured output on /v1/chat/completions: response_format (json_object, json_schema; honoured by llama.cpp and vllm) or vllm's native structured_outputs. Sending both is refused with 400. On /v1/responses the same thing is text.format.

Serving fixes

• Two vllm servers on one card no longer kill each other. A second server of one engine on a card takes a free port from that card's block. Before, two vllm recipes on one GPU restarted each other in a loop and answered with the wrong model.
• Start budgets per engine and recipe. vllm gets minutes rather than llama.cpp's seconds, a recipe can set its own ready_timeout, and a start that runs out says so instead of reporting a generic timeout.
• /status reports every card (gpus), and a job waiting for VRAM is judged on the card it is bound to.
• Storage finds every LLM's weights, NVFP4 and other vllm checkpoints included, not just GGUFs.

Upgrading

Back up stats.db first. On first start, giq renames the job-history columns in place (worker → modality, model → recipe) and moves the residency table to recipe names. History is kept.

Removed (each now returns 404):

Before	Now
GET /control/models, GET /stats/models	GET /recipes
POST /control/models/{worker}/{model}	PUT /recipes/{name}/residency
POST /control/models/{worker}/{model}/device	PUT /recipes/{name}/card
DELETE /control/models/{worker}/{model}	DELETE /recipes/{name}/residency
DELETE /storage/models/{worker}/{model}	DELETE /weights/{id}
the models list in GET /storage	GET /weights (/storage keeps the disks, paths and recipe files)

Renamed, with the old form still accepted for one release (a warning names it):

• worker in job bodies → modality. Job and stats JSON now say modality and recipe.
• GIQ_INSTANCES_DIR → GIQ_RECIPES_DIR. giq init moves $GIQ_HOME/instances/ to recipes/; the systemd unit's ReadWritePaths names recipes.
• worker: llm in a recipe file → modalities: [llm].
• llm/gemma-4-12b in residents: and gpu.bind → gemma-4-12b.

Deploy the dashboard with the backend. The 0.5.0 dashboard needs the 0.5.0 routes: install-debian.sh --ref v0.5.0 --ui-release v0.5.0.

For code that imports giq: giq.workers is now giq.adapters (LlamaCppAdapter, VllmAdapter, SdCppAdapter …), and giq.instances is giq.recipes.

Full changelog: v0.4.0...v0.5.0