v0.1.0

Coloma v0.1.0 first public release

The Continuous Verification and Optimization layer for self-hosted LLMs.
Runs 100% on your own hardware: nothing collected, nothing sent anywhere.

Optimize

• Auto-profile your model on your GPU and tune vLLM for the metrics you actually care about, without runtime OOM crashes
• Job-duration calculator: see how long X concurrent requests of Y prompt tokens take to generate Z tokens, so you can size --max-num-seqs and --max-model-len from data, not guesswork
• Compare tuned configs side by side, then deploy the winner in one click

Verify

• Drop-in OpenAI-compatible proxy sits between your workload and your endpoint, no client changes
• Idle-compute verification: re-runs heavier correctness checks on past predictions while the proxy is idle, zero added latency
• Structured-output schema validation + custom Python validators
• Live and saved traffic inspection

Plus base64 image optimization at the proxy and a built-in chat interface.

Get started

See the Quickstart:

cp backend/.env.template backend/.env   # then edit
uv sync && npm i && npm run build && npm run start

Come hang out on Discord!

Running open-weight inference in prod? I want to hear what you're on, @tschillaciML.

A star helps a lot ⭐