Local-first, visual LLM prompt evaluation studio.
What it does
- Define prompts with full version history
- Build test-case datasets
- Run evaluations against local models (Ollama) or any API provider (OpenAI, Anthropic, Groq)
- Score outputs with exact match, fuzzy, embedding similarity, and LLM-as-judge metrics
- Side-by-side run comparison with improved/regressed/same classification
- Regression dashboard tracking metrics over time
- Live run progress via SSE
- Everything stored locally — prompts, data, and keys never leave your machine
Quickstart
See the README for the 5-minute setup. Works with zero paid keys via Ollama.
Verified on: Linux. macOS should work; Windows via WSL.