See what your AI is actually doing.
Real-time token streaming, quality analysis, performance profiling, and cost tracking
for local LLMs. No cloud. No API keys. No cost.
Quick Start • Features • Screenshots • Who Is This For • Changelog
One command. 30 seconds.
npx llmxrayOr with Docker:
docker run -p 5174:5174 djovaneli/llmxrayOpen http://localhost:5174 and start chatting. That's it.
Prerequisite: Ollama running locally with at least one model pulled (
ollama pull llama3.2).
You run a local LLM. You chat with it. But what actually happened?
- How fast was each token? Which ones was the model confident about?
- Is the response quality degrading over long conversations?
- What would this have cost if you ran it in the cloud?
- Is the model repeating itself? Refusing? Generating gibberish?
- How does temperature 0.3 compare to 0.9 on the same prompt?
LLMxRay answers all of these, visually, in real time, for free.
Chat with any Ollama model and watch tokens arrive with confidence coloring — each token is tinted based on generation speed. Supports markdown, multi-turn conversations, file attachments, vision models, and slash commands.
Every response is automatically analyzed. Colored badges appear only when something is wrong:
- Repetition — excessive repeated phrases (4-gram analysis)
- Refusal — "as an AI language model" and 7 other patterns
- Gibberish — high non-ASCII ratio
- Empty — fewer than 10 words
- Truncation — hit the token limit without finishing
Up to 4 slots with independent model, temperature, and system prompt. Features include side-by-side streaming, word-level diff highlighting, metrics comparison, and one-click presets (Temperature Sweep, Deterministic Pair, Language Compare with Token Tax visualization).
- Latency percentiles (P50/P95/P99) for duration and TTFT
- Error intelligence — 7-category classifier with timeline
- Usage heatmap — 7x24 grid of your active hours
- Settings impact — temperature vs tokens/sec scatter plots
- Cold vs warm start tracking with model load history
Token usage per model/day with estimated cloud-equivalent pricing. See what you're saving by running locally.
Test model knowledge with multi-choice question suites. Uses real logprobs via OpenAI-compatible endpoint for accurate confidence measurement. Build custom suites visually or let AI generate them from a topic.
Embed text, visualize vectors, measure cosine similarity. Build a local knowledge base from PDFs, DOCX, and CSV — chunked, embedded, and searchable. All stored in IndexedDB. Zero cost.
Drag-and-drop node canvas for building tool definitions. Bidirectional code sync (edit nodes or TypeScript — both update). Probe APIs, auto-generate schemas, test with live execution.
Curate training data from your conversations. Tag, review, and export as JSONL for fine-tuning.
Every experiment (benchmarks, comparisons, chats, training pairs) is automatically archived in a queryable IndexedDB database with filters, trends, exports, and retention policies.
Full translations in English, French, Chinese, and Arabic. RTL layout support. Community scaffolds for Hebrew and Japanese.
|
Chat with token streaming and confidence
|
Model comparison — side by side
|
|
Session deep dive — metrics and timing
|
Benchmark with confidence radar
|
|
Embeddings — cosine similarity
|
System monitor — hardware and Ollama status
|
| You are... | LLMxRay helps you... |
|---|---|
| Developer | Debug prompts, profile latency, compare models, inspect tool calls, track costs |
| Researcher | Run controlled experiments with consistent settings across models and temperatures |
| Student / Educator | Explore model behavior visually — built-in Educators Kit with 9 interactive modules |
| AI team lead | Understand quality trends, error patterns, and resource usage across your local fleet |
npx llmxray
npx llmxray --port 3000
npx llmxray --ollama-url http://192.168.1.50:11434docker run -p 5174:5174 djovaneli/llmxray
docker run -p 5174:5174 -e OLLAMA_URL=http://host.docker.internal:11434 djovaneli/llmxraygit clone https://github.com/LogneBudo/llmxray.git
cd llmxray
npm install
npm run dev # http://localhost:5173| Layer | Technology |
|---|---|
| Framework | Vue 3.5 + Composition API |
| Language | TypeScript 5.9 (strict) |
| Build | Vite 7.3 |
| Styling | Tailwind CSS 4.2 |
| State | Pinia 3 (store-per-concern) |
| Charts | Chart.js 4, D3.js 7 |
| Canvas | Vue Flow (visual node editor) |
| Code Editor | CodeMirror 6 |
| Storage | IndexedDB (browser-native) |
| LLM Backend | Ollama (local) |
Streaming — Reads Ollama NDJSON via fetch() + ReadableStream. Tokens update the UI reactively through Pinia stores.
Token confidence — Approximated from inter-token latency (faster = more confident). Clearly labeled as approximation. Benchmarks use real logprobs via OpenAI-compatible endpoint.
Store-per-concern — Each domain has its own Pinia store: tokens, sessions, metrics, reasoning, comparison, embeddings, quality, cost, and more.
Hardware detection — Custom Vite plugin queries the OS directly (PowerShell/proc/sysctl) for accurate hardware specs.
| Command | What it does |
|---|---|
npm run dev |
Dev server (port 5173) |
npm run build |
Type-check + production build |
npm run test |
Unit tests (Vitest) |
npm run test:e2e |
End-to-end (Playwright) |
Contributions welcome! See CONTRIBUTING.md for setup and guidelines.
Community translations especially welcome — scaffold files ready for Hebrew and Japanese.
LLMxRay is the observation layer. Sentinel is the security and compliance layer — a transparent proxy that captures all LLM traffic with zero SDK integration. Prompt injection detection, PII scanning, agent trace reconstruction, and more. Currently in private beta.
LLMxRay is a trademark of Ivan Stankovic (LogneBudo). See TRADEMARK.md.
If LLMxRay helps you understand your AI better, consider giving it a star.
It helps others discover the project.
