Repository navigation
Releases
Releases · chelij/llama-wrap
v0.1.13 - Reliability and Benchmarking Patch
Reliability and Benchmarking Patch
Added
- Diagnostics output now streams into the GUI log line by line while Doctor/Probe/Bench/Stress run, with a new Cancel button; CLI
stressandbenchstream too and stop cleanly on Ctrl+C. - Bench now runs a warmup plus several streamed iterations and reports median TTFT, generation tok/s, and prefill tok/s when server timings are available.
- Stress timeouts scale from observed prefill speed, and stress records include TTFT and prefill speed when available.
- Server output is preserved in rotating per-session log files (
logs/, last 10 sessions kept). - Windows port inspection and full process-tree stop support.
- GUI preset search and shortcuts:
Ctrl+F,Ctrl+Enter,Ctrl+S, andEscapeto stop when focus is outside a text field. - Terminal-aware CLI colors,
--no-color/NO_COLOR,--quiet, partial preset matching, and typo suggestions.
Changed
- Launching no longer silently kills unrelated processes on the configured port. Matching inferers are stopped automatically; other processes require confirmation.
- Stop escalates to a full process-tree kill after a 5-second graceful-shutdown timeout.
- Auto-restart is capped at three attempts with exponential backoff and resets after Ready.
- New installs store
history.json, benchmarks, and logs in the per-user data directory; portable existing files andLLAMA_WRAP_HISTORYstill take precedence. History writes are atomic. --metricsis only appended for llama.cpp/llama-server presets.- Unknown imported flags can safely take negative values such as
--seed -1. - GUI window geometry persists, startup/diagnostic status animates, and interactive CLI menus are less repetitive.
Release artifact
llama-wrap-v0.1.13-linux-x86_64.tar.gz- SHA-256:
cd77daaed0d75a27dcd1efb4c32d0e48697796f974df7a02164a0c0d4b4d4508
v0.1.12
Release Notes
2026-06-24 - Ops Diagnostics Patch
This release expands llama-wrap from a launcher into a lightweight local-AI ops helper while keeping the existing preset format backward compatible.
Added
- Shared core logic for GUI and CLI command building, preset parsing, endpoint checks, benchmark formatting, preset bundle portability, context stress testing, and VRAM calibration helpers.
- CLI diagnostics:
doctor <preset>checks executable/model setup, host/port, port availability,/health,/v1/models, and/v1/chat/completions.probe <preset>sends a small OpenAI-compatible chat completion request.bench <preset>runs a short benchmark prompt and saves JSON results, with optional CSV output.stress <preset>runs the same context stress suite as the GUI.
- GUI Diagnostics row with
Doctor,Probe,Bench, andStressbuttons that stream output into the existing log panel. - Context stress suite with runtime context detection, fill/decode stages, sustained synthetic coding-agent turns, boundary probes, error classification, and practical working-limit summary.
- Runtime VRAM calibration from
llama-serverallocation logs and GPU process/total VRAM readings, stored per matching preset command/model signature. - GUI stop logging so Stop reports both the stop request and confirmed server stop.
- CLI preset bundle portability commands:
export-presets --out <file> [--portable] [preset-name...]import-presets <file> [--force]
- Local ignored output folder
.llama-wrap/benchmarksfor benchmark reports.
Changed
- GUI and CLI now share command construction and command-import parsing to reduce drift.
- The GUI VRAM bar uses calibrated estimates when a matching calibration record exists.
- Generated release/local data folders stay out of source control, including new ignore coverage for
llama-wrap/and.llama-wrap/.
Not Included
- No chat UI.
- No model downloader.
- No RAG or agent framework.
- No GUI export/import buttons in the Diagnostics row.
- No Stress progress popup.
v0.1.11c
Visual
- Refined dark color palette (deeper bg, softer contrasts)
- VRAM bar height 12 → 36px with per-segment GiB labels at 10pt bold
- Soft muted segment colors (blue, gold, rose, violet, green, coral)
- Confidence badge dot (●) next to VRAM estimate — green/yellow/red
Presets
- Added format_version: 1 to presets and history file for forward-compatible migration
Resilience
- Stats parser wrapped in try/except — future llama-server log format changes won't crash the app
UX
- Add Flag dialog now has a combobox with ~80 known llama-server flags as autocomplete suggestions
v0.1.11b
- Parse GGUF arrays (strings, MTP metadata) without corrupting offset
- Gracefully skip unknown GGUF types instead of crashing
- Bump array limit to 1024 for modern model metadata
- Extract scalars from array metadata values (e.g. [8] -> 8)
- Fix AttributeError '_saved_stats_var' -> '_session_stats_var' in preset loading
- Use attention.sliding_window for KV cache estimation (fixes Gemma 4 overestimate)
- Remove incorrect 'Draft model ignored' logic under --spec-type draft-mtp
- Always count -md draft model weights in VRAM estimate when present
v0.1.11
Add session stats (TTFT, tok/s), --metrics auto-enable, CLI tab compl…
v0.1.9b
fix: GGUF parser robustness, VRAM SWA estimation, draft model MTP han…
v0.1.10
Refine launcher defaults and controls
v0.1.9
v0.1.8
v0.1.7
Show server VRAM usage, total GPU usage, and total available - Query nvidia-smi per-process for llama-server VRAM (not total GPU) - Display: Server usage | GPU total_used / total_available - VRAM bar shows both server marker (white) and GPU marker (gray) - Fall back to total GPU usage during model loading when PID not yet visible - MTP speculative KV cache included in estimates - Warn when draft model set but MTP is active