Repository navigation
✨ Added
- Compare over time — a Trend page charts one model's speed across its history. The verdict is judged against that model's own run-to-run spread, so a 5% dip on a steady model is reported and a 23% dip on a jumpy one is not: it says "regression" only when the change exceeds the noise it already shows, with a 5% floor so three near-identical runs can't make anything look significant
- Inter-token latency — the gap between consecutive tokens is recorded during streaming and reported as a median plus a 95th percentile. TPS averages the whole run, so a model that starts fast and then crawls still looks healthy; the p95 is what exposes it, and a column whose p95 is more than 2× its median now shows a marker. Stored in the history DB (schema v5), so it survives a reload and a backup round-trip
- 128K context sweep — the sweep now offers 128K alongside 2K…64K for long-context models. It stays out of the default set: it is slow, and most local models cannot hold it
- KoboldCpp and Jan detection — both are found by the automatic scan (KoboldCpp on 5001 under its
/api/v1OpenAI layer, Jan on 1337). Probed after Ollama, LM Studio and llama.cpp, so an unreachable extra never delays the servers most people run - Sound effects and desktop notifications — short synthesised cues mark a run starting and finishing (a suite gets its own three-note resolve, an all-error run gets a falling tone), and a desktop notification fires when a long run ends while the window is in the background. Cues are generated with the Web Audio API rather than shipped as audio files, so there are no assets to load and nothing breaks if a webview refuses to start an
AudioContext. New "Sound & alerts" section in Settings, with a volume slider that previews the real cue - Race view — a live timing board above the columns: one lane per model, ranked by tok/s as the run streams, with TTFT split times and the model's own color. Bars encode speed, not characters written, so a model that rambles for 800 tokens doesn't outrank one that answers in 80. Toggled with the Race button in the results toolbar
- Share card (PNG) — saves a run as a fixed-size card (prompt, per-model speed bars, TTFT, token counts, cloud cost) for posting. Renders on a fixed dark ground so the image looks the same regardless of the sender's theme
- One color per model, everywhere — the columns, results chart, Test Suite scoreboard and history chips now derive a model's color from its name instead of its position, so adding a column or shuffling for blind mode no longer repaints the comparison. Colors are assigned collision-free within a run
- Per-model generation parameters — any Bench column can override temperature, top-p and max output tokens from the column header (a dot marks overridden columns; unset fields still inherit the global Advanced settings), so one model can be compared against itself at different temperatures
- Re-run from History — a run can be started again straight from its history card, restoring the prompt, columns, global sampling settings and per-model overrides that produced it
- JSON Schema checks for Test Suite outputs (alongside exact/contains/regex/JSON), with path-level errors and a compact validator covering
type,required,properties,items,enum,const, bounds andallOf/anyOf/oneOf/not - Context sweep — runs one model against the same prompt at 2K…64K context windows (neutral filler, fixed 256-token answers) and charts tok/s and TTFT against window size, with the percentage drop versus the smallest window
- Cloud spend dashboard — new Spend page with total cost, average per run, a 30-day cost trend, and per-provider / per-model breakdowns built from the cost frozen into each run (local runs and runs without exact tokens stay out)
- Bundled fonts — Inter (UI), JetBrains Mono (metrics) and Vazirmatn (Persian) now ship with the app instead of relying on whatever the OS has installed, so typography and Persian rendering are identical on every machine
- Per-connection URL variables — some providers can't be reached from a static base URL. Cloudflare Workers AI needs the account id in the path (
…/accounts/{ACCOUNT_ID}/ai/v1), so the Add/Edit key dialog now renders an Account ID field (with the dash.cloudflare.com hint) under the key, and the value is stored per connection. Every request — test, model refresh, per-model probe, bench and suite runs — interpolates it, and a connection without one is skipped rather than sent to a URL that can only 404 - 18 more cloud providers — Cloudflare Workers AI, Venice AI, SiliconFlow, Hugging Face Inference Providers, Hyperbolic, Featherless, Nous Research, LLM7, Morph, Cerebras, Fireworks AI, Alibaba DashScope, Baidu Qianfan, Tencent Hunyuan, MiniMax, Dahl Inference and Cohere, pushing the catalog from 19 to 36 entries. All ride the existing Bearer-auth streaming client; icons drop into
public/providers/<id>.png - Low-sample badge and Wilson bound on the leaderboard — a model with fewer than 5 successful runs is flagged instead of letting a 1/1 record top the board, and each win rate now shows its Wilson 95% lower bound next to the raw percentage
🐛 Fixed
- URL-variable values could escape their segment: the Cloudflare account-id field is substituted into the base URL, and any value containing
/,?or#would have retargeted the request path or injected a query. Values are now validated against the RFC 3986 unreserved set (-._~A-Za-z0-9) and percent-encoded; anything else leaves the placeholder, blocks Save/Check, and is reported in the field instead of producing a silent 404 - Backup restore was lossy: importing a history backup dropped each result's
reasoningtrail, and would have dropped inter-token latency too. A round-trip now preserves every field the app can store - Phantom servers in the detection scan: a service answering
200with unrelated JSON was reported as a local server with an empty model list. With five ports now probed, that was much likelier to happen; a candidate must return at least one model to count as detected - AMOLED flattened every translucent panel: the theme matched compiled utility classes with
[class*='dark:bg-slate-900'] { … !important }, which also caughtbg-slate-900/60and forced it solid — 38 translucent surfaces (cards, drawers, headers) lost their opacity in AMOLED. The theme now remaps Tailwind's own--color-slate-*variables, socolor-mix()keeps working, no!importantis involved, and new components theme correctly without knowing AMOLED exists. Chrome colour is unchanged: pure black canvas, same surface steps - Overscroll "slack" at the window edges: dragging with a touchpad or touchscreen bounced/slid the whole page and revealed a gap behind the app. WebView2 swipe-navigation and elastic overscroll are now disabled, the document can no longer scroll, and inner panes contain their own overscroll
- Settings panel never animated: it used
animate-in/slide-in-from-right, which come fromtw-animate-css— a package this project doesn't install, so the classes silently did nothing. Replaced with a real keyframe that also mirrors under RTL - Delete button inside a button in Quick Prompts: user presets wrapped a delete button in the preset button, which is invalid HTML and left the delete action unreachable by keyboard. The pill is now a container with two sibling buttons
- Untranslated updater dialog and model picker:
updater.*andbench.selectModelwere referenced in code but existed in no locale, so those strings fell back to English in all six languages
🔧 Changed
-
Prettier is actually enforced now —
.prettierrcexisted since 1.0 but was never applied, soprettier --check .failed on 40 of ~120 files. The whole tree is formatted once,npm run format/format:checkexist, and CI gained a Prettier gate next to ESLint -
1.6.0 has a date and a clean section —
[Unreleased]became[1.6.0] - 2026-10-24, theK127typo became128K, the provider count (19 → 36) and the missing Share card / Race / Trend / ITL / notifications entries were added, and README/README.ru/README.zh/SECURITY.md now agree with the code -
Race view re-renders 40% less while streaming — the board's clock ticked every 120 ms and re-derived every lane on each tick, and a fresh
modelColorHex()call per lane per render. Ticks now land on a coarser clock, rows are memoized (LaneRow) so only lanes whose numbers moved repaint, and colors resolve once per lane identity -
Font licences ship with the build — Inter, JetBrains Mono and Vazirmatn are OFL 1.1, which requires the licence to travel with the font. The fonts were copied into
dist/assets/but their LICENSE files were not, so an installer shipped fonts without the notice.scripts/copy-font-licenses.cjs(part ofnpm run build) now writes them todist/fonts-licenses/, andTHIRD_PARTY_NOTICES.mdindexes them -
Notification permission is requested at opt-in, not mid-run — turning on desktop notifications in Settings now asks the OS for permission right then; a run finishing in the background only posts if it was already granted. Previously the request fired from the run-completion callback, so the one-shot system dialog appeared at the worst possible moment and a denial burned the only chance the OS gives
-
Settings has a manual "Check for updates" — the updater only ran once per launch, so verifying a release meant restarting and hoping. The new button reports all four outcomes distinctly (update available / already latest / check failed / desktop-app only), which is what makes "yes, the endpoint is reachable and signed" a testable result
-
Code highlighting no longer ships 450 grammars —
PrismAsyncLightimportsrefractor/all, so the bundler emitted one chunk per Prism grammar: 1057 JS files, ~3.6 MB of installer assets, for languages no model answer uses. The renderer is nowhighlight()bound torefractor/corewith the 16 grammars insrc/utils/codeLanguages.tsregistered — 76 chunks, 2.09 MB (−43%) — and an unregistered fence label falls back to plain text instead of requesting a chunk that isn't there -
Heavy views are code-split: History, Leaderboard, Settings, Diff, Test Suite, Cloud, Spend, the context sweep and the results chart now load on demand — the startup bundle dropped from ~1.1 MB to ~275 KB, so the Bench paints sooner on launch
-
Accessibility pass: 37 icon-only controls now carry a translated
aria-label(up from 13, several of them hardcoded English), vote and favourites toggles exposearia-pressed, and the toast stack is anaria-liveregion so screen readers announce results instead of missing them -
Contrast fixes measured against WCAG AA:
slate-400on light surfaces was 2.56:1 and the inverted pairtext-slate-400 dark:text-slate-500failed in both themes (2.56 / 3.07) — both now resolve totext-slate-500 dark:text-slate-400. Accent text (emerald, cyan, amber, rose) moved to the-700step on light with an explicit dark twin, taking metric rows from 3.2–3.9:1 to 5:1+ -
Documented micro type scale:
text-micro(11px) andtext-nano(10px) replace 95 arbitrarytext-[11px]/text-[10px]utilities. Nothing renders below 10px any more (8px and 9px outliers removed, 10px prose promoted to 11px), and 10px is reserved for uppercase mono labels and badges -
Tauri version gate moved from CI to the release workflow: Dependabot still opens the npm and Rust halves of a Tauri bump as separate PRs, so gating every PR made both halves permanently red and mutually blocked. A mismatch now only fails at tag time — which is where
tauri buildwould actually break -
Dependabot back to per-ecosystem
groups: the 1.5.0multi-ecosystem-groupssetup did not consolidate the two halves in practice and stopped opening the Rust half at all; per-ecosystem groups keep both ecosystems updating
📦 Dependencies
- Tauri
2.12.3→2.12.4andtauri-build2.7.2→2.7.4(npm@tauri-apps/api2.12.3→2.12.4),@tauri-apps/plugin-updater/tauri-plugin-updater2.13.3→2.14.0,@tauri-apps/plugin-sql/tauri-plugin-sql2.5.0→2.6.0,@tauri-apps/plugin-opener/tauri-plugin-opener2.7.1→2.7.2,@tauri-apps/plugin-process2.4.0