Skip to content

v2.3.2 - Pause Function and adjusted thinking cap

Choose a tag to compare

@Maximo101 Maximo101 released this 12 Nov 06:44
· 27 commits to main since this release
055f347

Pause/resume, thermal guard, auto thinking budget (work on v2.3.2, 2026-11-12)

Pause / resume

  • Same-process pause controlled by run_control.json in OUTPUT_DIR (polled ~1 s). Works on Windows and for a run on another machine sharing OUTPUT_DIR. CLI: --pause, --resume (no arg + live run); settings page: Pause / Resume buttons; /api/pause, /api/resume.
  • Temporary state: run_<stamp>.pause.json + the existing run_<stamp>.rows.json. Both deleted when the run ends or resumes. --resume <run> still recovers a process lost while paused.
  • Manual pause: current model's speed phase finishes; evals stop taking new questions, in-flight ones finish and are graded. The batch probe cannot be paused.
  • Thermal: MAX_GPU_TEMP (0 = off, else 40-110), GPU_TEMP_RESUME_DELTA (default 10), GPU_TEMP_MAX_RESTARTS (default 3). Own GPU monitor thread at 500 ms (never inside token generation); trip is latched; resume after 3 consecutive cool samples. Not enforced, with a loud warning, when the server is remote or no temperature can be read.
  • Fresh restart: PauseRestart(BaseException) raised at checkpoints only inside CONTROL.restartable(). Speed test restarts wholesale; long-context restores its row snapshot; an interrupted eval DatasetRun is discarded and re-run (answered questions come from the result cache). After GPU_TEMP_MAX_RESTARTS restarts of one line, PauseGaveUp ends the run like Ctrl+C (exit 130, resumable).
  • Time paused is recorded in phases.paused_s (it is also inside speed_s / eval_s).

Thinking budget

  • EVAL_THINK_BUDGET=auto is now the default: EVAL_THINK_BUDGET_PCT (25) % of the model's own context, raised to EVAL_THINK_BUDGET_MIN (4096) and held to EVAL_THINK_BUDGET_MAX_PCT (60) % of the context (ceiling wins on a tiny context), rounded down to a multiple of 1024. Unknown context: 16384. A number or 0 still fixes it. Validation: PCT 5-90, MIN >= 256, MAX_PCT 10-90, PCT <= MAX_PCT.
  • Logged per model: thinking budget 32,768 tokens (25% of 131,072 ctx) (or "floor ..." / "ceiling ..."). Exported in results.json / dashboard meta (think_budget, think_budget_pct, _min, _max_pct); --compare refuses runs that differ.
  • Settings page: hover tip on the four keys lists the resulting budget for 4k..1M contexts from the current field values (JS twin of auto_think_budget, tested under Node).
  • Consequences: the budget is in the result-cache key, so models whose budget changed are re-asked (EVAL_THINK_BUDGET=16384 keeps old cache and comparability). A bigger budget fits fewer requests in the shared KV pool: in-flight drops (4 -> 3 at 128k in the mock). Scores of previously capped models may rise; not measured.
  • Evidence (user's 2026-11-12 server log): many models hit the 16,384 cap on ~5-15% of questions; most recovered by the No-Thinking retry (time cost), the LFM2.5 builds show "0 recovered, N counted as wrong".

Rebrand (display strings only)

"Local LLM Benchmark Suite" in the dashboard, settings page, markdown report, argparse description, .env.example header, __init__. Left for the owner to decide: GitHub repo / Project name, REPO_URL and clone URLs, tool.name in the JSON and the notify body, the markdown front-matter tag, historical release notes, regenerated demo dashboards.

Open items

  • Not verified: real GPU / NVML thermal behaviour, a real llama-server, Docker, Hugging Face.
  • Suggested commits: feat: pause/resume and GPU thermal guard, feat: auto thinking budget with floor and ceiling, fix: rebrand display strings, and docs: (README) separately.
  • No new third-party dataset, method or library: no credits/licence change.