1.8.5

[1.8.5] - 2026-08-28

Added

• Anti-Loop: two-tier defense against stuck model output. Real models still get stuck in stable output attractors — repeating a sentence, a paragraph, or a <think> fragment for hundreds of tokens — even with --repeat-penalty and DRY tuned. This release ships a two-tier defense that operators can turn on per model preset from a new Anti-Loop section in the launch form. Off by default because both tiers change generation behavior and detection thresholds have to be tuned per model class; existing launches produce byte-identical CLI to before. Every hook in the streaming path is wrapped in try/except Exception that logs and falls through, so a detector bug can never break a real user's stream.

  • DRY sampler (soft, sampling-time). Exposes llama.cpp's --dry-multiplier / --dry-base / --dry-allowed-length / --dry-penalty-last-n as UI fields. dry_enabled: true AND dry_multiplier > 0 are BOTH required to emit anything — llama.cpp treats multiplier=0 as disabled, so emitting the four flags with a zero multiplier would be noisy no-ops. dry_base < 1.0 is rejected at the preset boundary (llama.cpp silently ignores those, which would surface as a mystifying no-op at runtime instead of a clear error). dry_penalty_last_n empty means "flag omitted, llama.cpp default (ctx-size) applies"; 0 explicitly disables the penalty window per llama.cpp's docs; negative values are rejected because llama.cpp's arg parser throws on them and the container would die at startup with an opaque error. All four are baked into the container at launch, so preset edits take effect on relaunch.
  • Output Loop Detection (hard, post-hoc). llamaman's proxy watches the model's streamed assistant-visible output (both content AND reasoning_content — thinking-phase loops matter as much as content-phase) and, if a large chunk of text repeats often enough, closes the upstream connection (which stops llama-server generating) and injects a synthetic terminator into the client stream — OpenAI clients see a finish_reason=stop chunk with a [llamaman: output loop detected, ending turn] marker; Ollama clients see done_reason: "loop_detected" on a done: true line. Detection algorithm is deliberately simple: take the last min_chunk_chars of the buffer, count exact occurrences, flag when ≥ min_repetitions. Catches every runaway loop I've seen in the wild without false-positiving on code with for-loops / poetry choruses / markdown tables at the default 200-char × 3-rep thresholds. Doesn't catch drifting near-repeats ("The answer is: The answer is." with slight punctuation drift) — those get caught by DRY at sampling time. The five thresholds (min_chunk_chars, min_repetitions, max_buffer_chars, scan_every_n_tokens, scan_interval_s) are all preset-live: the next request picks up new thresholds without a relaunch (in-flight streams keep the thresholds captured at attach time).
  • Two-trigger scan cadence. Inline scans run every N chunks (default 64, amortized to hundreds-of-ms detection latency at typical generation speeds); a background worker on the existing 5s poller catches slow streams that emit tokens too slowly to hit the inline threshold (default scan_interval_s = 10). The worker is a cheap no-op when the registry is empty — no dedicated thread, and the check is a dict-emptiness test.
  • Where it watches. Compat routes on :42069 (Ollama /api/chat, /api/generate; OpenAI /v1/chat/completions, /v1/completions) — always, whenever the toggle is on for the target instance's preset, because llamaman is doing the streaming in-process either way. Per-instance ports (8000-8020) — only when a sidecar proxy is already present for the instance (Idle Timeout, Max Concurrent, or Proxy Sampling Overrides also on). Loop detection does not, by itself, spawn a sidecar proxy; the UI hint text points this out so operators know to combine with one of the other three if they want direct-port coverage. This is a deliberate design choice — the entire purpose of llamaman is that you route through its proxy; a direct-hits-only path isn't a supported topology to bolt features onto.
  • Per-request buffering. With max_concurrent > 1 an instance has N turns in flight, so the registry is dict[stream_id → TurnBuffer], not dict[inst_id → ...]. Each TurnBuffer holds a rolling list[str] capped at max_buffer_chars (default 8192, hard cap 64 KB), plus its captured thresholds and a threading.Event for the kill signal. Buffer growth is O(1) on feed (append, cap-check via truncation to the last N chars); scan runs outside the lock so a slow scan can't stall the feed side.
  • Boundary validation. parse_dry_config and parse_loop_detect_config in core/dry_sampling.py / core/loop_detect.py are the single source of truth — every launch route, preset save, and merge whitelist routes through them. Every threshold has floor/ceiling bounds and a "max_buffer_chars ≥ min_chunk_chars × min_repetitions" cross-check that rejects configs where detection would be silently impossible.
  • Coverage. 143 new tests across tests/test_dry_sampler.py, tests/test_loop_detect.py, tests/test_loop_detect_proxy.py, tests/test_loop_detect_compat.py: parser bounds, algorithm correctness on realistic content (code with for-loops, poetry choruses, markdown tables, numbered lists, drifting near-repeats), rolling-cap memory safety, thread-safety under 8-worker concurrent feed, SSE extractor edge cases ([DONE], mid-chunk boundaries, reasoning_content, llama.cpp native format), defensive "broken scanner must not break the stream" invariant, sidecar proxy fork end-to-end, and both Ollama-native + OpenAI compat generators.

• Reasoning Format dropdown. New Model Settings → Reasoning Format select maps to llama-server's --reasoning-format (values: auto / none / deepseek / deepseek-legacy, default auto). Controls how model thought tags are parsed out of streamed output — routed to reasoning_content vs left inline in content — which matters for DeepSeek-R1, Qwen-thinking, and any model that emits reasoning tokens. auto omits the flag entirely so llama.cpp's own default applies; the three explicit values are emitted verbatim, and unknown / hand-crafted values fold to auto at the boundary in normalize_reasoning_format() so a corrupt preset can never make llama-server refuse to start. Shared preset field (not per-node hardware), same tier as flash_attn and cache types. 17 tests in tests/test_reasoning_format.py.

• CPU Threads Batch field. New Container & Proxy Settings → CPU Threads Batch number field maps to llama-server's --threads-batch. Separate thread pool for batch / prompt processing (the prefill / evaluation phase), distinct from the generation threads set by CPU Threads. Useful on CPUs with a big/little split (e.g. Intel P+E cores) where prefill benefits from more threads than steady-state generation, or when you want prefill to use every core but keep generation on the performance cores only. Blank = flag omitted, llama-server falls back to the --threads value on its own — we never re-emit that value server-side. Per-node hardware key (CPU core counts vary per node), so it participates in the cluster preset's node_overrides block same as threads. 14 tests in tests/test_threads_batch.py.

• Auto-clipping for info-tip tooltips extended to respect the model library sidebar. The existing updateInfoTipClipping() fallback in static/js/boot.js only measured tooltip overflow against the viewport edges (0 … window.innerWidth), so a tooltip on a leftmost field of the launch panel — inside the viewport but partially overlapping the 260px sidebar's visual territory — stayed centered and rendered on top of the model rows. The updated function measures .sidebar via getBoundingClientRect() and, when the info-tip icon is to the right of the sidebar, uses the sidebar's right edge as the effective left-overflow threshold. When that check overflows, the existing clip-left class kicks in and the tooltip anchors its left edge at the icon, extending only rightward into the launch panel where there's room. Called on every open (not cached) so a collapsed / expanded sidebar Just Works, guarded defensively for the case the sidebar element is missing (future layouts), and only treats the sidebar as an obstacle for tips outside it — a tip inside the sidebar itself wouldn't get force-anchored based on its own container.

Changed

• GPU Layers label + tooltip corrected to match current llama.cpp value semantics. llama.cpp changed what --n-gpu-layers values mean without changing the flag name: -1 used to mean "all layers" and is now "auto" (a smart VRAM-aware picker that can, and does, leave layers on CPU), -2 is now "all" (force offload every layer), any positive integer is exact. The old llamaman UI still said -1 = all and defaulted to -1, so every launch was silently sending "auto" while the operator thought they were sending "all" — cases like "I set GPU Layers to -1 but not everything is on the GPU" were the observable symptom. The label now reads GPU Layers (-1 = auto, -2 = all, 0 = CPU only) and the tooltip spells out that -1, -2, and positive integers all mean different things, plus the aliases -ngl / --gpu-layers / --n-gpu-layers route to the same flag internally in llama.cpp's arg parser (verified against common/arg.cpp:2811). No behavior change to the field itself — llamaman still sends whatever the operator typed — this release only aligns the UI with what llama.cpp actually does with those values today.

• README.md and DOCKERHUB.md compressed to something scannable. README.md went from 847 → 596 lines (~30% shorter), docs/DOCKERHUB.md from 398 → 207 (~48% shorter). Consolidated the four near-identical GPU-vendor Quick Start blocks into one templated block plus a lookup table for LLAMA_IMAGE (NVIDIA / AMD / Intel / CPU). Collapsed the launch-settings reference into terse per-section tables that defer flag semantics to the llama.cpp docs, with a top-of-file link to github.com/ggml-org/llama.cpp/blob/master/tools/server/README.md — llama-server flags are their source of truth and drift across versions, so mirroring them here just accumulated stale text. Added the new Reasoning Format, CPU Threads Batch, and Anti-Loop entries to the reference tables. Dropped redundant prose between features/setup/troubleshooting sections. No content removed that operators would need — just the same information densified.

• Version + repo-URL footer in the UI. The app's UI now shows the running version (from the VERSION file, exposed via a Jinja context injector) and a link to the source repo in a small footer. Read-only, cosmetic — makes it easier for operators to answer "which version is this?" without shelling into the container.