Skip to content

v0.81.29

Choose a tag to compare

View all tags
@github-actions github-actions released this 19 Jun 21:59
v0.81.29
f77dbcb

Fixed

  • External GPUs (eGPU) run at full speed. The Metal backend forces shared
    (system-memory) buffers for external GPUs, so weights stream over Thunderbolt every
    op (~0.8 t/s). ToshLLM now forces private, VRAM-resident buffers
    (GGML_METAL_SHARED_BUFFERS_DISABLE) automatically when an external GPU is selected,
    with a manual "VRAM-resident weights" toggle (shown only when an eGPU is present) for
    the default case where macOS picks the eGPU. Likely also clears the
    failed to decode prompt batch (res = -3) benchmark error on eGPUs.

Added

  • Self-contained server log header. Each server start now logs the app version,
    engine, model, detected GPUs (flagging external/eGPU), the GPU selection, the resolved
    settings and the exact args/env (API key redacted) — so a single pasted log is enough
    to debug a remote setup.

Full Changelog: v0.81.28...v0.81.29

Assets 4
Loading