Skip to content

Strata v0.1.35

Choose a tag to compare

View all tags
@Niko1221 Niko1221 released this 02 Oct 09:05
· 300 commits to main since this release
v0.1.35
d9ab843

Windows AMD no longer crashes on the first prompt, the low-RAM mode works on 32 GB Windows PCs, and a batch of fixes from your reports.

AMD on Windows (#468, #461): the engine now uses the HIP runtime it ships with. Windows looks for a program's DLLs
in its own folder, then in System32, and only then on PATH, where the engine's rocm\bin is. AMD drivers that
install their own amdhip64_7.dll in System32 won, and the bundled libraries crashed on the first prompt (an access
violation on a W7900, hipErrorInvalidDeviceFunction on a 7900 XTX). Setup now puts the bundled runtime next to
strata.exe (every start, so a 0.1.34 install is fixed by running START-HERE.bat), the zip ships it there, and the
engine log names the runtime it loaded.

Low-RAM mode on 32 GB Windows PCs (#467): the resident mode (experts the GPU doesn't hold kept in RAM) never
engaged. Windows counted the files the GPU cache had just read as used memory. The engine now releases them before
it checks, and when the RAM still can't hold every expert, it keeps the hottest ones that fit, instead of falling
back to reading all of them from the SSD.

Fixes:

  • Multi-GPU (#448): a small second card in a layer split capped every prompt chunk at 512 tokens (6x slower
    prompts). The log now names that card, and setup recommends the big card alone (the small one is better as an
    extra expert cache, docs/SECOND_GPU.md).
  • Server:
    • messages sent as a JSON string works, and a malformed list is a 400 (#460);
    • /metrics shows the speculative draft counts per request and in total (#457).
  • Setup:
    • an empty or corrupt config from an earlier install no longer stops a new copy, and configs are written safely
      (#459);
    • a system ROCm without HIP's development files is skipped for the AMD build, and setup uses AMD's wheels instead
      (#446);
    • --gguf-dir says which files it can't use and which it can (#444);
    • --calibrate works with a helper card (--expert-cache-device1), and says so when tuning fails (#447).
  • Docs: the Coder is weaker outside code, including Chinese and other non-English text; use the unpruned sizes for
    those (#438).

Checked before the release:

  • Same answers as 0.1.34 on all four quants (Q2_0, IQ3_XXS, IQ3_S, the Coder), 10/10 each with a fixed cache,
    including the prompt path's internal state at 4K and 20K tokens.
  • The same speed: 5-pair A/B on Q2_0 and IQ3_S, -0.2 / -0.7 / +0.1 / -0.3%, with the same expert slots.
  • #467 here:
    • IQ3_XXS in the resident mode: the trim freed 3.5 GiB (51.1 -> 54.5 GiB available), and 38.7 GiB of experts were
      kept in RAM;
    • the same tokens as the mapped mode.
  • Real use at a 57K-token prompt through the server on Q2_0 and the Coder (incl. a cancelled long prompt and the
    next request).
  • Linux (WSL, RTX 5070): Q2_0 byte-identical to 0.1.34 (10/10), and the default settings pass.
  • AMD on Linux (R9700): the HIP build of the release and its tests, all passing except the two that need a model
    fixture or an AVX-512 CPU, as before.
  • AMD on Windows: the zip carries amdhip64_7.dll and amd_comgr.dll next to the exes; setup's copy step is
    tested. The fix itself awaits the reporters' confirmation (no System32 HIP runtime on our test PC).
  • Tests: the tools' and setup's (255, including the golden check that --yes configs are unchanged), the server's
    (149).

Updating: get the latest files (git pull, or download and unzip anywhere), then run START-HERE.bat (Linux:
./setup.sh). Setup installs engine 0.1.35.

The ready-made Strata engines for Windows, which START-HERE.bat fetches by itself:

  • strata-windows-x64.zip: NVIDIA (RTX 20 / 30 / 40 / 50: sm_75, sm_86, sm_89, sm_120 + PTX), CUDA 13.0, needs an
    NVIDIA driver 580 or newer. Contents: strata.exe, strata-vision.exe (the optional image encoder), BUILD.json.
  • strata-windows-x64-hip.zip: AMD (gfx1100, gfx1101, gfx1102, gfx1200, gfx1201, gfx1030), ROCm 10.2.0a20260930
    from AMD's TheRock builds, needs a current AMD driver. Contents: strata.exe, strata-device.exe, the HIP runtime
    next to them, BUILD.json, rocm\ (the ROCm libraries and their licenses).
Assets 4
Loading