Skip to content

Strata 0.2.4: DeepSeek V4.1-Flash in the Terminal (Terminal 0.2.4.2)

Latest
Latest

Choose a tag to compare

View all tags
@Niko1221 Niko1221 released this 24 Oct 02:18
· 27 commits to main since this release
v0.2.4
96ca438

Strata 0.2.4.2: DeepSeek launch and install fixes

Fixes for strata launch with DeepSeek and for the one-line installer. The Terminal's Control (0.1.49.2), install.sh and the three DeepSeek engine packages (0.1.49.2) changed. The app stays 0.2.3 and every Qwen engine package stays as it was, so there is nothing new to download for them.

What is fixed

  • Faster DeepSeek on AMD Ryzen 3000 / Threadripper 3000 / EPYC 7002 CPUs. The 2.00 and IQ1_M sets answered up to 3x slower there; the answers are unchanged. The DeepSeek engine is now 0.1.49.2 (strata-dsv41-0.1.49.2-windows-x86_64-cuda-cuda13, -linux-x86_64-cuda-cuda13 and -linux-x86_64-hip-rx9070).
  • DeepSeek setup no longer stops on a Qwen picture file in your models folder (#3927, #3917). If the folder you share between models held a Qwen vision file, DeepSeek setup picked it up by mistake and failed. It now ignores any picture file that is not DeepSeek's.
  • Model files you already have are recognised for DeepSeek in the guided screen too, including Unsloth's UD-Q8_K_XL set (#3914). The "use model files I already have" screen called a downloaded UD-Q8_K_XL file unreadable. It now recognises DeepSeek sets by their file names, as strata launch --model-path already did.
  • The right DeepSeek engine runs when you have more than one Strata install. Launch now picks the newest DeepSeek engine package of the install that is running, and of the install your model's settings came from, instead of an old path saved in the model's settings.
  • A plain message when GitHub cannot be reached. DeepSeek setup checks for 10 seconds before it downloads anything and says in one line that GitHub is not reachable, instead of retrying for minutes.
  • Faster returns to a conversation. DeepSeek keeps recent conversations (and a shared system prompt) on disk, so a returning agent or a sub-agent with the same system prompt answers sooner; up to 2 GB, cleaned after a week.
  • The DeepSeek engine updates itself. A newer DeepSeek engine package in the signed feed is installed on the next strata launch (see Updating).
  • DeepSeek no longer downloads the Qwen engine. A DeepSeek launch does not fetch or test-start the Strata (Qwen) engine, the llama.cpp source or CUDA libraries it never uses. strata launch --plan no longer promises that download.
  • The disk check counts only what is still to download. If most of the model files are already in your models folder (also under the first release's file names), the model list no longer says "tight: needs more disk space". The room for the prepared model is still counted.
  • Smaller wording fixes. A setup message that ended with a full stop no longer gets a second one ("engine.."), and the second sentence of "The model could not be installed. The model does not answer sensibly after the preparation: ..." starts with a capital.
  • The install one-liners keep your terminal open on an error. install.sh is now a function that returns an error instead of closing the shell it runs in, and the commands in the README and docs download it first: curl -fsSL https://github.com/Niko1221/Strata/releases/latest/download/install.sh -o install.sh && sh install.sh.

Updating

  • DeepSeek engine: it updates itself on the next launch. When the signed feed has a newer DeepSeek engine package for your PC (0.1.49.2 over 0.1.49), strata launch downloads it, checks it, starts it once, switches to it and removes the old one. If that fails you see one line and the engine you have keeps running; with no internet the launch goes on without waiting. An engine you gave with --dsv41-package or STRATA_DSV41_EXE is never replaced.
  • From 0.2.4 or 0.2.4.1: run strata update in the Terminal. It fetches Control 0.1.49.2 and the next strata command runs it (strata --version says 0.2.4.2).
  • New installs: use the command in the README (download install.sh, then run it).
  • Mac: the Mac Terminal gets Control 0.1.49.2 too. DeepSeek is not on the Mac yet.

Downloads

Everything from 0.2.4.1 is attached again, so the release page still holds every file. New in this release: the three Control packages (strata-control-0.1.49.2-windows-x86_64.zip, -linux-x86_64.tar.gz, -macos-arm64.tar.gz), the three DeepSeek engine packages (strata-dsv41-0.1.49.2-windows-x86_64-cuda-cuda13.tar.gz, -linux-x86_64-cuda-cuda13.tar.gz, -linux-x86_64-hip-rx9070.tar.gz), install.sh, the feed and SHA256SUMS.txt. The installers, AppImages, dmg and Qwen engine packages are the 0.2.3 files.

Update 0.2.4.1: DeepSeek setup now works from the installed Terminal. The 0.2.4 Terminal package was missing the DeepSeek preparation scripts, so setup stopped at "Preparing your model". If you installed 0.2.4, run strata update, then run your DeepSeek setup again. New installs get the fix automatically. The RTX 3060 figure below is corrected to the release defaults (8 to 10 tokens/s; DSpark stays off by default on 12 GB cards).

Strata 0.2.4: DeepSeek V4.1-Flash in the Terminal

Strata now runs DeepSeek V4.1-Flash on a PC with one 12 GB graphics card and 46 to 64 GB of RAM. The engine keeps the experts it uses most in RAM and streams the rest from your SSD.

DeepSeek is in the Terminal (strata launch, strata setup) in this release. The desktop app is unchanged (it stays 0.2.3) and gets DeepSeek in a later update.

What you get

  • DeepSeek V4.1-Flash in the model list of Setup, in three sizes from ISTA-DASLab:

    set download quality against the original model
    IQ1_M (1.75 bits) 239 GB perplexity +21% (KL 0.217); code and agent text +8.9% and +10.2%; weaker on plain English prose. ISTA-DASLab's model card: keeps 98.7% of the base model's task average (AIME25, GPQA-Diamond, LiveCodeBench v6)
    IQ2_XXS (2.00 bits) 257 GB perplexity +15.5% (KL 0.170); code and agent text +8.6% and +5.3%
    IQ2_XS (2.25 bits) 274 GB perplexity +7.3% (KL 0.107); code and agent text +3.2% and +2.2%
    IQ4_NL (Engram table, not a model to pick) 111 GB, already in all three downloads above The 4-bit Engram lookup table the model reads from RAM or SSD; all three sets use this one file

    "Perplexity" is how surprised the model is by a fixed set of 4,995 test texts; lower is closer to the original. Pick IQ2_XS for coding and agent work when you have the disk. Setup shows all three sizes and warns if your PC is short of disk or RAM. It does not stop you.

  • Setup downloads the files, builds the model folder and starts the engine. You set no flags.

Speed on real PCs

Output is the speed of the answer. Prompt is how fast Strata reads what you send it.

PC output prompt
RTX 5070 12 GB, 64 GB RAM about 5 tokens/s (code) 231 tokens/s at 8K
RTX 3060 12 GB, 46 GB RAM about 8 to 10 tokens/s (code 9.9, agent work 7.8) 149 tokens/s at 8K
1x Radeon AI PRO R9700 (AMD), IQ2_XS about 11 tokens/s not measured
4x Radeon AI PRO R9700 (AMD) about 31 tokens/s (55 to 69 with DSpark) about 505 tokens/s at 8K, about 818 at 32K

These are the 2.00-bit set. The 2.25-bit set reads 13% more data per token, so expect it to be a little slower. Your drive matters: the 12 GB cards spend most of each token waiting on the SSD and on the CPU, and a slow NVMe drive lowers every number above.

How it works

  • A RAM expert store that sizes itself. The model is a mixture of experts: each token uses a few of thousands of small blocks. Strata keeps the busiest ones in RAM, takes whatever RAM is free, and reads the rest from the SSD with direct reads. On the RTX 5070 PC with 64 GB this gave 4.3 to 4.9 tokens/s where memory-mapping the files gave 1.8 to 2.2.
  • Hot-expert profiles ship with the engine, so the first prompts do not start cold.
  • Faster prompts. The engine now reads long prompts layer by layer in big chunks instead of 16 tokens at a time. A 200-token prompt went from 3.6 to 6.4 tokens/s on the RTX 5070 PC, and 8K prompts reach 231 tokens/s.
  • Faster prompts on Radeon AI PRO R9700. New matrix-core kernels (RDNA4) read prompts 36% faster at 8K and 49% faster at 32K on 4 cards (370 to 505 and 550 to 818 tokens/s). Answer speed does not change. They are on by default on these cards.
  • Faster long prompts. A prompt over about 3,700 tokens skips work that the last 20 layers cannot use (a 16K prompt is about 34% faster). In 99.5% of the 799 positions we compared, the model's top choice for the next word was the same with and without the skip.
  • Thinking budget. When neither your request nor your config sets one, DeepSeek thinks for at most 32,768 tokens. A request can set its own budget; 0 turns the limit off.
  • DSpark (a small draft model from Unsloth) is on where it measured faster and off on CPU-bound PCs. On 12 GB cards DSpark stays off by default; turning it on (--dspark on) adds 2-13% on the RTX 3060 PC. On the 4x R9700 box it raised 31 tokens/s to 55 to 69. On the RTX 5070 PC with a 6-core Ryzen 5 7600 it gave no gain at any setting (code 4.64 tokens/s with it off against 4.33 at best with it on; agent 3.41 against 2.86 to 3.21), because checking the drafts costs the CPU about what the saved tokens are worth. There --dspark auto does not load the drafter.
  • Faster CPU rows. The engine has new exact rows for the experts the CPU computes: AVX-512 on Zen 4 and Zen 5, AVX2 on other CPUs, chosen automatically. On the Ryzen 5 7600 PC with an RTX 5070 (2.00 set, DSpark on, three alternating pairs, identical text in all six runs), decode went from 4.08 to 5.11 tokens/s (+25%) and prompt reading rose 26%. This was measured on one CPU, a Zen 4. Zen 5 gained more in the kernel tests.
  • Cards and engines: tested on an RTX 3060 (Linux, full model), an RTX 5070 (Windows) and the Radeon AI PRO R9700 (gfx1201, Linux). The RTX 30 kernels (sm_86) were also checked on an RTX A4000. Windows on RTX 30 is built but was not run here. RX 9070 (gfx1200) is built but untested. DeepSeek runs on its own engine package (see below).

Needs

  • Disk: 239 GB (IQ1_M), 257 GB (IQ2_XXS) or 274 GB (IQ2_XS). The download includes the Engram table and the DSpark drafter.
  • RAM: 46 GB worked on the RTX 3060; 64 GB is the tested comfortable size. More RAM means a bigger store and faster answers.
  • Graphics card: 12 GB or more.

Known limits

  • Prompt speed on 12 GB cards depends on the drive. A slow NVMe drive holds the prompt below the numbers above.
  • DSpark is off on PCs where the CPU is the limit.
  • Not in this release: Mac and Strix Halo. DeepSeek V4.1-Flash does not run there yet.
  • Output on 12 GB cards is 4 to 8 tokens/s. That is usable for chat and slow for long agent runs.
  • The same prompt can give a slightly different answer from one run to the next. At some words the model's top choices are nearly tied, and small differences in how the numbers are added up decide between them.
  • If you set up DeepSeek on 4 or more AMD cards before 0.2.4, run setup again to get the faster prompt settings.

Updating

  • From 0.2.3 or 0.2.3.1: run strata update in the Terminal. It fetches Control 0.1.49. Your Qwen engine does not change: every engine package is still the 0.1.48 file, byte for byte.
  • Mac: the Mac Terminal gets Control 0.1.49 too. The Mac engine stays 0.1.48, and DeepSeek is not on the Mac yet.
  • New installs: the install command (install.ps1 on Windows, install.sh on Linux) installs this version.

The DeepSeek engine is a separate download

DeepSeek runs on its own engine package, strata-dsv41, which sits beside your Qwen engine and never replaces it. There are two ways to get it.

  • Automatic: choose DeepSeek in strata launch or strata setup. Strata downloads the package that fits your PC, checks its signature and installs it.
  • By hand: download the package for your system from this page, check it against SHA256SUMS.txt, and run strata launch --model ist-das-2.00bpw --dsv41-package PATH-TO-FILE (or python setup.py --family deepseek41 --dsv41-package PATH-TO-FILE). Strata checks that it is the DeepSeek engine, that it fits your system and that its signature and file hashes are good, and refuses anything else. You can also put the file in the engine-dsv41 folder next to engine.
System File
Windows, NVIDIA RTX 30 / 40 / 50 strata-dsv41-0.1.49-windows-x86_64-cuda-cuda13.tar.gz
Linux, NVIDIA RTX 30 / 40 / 50 strata-dsv41-0.1.49-linux-x86_64-cuda-cuda13.tar.gz
Linux, Radeon AI PRO R9700 / RX 9070 strata-dsv41-0.1.49-linux-x86_64-hip-rx9070.tar.gz

Downloads

Everything from 0.2.3 is attached again, so the release page still holds every file. New in this release: the three Control packages (strata-control-0.1.49-windows-x86_64.zip, -linux-x86_64.tar.gz, -macos-arm64.tar.gz) and the three DeepSeek packages above. The installers, AppImages, dmg and the Qwen engine packages are the 0.2.3 files.

Thanks

Strata is free and open source. If it runs well on your PC, a coffee keeps the work on it going:

Buy Me A Coffee

Assets 41
Loading