Halowin Bubo v0.1.2

Halowin Bubo v0.1.2 runs the ISTA-DASLab GSQ-RCO files IQ3_S, IQ3_XXS, IQ2_XS and the Flash Next Coder as published, reads prompts of 4,096 or more new tokens faster in an optional fast mode, can store the context cache in half the memory, and adds in-engine abliteration, off by default.
It targets Strix Halo laptops with Ryzen AI Max and Radeon 8060S graphics at approximately 70 W.
The base is gufo v0.26.0.

Output for a fixed seed can differ from v0.1.1.

What is new

• Run it in a lot less memory: ISTA quant support. Bubo reads the published ISTA-DASLab GSQ-RCO GGUF files IQ3_S, IQ3_XXS and IQ2_XS of Qwen3.8 Flash Next without conversion, including the IQ formats of the expert tensors. On ISTA's own benchmarks (mean of AIME25, GPQA-Diamond and LiveCodeBench v6), IQ3_S scores 93.26 against 93.12 for the full-precision BF16 model, and IQ2_XS scores 89.16, 4.0 points (4 %) lower. The main files are 55 GB and 39 GB; with the 28.8 GB embedding file the downloads are about 84 GB and 68 GB. Bubo also runs the Flash Next Coder (GSQ-RCO, 256 experts). The app lists the Coder. With the MTP predictor the Coder writes about 69 tokens/s instead of about 35 without it (about twice as fast), and the text is identical. Its weights take about 29 GB of GPU memory (about 32 GB with the draft file, about 38 GB for the running server at a short context), and it reads a 22K-token prompt at 1,316 tokens/s. The download is about 61 GB (58.4 GB of model files and 2.8 GB for the MTP file), and it does not take image input. On a 22K-token prompt, the published IQ3_S, IQ3_XXS and IQ2_XS files read at 1,253, 1,260 and 1,213 tokens/s.
• Fast prompt reading. bubo serve llm --prefill-mode fast, or Prefill mode on the Server page in the app. About 16 % faster reading of prompts that have 4,096 or more new tokens (1,718 instead of 1,482 tokens/s on a 22K-token prompt) with no noticeable reasoning regression, by using a tuned number of experts while reading. New tokens are the prompt tokens that the cache does not restore. Shorter prompts read as in normal mode. Decode unaffected. The mode works on models with 512 experts. The Coder (256 experts) and the profile exact refuse it. The default is normal.
• Smaller context cache. --kv-cache-type q8_0, or "KV cache" with the choice "Q8 (half the memory)" on the Server page in the app, stores the context cache of Qwen3.8 Flash Next in 8-bit form. The KV cache is 47 % smaller (4.5 GiB to 2.4 GiB at 196,608 tokens), so each extra 64K session needs 1.1 GB instead of 1.9 GB and more or longer conversations fit. The default stays f16. Existing caches rebuild when the type changes. Flash Next only.
• In-engine abliteration (off by default). --refusal-direction on, or "In-engine abliteration (Qwen3.8 Flash Next)" on the Server page in the app, turns on the projection of a refusal direction out of the model's outputs. The direction file is in the refusal folder beside bubo.exe, and the zip includes it. It applies to Qwen3.8 Flash Next models that are not already abliterated. For other models, such as the Huihui builds and the Coder, Bubo refuses to start with it on and says why. A restart is needed.

Speed

Bubo v0.1.2, standard Flash Next file, default settings, 70 W, tokens/s: prose 57.1, Python 66.3, C++ 77.5, six-prompt mix 65.6, long agent session writing 66.2, code edit (rename) 109.0, code edit (comment) 97.6, 32K cold prompt 1,415, 2K cold prompt 1,320. Method and every row: Benchmarks.

Other changes

• More API: POST /apply-template; more fields in GET /props; repeated --served-model-name; IPv6 listen addresses. Tool calls: more than 128 tools work; integer arguments keep their written form; optional arguments stay; whitespace between parallel calls is not content. /health answers while the model loads.
• Thinking: when the model tries to end its thinking early, the server can add a short phrase so it keeps thinking, up to 2 times for each request. On in the default and subagents profiles, off in exact.
• Opt in: --enable-yarn lets --context go above 262,144 on Flash Next (without it Bubo refuses a larger context). bubo serve embedding serves EmbeddingGemma 2 on /v1/embeddings; Bubo does not ship the weights.
• Prompt reading on IQ and K-quant files is about 5 % faster by default.
• The repeat guard now also ends answers that repeat a passage of up to 128 tokens (it was 16).
• The memory check now counts the Q8 KV cache and the Coder exactly.
• --cache-ram-bytes off works: Bubo keeps no RAM checkpoint.
• Bubo checks a draft, projector or head file that you pick yourself from its headers before the model loads.
• A Q8 KV cache setting on a model that does not support it falls back to f16 instead of failing.
• Fixed: DiffusionGemma tool calls without the opening tool-call marker are now read as tool calls when they have the closing marker and a declared tool name (issue #11).
• Fixed: requests for token log probabilities (logprobs) could fail with a server error on Qwen3.8 Flash Next when MTP was on.
• bubo diagnose reports the revision and the compiler of the build.
• The lower-memory build writes 19–28 % faster than in v0.1.1, and reads a 32K-token prompt about 7 % faster (same laptop, 70 W).

Known issues

• Bubo is not code-signed. Windows can ask you to confirm. See Windows security prompts.
• If you use --cache-disk, the first start of v0.1.2 deletes the disk-cache files of v0.1.1 in that folder (*.kvc and temporary files, top level only), and the first requests after that run cold. Use a separate folder for each server.
• The option --cache-disk-staging-bytes is ignored on Qwen3.8 Flash Next, and the server logs a warning.
• Deep sessions and images need more memory. Reduce --context if memory is limited.

The full list: Known issues.

Install

Download bubo-v0.1.2-windows-x64-gfx1151.zip (71,952,963 bytes, SHA-256 153903f2a60820e15d126e6db8df7406aba325bbb77378e17d5bea1a87ca1a2d). After you extract it, bubo --version prints bubo 0.1.2 (53e4c3e4b), engine gufo 0.26.0.

Credits and licenses

Thanks to the gufo contributors and pixmaate/gufo, and to ISTA-DASLab for the GSQ and RCO quantizations (arXiv:2604.18556, arXiv:2605.00649).
Their code retains the MIT license.
AMD runtime components retain their separate licenses.
Model files retain the licenses of their authors. The Flash Next Coder uses the Qwen Community License 1.0.
New third-party notices: libsndfile, libsoxr and their audio libraries (LGPL and others), LLVM OpenMP, the IBM Plex Sans font (SIL Open Font License 1.1).

---

The v0.1.1 notes are in the release history of this repository: v0.1.1.

Halowin Bubo v0.1.1

Halowin Bubo v0.1.1 adds the Bubo app, eight more models, a memory guard and setup checks. It is faster than v0.1.0 on the same laptop.
It targets Strix Halo laptops with Ryzen AI Max and Radeon 8060S graphics at approximately 70 W.
The base is gufo v0.10.0 with selected upstream fixes.

Sampled output for a fixed seed can differ from v0.1.0.

Speed

Same laptop, same model file, approximately 70 W. Bubo v0.1.1 against Bubo v0.1.0:

Work	v0.1.1	v0.1.0	Change
Mixed work (agent, code, prose)	66.0	59.5	+11 %
Code edit, rename	107.5	97.3	+10 %
Code edit, comment change	97.4	90.8	+7 %
C++ code	74.8	69.7	+7 %
Cold 2K prompt	1,328	1,227	+8 %
Prose	56.4	52.9	+6 %
Cold 32K prompt	1,384	1,318	+5 %

Python code and reading new text in an agent session are level with 0.1.0. Speeds are tokens per second. Conditions and every row: Benchmarks.

New

• The Bubo app. start.cmd opens a desktop app. The app downloads the engine files and a model after you approve them, starts the server, and shows the connection address, the requests and the log. It starts the same bubo serve server as the command line. It can update Bubo with one click. See Use the app.
• Lower-memory Flash Next. The app lists a lower-memory build of Qwen3.8 Flash Next (Models, then Discover, then Download). It is a download of about 90 GB. At the default context of 200,000 tokens, Bubo estimates about 66 GiB of GPU memory for it instead of about 91 GiB, which is about 25 GiB less. It reads prompts at about 0.88 to 0.92 of the speed of the standard build. It writes code and agent text at about 0.75 to 0.9 of that speed, and prose at about the same speed. These are ranges from one run. On a 35-question reasoning check (GPQA Diamond, AIME 2026, HMMT 2026, IMO 2026) it answers 29 correctly, the same as the standard build and Bubo 0.1.0 within the noise of a set this small. Its dense tensors come from the Huihui model and its expert tensors from ISTA-DASLab's GSQ-RCO quantization of Qwen3.8 Flash Next. See Licences.
• Shorter wait for the first token. On the next request of a conversation, the first token arrives about 40 ms sooner at 32K tokens of context (527 ms instead of 567 ms, 7 % sooner) and about 113 ms sooner at 100K (515 ms instead of 627 ms, 18 % sooner). The wait in the queue falls from about 52 ms to 7 ms at 32K, and from about 130 ms to 8 ms at 100K. The output is identical.
• More models. The app downloads Qwen3.8 27B, Qwen3.6 27B, Cyber-Tiel Coder 35B-A3B, Gemma 4 26B-A4B and 31B (each in two builds: Unsloth QAT and Huihui abliterated QAT) and DiffusionGemma 26B E38 (abliterated, 4-bit). These are 16 to 23 GB each. All of them accept images. See Models supported.
• DiffusionGemma accepts images. Bubo downloads the vision file for DiffusionGemma with the model. In a test on the release package, a request with one image returned an answer that described the image correctly.
• 96 GB machines. Use the lower-memory build of Qwen3.8 Flash Next. Set Variable Graphics Memory (AMD Software: Adrenalin Edition > Performance > Tuning) to the largest value offered (72 GB if available, at least 64 GB), then restart Windows. At 72 GB, Bubo uses the default 200,000-token context for one session. At 64 GB it does too when about 16 GB of system memory is free, and picks a smaller context when less is free. These values are calculated: Bubo has not yet run on a 96 GB machine.
• Memory check before Start. A tight setup starts only after you tick the override for it. A setup that does not fit never starts.
• Live progress. The Requests page of the app lists each running request with its phase (Reading or Writing), its progress and its speed in tokens per second, and the Server page shows a one-line summary. bubo status --watch shows the same in a terminal.
• Smaller updates. An update keeps the files that did not change.
• Memory guard. Bubo keeps free memory above a floor of 4,096 MiB. A request that needs new memory when free memory is below the floor is refused with HTTP 503 (insufficient_memory, Retry-After: 5) or HTTP 507 (insufficient_memory_permanent). Under Windows low-memory pressure, Bubo drops cache snapshots first. It stops a running request only as the last step (HTTP 503, memory_pressure). GUFO_CACHE_HOST_FLOOR_MB=0 turns the guard off. See System setup.
• Memory estimates. bubo models estimate --model PATH prints a memory estimate and a recommended context. bubo models list shows the prepared packages and the models in your folders.
• Setup checks. bubo diagnose shows gpu_reservation, page_file and power_mode. bubo serve writes one WARN line at the start when the GPU memory reservation or the page file differs from the tested setup. A warning never stops the server.
• Cache details in the response. usage.gufo has cache_snapshots_skipped and cache_snapshot_skip_reason when the cache refuses a snapshot copy. cache_miss_reason names the cause when the cache removed or refused a checkpoint. See options.
• MTP for Gemma 4. A Gemma 4 package that has an MTP file uses MTP by default. --no-mtp turns it off. On the Huihui abliterated QAT builds, medium-length replies are about 1.1x faster on Gemma 4 26B-A4B (about 67 tokens/s instead of 60) and about 1.9x faster on Gemma 4 31B (about 21 instead of 11), with the draft model on and the same laptop.
• Gemma 4 images. Gemma 4 26B and 31B accept images through the vision file.
• Repair. bubo prepare replaces a damaged model file in the model folder after you approve the download. The plan lists the file as replaces a file that did not match. If you answer no, nothing changes (exit code 3). A file that you supplied, or a file outside the model folder, is not replaced.
• Stall watchdog. If generation stops making progress, the server exits as device_lost (exit status 75). The app starts the server again once.

Changed

• NPU prefill is off by default. After the prompt-reading speed-ups, it adds less than 1 %. Turn it on with --npu-prefill on. If you turn it on, the server uses it only with AMD XRT runtime 32.0.20102.3931. With any other runtime, the server writes a warning, reports the NPU as faulted, and uses the GPU. Decode does not change. See NPU prefill.
• Default RAM cache is 8 GiB (8,589,934,592 bytes) in the default profile. In v0.1.0 it was 6 GiB. The subagents profile stays at 4 GiB. An explicit --cache-ram-bytes value always wins.
• The RAM floor of the snapshot cache is 4,096 MiB by default and counts the host memory that a snapshot really uses. In v0.1.0 it was off.
• Model name. GET /v1/models lists the model's own id and bubo, and both names work. For another name, set --served-model-name.
• Default context. For Qwen3.8 Flash Next the default context is 200,000 tokens (v0.1.0: 262,144). The largest is still 262,144 (--context 262144).
• Image limit. On Qwen3.8 Flash Next the default limit is 65,536 image tokens for each request. All images in the prompt count. The limit reserves 640 MiB for each session. In v0.1.0 the reservation followed the context size. Change the limit with GUFO_MAX_IMAGE_TOKENS.
• --help works before bubo prepare runtime. The help text of every command is part of bubo.exe.
• Package layout. bubo.exe is in the bin folder. start.cmd is in the package folder. The engine files go to %LOCALAPPDATA%\Bubo\runtime\rocm-10.0.0-gfx1151 by default. Change the folder with --runtime-dir or BUBO_RUNTIME_DIR. See install.
• Disk cache options are no longer in start.cmd. Use bubo serve llm --cache-disk (see the quickstart).

Fixed

• The routed-lanerow fallback INFO line is written once for each format and shape, not once for each layer and pass.
• The device_lost log line says remedy=restart_process exit_status=75. In the app, the server starts again once, after 5 seconds.
• usage.completion_tokens_details.reasoning_tokens is in Chat Completions responses, for non-streaming and streaming requests.
• API: a max_tokens of -1 or null keeps the server default. CORS preflight answers with the headers that the page asks for. Errors on /v1/messages and /v1/messages/count_tokens use the Anthropic error body. Responses streams send a keepalive event.
• Streamed tool calls: when the model sent a tool call with an empty parameter value, the stream could end with an error ("generation failed") and no final event. The call now arrives with the empty value. Requests without streaming were not affected.
• /v1/responses: when a tool has no strict field, or strict is null, Bubo uses the tool schema as the client sent it. Optional properties stay optional. strict: true and strict: false work as before. /v1/chat/completions did not change. (upstream #643)
• The command line accepts empty Windows environment values.
• The model scan follows OneDrive folders. It refuses symbolic links and junctions only.
• The docs listed /v1/messages/count_tokens as not implemented. It works and returns input_tokens.

Known issues

• Bubo is not code-signed. Windows can ask you to confirm. See Windows security prompts.
• The repeat guard stops repeated text in answers o...

Halowin Bubo v0.1.0

Halowin Bubo v0.1.0

First release from HaloForge Labs.

Halowin Bubo brings the gufo engine to native Windows with AMD HIP.
It targets Strix Halo laptops with Ryzen AI Max and Radeon 8060S graphics at approximately 70 W.
The base is gufo v0.10.0 plus main commit 97b8645f.
The Windows layer comes from pixmaate/gufo windows-port.

Included

• Optimal defaults for Qwen3.8 Flash Next.
• Four profiles: default, stock, exact, and subagents.
• An OpenAI-compatible local API.
• All gufo model families listed in the README.
• Fast restarts with the optional disk prompt cache: a 30K-token conversation continues in about 2 s after a restart instead of processing the whole prompt again (about 25 s). It is off by default (start.cmd -CacheDisk <folder>). A saved snapshot is about 954 MB for 30K tokens. See the README section.
• RAM prompt cache.
• NPU prefill (on by default when usable): prefill is about 5 to 7 % faster on an 8K-token prompt and about 3.5 to 5.5 % faster on a 16K-token prompt at 70 W. Decode does not change. Prompts under 4,096 tokens do not use it. It needs the AMD NPU driver and XRT, and it is off in exact and stock. See NPU prefill.
• Optional grouped decode expert kernels, adapted from Strata (MIT): off by default. Turn them on with GUFO_EXPERT_V2K=1. They are not available in exact.
• Resolved settings output through --print-effective-settings.
• Windows notices and component licences.

Default settings

Item	Default
Profile	default
Sessions	1
Context	262144 tokens
Reasoning	Thinking on, medium effort
Adaptive thinking	Marker bias map low:-2,medium:-1,xhigh:0
Behaviour	Policies on, with the engine loop breaker
Speculative decode	MTP when the matching sidecar is supplied
MTP control	survival, draft vocabulary auto, prompt lookup on
Lenient acceptance	bounded, thinking spans with MTP
Dense twin and draft head	Use the matching derived files outside exact
RAM cache	6442450944 bytes (6 GiB)
Disk cache	Off
Local API	127.0.0.1:8080

The default output is not byte-exact.
Use --profile exact for reproducibility with the same build, machine, files, prompt, seed, and request settings.
Exact disables speculation, behaviour policies, the dense twin, the draft head, and prompt caches.
It uses one session and fixed prefill geometry.
Each request processes its full prompt.

Exact-profile result: in exact mode, Bubo gives the same tokens as the stock reference. We tested this on the release source tree (the full exact proof ran on build 163762d9; the release revision 3ff74dce differs from that tree only in the Windows launcher script and its test). It is not a promise for all builds:

• --exact matched the stock-flag reference on 24 fixed cases (short, 8K and 33K natural, thinking, tool, stop, 4095/4096/4097 edges, image, long decode, non-ASCII) at temperature 0 and 1, seed 1. There were five exact passes and one retained pass, and all 24 cases matched in each. The comparison covered text, reasoning, tool calls, token ids and finish reasons.
• Logprobs were compared only where the product supports them (thinking-off cases). The other cases were compared by text and token ids.
• The prompt and continuation cache is off in --exact. Cached restores are not bit-identical to a fresh prefill (3 of 3 measured cases differed) and are outside this guarantee.
• One machine (Windows, AMD Radeon 8060S, ROCm path). Other hardware, seeds and prompts are not covered. No speed claim.
• The exact and stock profiles refuse the NPU and grouped expert kernel switches and reject the tested production environment overrides (GUFO_DENSE_TWIN_MAX_N). Other user overrides were not tested.

Bubo is tuned for Qwen3.8 Flash Next. The other models run with upstream gufo settings.

Runtime and model setup

The package contains no AMD files or model weights.
Run bubo prepare. It downloads three official ROCm 10.0.0 wheels for gfx1151 (967.2 MiB) and eight pinned model files (108.6 GiB) after you approve a displayed plan.
It checks the SHA-256 hash of every file, resumes stopped downloads, honours HTTPS_PROXY, and writes bubo-package.json last.
The default checks find a wrong file, a wrong variant and a truncated file. --verify forces a full SHA-256 check.
The server finds the model files through the manifest. It never hashes model payloads at startup. --verify-model checks all eight files on request.

The dense twin and Q4_K draft head come from the HaloForge Labs Hugging Face repository (commit bb72204d).
Qwen licences apply to these derived model files.

Measured performance

The 70 W laptop measurements include 1,364 tokens/s for prompt processing (pp2048, release build v0.1.0, 2026-10-12) and 51.6 tokens/s across real agent requests (2026-10-10).
A copy-heavy rename edit reaches up to 97.4 tokens/s on its second request with the prompt cached (release build v0.1.0, 2026-10-12).
See the README for the conditions.

Reasoning check against stock gufo

Same 35 requests, same seeds, same laptop in Performance mode.

Set	Halowin Bubo correct	Stock gufo correct
GPQA Diamond	20/24	19/24
AIME 2026	3/4	2/4
HMMT 2026	3/4	4/4
IMO 2026 (final answer)	3/3	2/3
Total	29/35	27/35

Same or better accuracy, and about 24 % faster decode. The 95 % intervals overlap.

Fixed

• In rare cases, the thinking phase could not end and repeated text until the context limit.

Known issues

• The answer loop breaker detects repeat periods up to 16 tokens. Longer answer repeats can continue.
• Disk prompt cache is off by default to limit SSD writes. Automatic staging can limit snapshots near 70K tokens.
• Deep sessions and images need more memory. Use the documented memory setup or reduce context.

Credits and licences

Thanks to the gufo contributors and pixmaate/gufo.
Their code retains the MIT licence.
AMD runtime components retain their separate licences.
Model files retain the Qwen licence.

See install, quickstart, options, and profiles.