Repository navigation
Hy4-Flash now starts on NVIDIA Blackwell cards (RTX 50 / RTX PRO 6000) and decodes faster on small cards and on several GPUs, DeepSeek reads pictures much faster in long chats, and there are fixes from the community and for the Monitor.
- Hy4-Flash on NVIDIA Blackwell (RTX 50 / RTX PRO 6000) (#1073). On Blackwell cards built with CUDA 12.9 or 13.0 the engine stopped at its warm-up; it now starts and runs. After a GPU fault the engine's error also names the launch that failed first.
- Faster small cards. Where the CPU computes the experts that do not fit the card, Hy4-Flash decodes faster with new CPU kernels for every expert format Maya's sizes use, chosen and checked at the start on your CPU: about +20-27% on an 8 GB RTX 3050 with Maya-XS, +8% on one Tesla V100 and +10-14% on two (Maya-M), and +7-10% on a Tesla V100 beside a Gorgon Halo PC. Same answers as before.
- Faster Hy4-Flash on several GPUs (#1068). On three or more GPUs that cannot hold every expert,
--tp autonow runs tensor parallel with the CPU computing the rest: about +12% decode and up to +26% prompt reading on three Tesla V100s (Maya-M, 16K prompt). Two GPUs keep the layer split by default;HY4_TP_LANE=1/0switch it. - Faster Hy4-Flash on 4 or 5 GPUs (#1062). Each expert now stays whole on one GPU in these groups, so they need less VRAM and run faster.
HY4_TP_EXPERTS=sliceskeeps the old layout. --tp autotries one GPU fewer (#1078). Where the largest group of GPUs cannot hold the model,--tp autonow tries tensor parallel on one GPU fewer before the layer split: on a contributor's three 24 GB cards (Maya-S with MTP) decode was about +80%.HY4_TP_DOWN=0keeps the old choice.- Pictures in long DeepSeek chats (#1007). Conversations with pictures keep their restore points, so a new picture no longer makes the engine read the whole chat again: in our test (a 24.7K-token agent chat) a new picture read 284 tokens instead of 24,743.
- Opt-in: faster drafting on small cards (
--fast-drafting on). Where the small DFly2 drafter does not fit at your context but would with an 8-bit cache, the setup now asks "Faster drafting?" (the default keeps the cache as it is): about +8% to +28% decode on an 8 GB RTX 3050 at 8K. With the 8-bit cache the wording can differ slightly. - Setup on Windows counts the VRAM the engine really gets. The setup no longer picks a drafter the engine then leaves out on 8 GB cards; Maya measures the part Windows keeps and remembers it per PC.
- 4-bit expert copies decided once the CPU is warm. Whether they are faster on your PC is now timed during the first answers and remembered, so the choice is the same on every start; where they do not help, none are built.
- DeepSeek. No more hang right after the load on two NVIDIA GPUs (#1006); the engine recovers after a failed peer all-reduce (#1059); with
--parallel, a request sent again after its client gave up finds its own conversation and the dropped one stops at once (#1057); faster EXL3 kernels on AMD (about +11% prompt reading on a Radeon 8065S; #968, #1000, #1008); faster starts where the page cache is full (#1042: 296 to 105 seconds on the contributor's PC); the opt-in fused step also runs on EXL3 packs (#1001);cold=1for benchmarks (#1012); the vision tower's AMD streams are made at the first picture (#1070); a file whose unbuffered reads are refused is named once (#1076);DS4_FLOW_COPY_DMAcovers RAM copies across CPU sockets (#958). - Builds and setup. Maya builds on Rocky Linux / RHEL 9 (#1080); the Windows setup finds Visual Studio 2026 Insiders / Preview (#1067).
- Fixes. The Monitor's GPU readings no longer keep restarting when one reading fails, and
--reporthas a "GPU readings" section; an engine that aborts is no longer blamed on RAM; request statistics count every engine call after a tool call or resumed answer (#1058);HY4_EG_POPULATEworks on Windows (#1069). - Dashboard. Hy4-Flash shows Tencent Hunyuan's mark, like the other models' makers.
- Credits. The fix for Hy4-Flash on Blackwell cards (#1073) was contributed by @wegvon; tensor parallel with the CPU lane (#1068) and whole experts on 4 or 5 GPU tensor-parallel groups (#1062) by @alesha-pro; community fixes: #1067 by @cobra91, #1059 by @LostVector, #968, #1000, #1001, #1008 and #1012 by @pepuscz, #958 by @PrimusInterParis, #1070 by @paltian, #1042 by @Licenser, #1078 by @Antero128, #1080 by @OscarSheng. The rest of this release is by @mw00 (peasantsmith).
Full list: CHANGELOG.md.
Supported by @wjHunter, @os-groot, @davedgd, @jeanluchayes, @sugattor, @vikcious, @ThatSimpleTech, @router0mail and @davidfernengel.
Maya is free; if it helps you, you can sponsor it: https://github.com/sponsors/mw00