dasLLAMA: Qwen 3.5/3.6 support (qwen35/qwen35moe — Gated DeltaNet hybrid)#3448
Merged
Conversation
Qwen 3.5/3.6 ships tokenizer.ggml.pre="qwen35": qwen2 single-digit splitting,
but letter runs are [\p{L}\p{M}]+ and the punctuation class excludes \p{M}
(llama.cpp unicode_regex_split_custom_qwen35). Adds cpt_is_mark (the 0x10
accent-mark flag was already in the RANGES transcode), threads a `marks` flag
through pretok_split's letter and punctuation rules, dispatches on pre=="qwen35",
and gates against the llama.cpp ggml-vocab-qwen35 corpus.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Adds support in dasLLAMA’s BPE pre-tokenizer for the Qwen 3.5/3.6 regex variant where letter runs absorb Unicode combining marks (i.e., [\p{L}\p{M}]+), matching llama.cpp’s unicode_regex_split_custom_qwen35. This extends the existing shared llama3/qwen2 pre-tokenizer logic while keeping existing pre-tokenizers byte-identical by default.
Changes:
- Introduces
cpt_is_mark(\p{M}) indasllama_unicode.dasbased on the existing range-flag table. - Threads a
marksswitch through the sharedpretok_splitletter + punctuation rules, and dispatches it fortk.pre == "qwen35". - Adds a tokenizer corpus gate test for
ggml-vocab-qwen35.gguffixtures.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| tests/dasLLAMA/test_tokenizer.das | Adds a new corpus gate test for the qwen35 tokenizer fixtures. |
| modules/dasLLAMA/dasllama/dasllama_unicode.das | Adds cpt_is_mark to classify Unicode combining marks via the transcoded range flags. |
| modules/dasLLAMA/dasllama/dasllama_bpe.das | Extends the shared pre-tokenizer split logic with a marks mode and enables it for pre == "qwen35". |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
borisbat
changed the title
borisbat
marked this pull request as draft
July 12, 2026 15:41
…0.8B The qwen35 arch (Qwen 3.5/3.6): 3 of every 4 layers are Gated-DeltaNet recurrent linear attention (causal conv1d k=4 + SiLU, per-v-head delta rule on f32 128x128 state, z-gated RMSNorm out), the rest full attention with a 2x-wide [q|gate] projection (out ⊙ σ(gate) before wo) and partial NEOX rope (64 of 256 dims; text-only M-RoPE degenerates per the qwen3vl precedent). - Config: recr_mask/rope_dim/q_gated + ssm geometry, arch opt-ins hybrid_deltanet/partial_rope; layer_n_kv_heads = 0 on recurrent layers makes every KV prefix walk (session slabs, pool blobs, q8 validation) allocate zero bytes for them - Loader: recurrent-layers mask (array or full_attention_interval), partial-rotary opt-in (others still refuse), ssm_* tensors, panic on MTP tensors; per-layer q8 repack covers the deltanet matrices and skips -1 attention slots (both repack walks) - Session: per-recurrent-layer conv history + S state (forward-only; dn_pos guards rewind, position 0 resets); prefix cache refuses hybrid models - Kernels: softplus/l2_norm_rows (ggml-exact), partial-rope leaves (direct + table), dn_conv_step / dn_delta_head matching ggml's float accumulation order; eval_batch rides the attn_is_std=false fallback - qwen35moe_blocks pre-wired (registration lands with the 35B stage) Parity vs llama.cpp simple_ids on Qwen3.5-0.8B-Q8_0: 40/40 token-for-token on the counting fixture (frozen in test_parity), the high-entropy "Once upon a time" prompt, and the both-sides-q8_0-KV arm. Full suite 42 tests / 0 failed — existing arches undisturbed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Registers qwen35moe: the qwen35 hybrid attention + the existing routed-MoE FFN (softmax top-8-of-256 renormalized + the qwen2moe-style sigmoid-gated shared expert, both metadata/tensor-detected). Loader deltas the 35B file surfaced: feed_forward_length is optional for pure-MoE arches (every FFN is router + experts), and F32-on-disk ssm_beta/ssm_alpha stay fp32 in fblob and matmul via mm_fblob (llama.cpp matmuls them f32; the q8 path remains for quantized files like the 0.8B). First model with v-heads (32) != k-heads (16), exercising the modulo GQA-repeat. Parity vs llama.cpp simple_ids on Qwen3.6-35B-A3B-UD-Q4_K_M (Q4_K/Q5_K/Q6_K kq tier, x64-gen grp8 repack): 40/40 token-for-token; fixture frozen in test_parity running BOTH MoE prefill paths (grouped + reference). 0.8B fixture stays green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
DAS_JOBQUE_AFFINITY unset now means 0 (no placement) instead of 1 (ideal-CPU hint) — a library should not impose thread placement on host processes by default; boxes that want the SMT-pair spread opt in with DAS_JOBQUE_AFFINITY=1 (hint) or 2 (hard mask). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…35 perf) The deltanet prefill gets llama.cpp's chunked form (its own prefill default, CS=64) behind set_deltanet_prefill_mode: per chunk the in-chunk delta interactions solve as a unit-lower triangular system (forward substitution), outputs and the state advance become register-tiled gemm_f32 calls, and the f32 state is touched once per chunk instead of per token. The causal conv batches over the window (tap-major weights, float4 rows) with the persistent history handled at both seams. Per-v-head slabs, threaded over heads. Chunked is the runtime default (matching llama.cpp); the parity fixtures pin recurrent (the ggml-float-order bit-exact form) and every hybrid fixture adds an explicit chunked arm. Decode: the per-token delta rule now threads over v-heads work-proportionally — heads own disjoint state slabs, and keeping workers hot through the formerly-serial section speeds the surrounding dispatches too. 16 threads on the 3990X (llama-bench -t 16 as reference): - Qwen3.5-0.8B-Q8: pp512 225 -> 756 tok/s (llama.cpp 520), tg64 40.5 -> 54 (41.2) - Qwen3.6-35B-A3B-Q4KM: pp512 65 -> 97 tok/s (llama.cpp 105), tg64 12.3 -> 16.4 (12.7) tests/dasLLAMA/test_deltanet.das: chunked==recurrent on a 150-token multi-chunk window (argmax + 8-token greedy lockstep, proving the state handoff into decode); logit divergence measured in-family with the accepted flash-vs-classic attention spread. test_kv_paged stack bump: the Stage-2 Session dn_* fields tipped the eval_batch frame over 64K (pre-existing on the branch, surfaced by the full-directory sweep). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
borisbat
marked this pull request as ready for review
July 12, 2026 18:38
recurrent_layers array length now validated like feed_forward_length; full_attention_interval must be positive (a zero from a malformed file was a % 0 — hardware exception under JIT, not a clean panic); conv_dim must be a multiple of 4 at load (the batched conv reads float4 rows through raw pointers, outside das bounds checks). Plus the RANGES bit legend gains the 0x10 mark flag cpt_is_mark reads. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
group_count 0 in a malformed file was another % 0 in the geometry check; state_size/group_count/time_step_rank now validate positive first, closing the class — no unvalidated metadata feeds a modulo in the hybrid block. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…en35 # Conflicts: # modules/dasLLAMA/benchmarks/prefill_perf.das
A pure-MoE file (feed_forward_length absent, hidden 0) that also omitted the expert_feed_forward_length / expert_shared_feed_forward_length keys failed open to zero-width experts through the hidden-derived fallbacks; both widths now validate positive with a clear panic. 35B load smoke: n_ff_exp=512, n_ff_shexp=512 unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
pull Bot
pushed a commit
to forksnd/daScript
that referenced
this pull request
Jul 13, 2026
…GaijinEntertainment#3448) Every dasLLAMA test that requires the transformer umbrella failed the full AOT sweep with error[50101] on dasllama_arch_qwen35 symbols — the module landed without its tests/aot/CMakeLists.txt registration (per-PR CI only builds the tests/language subset, so the gap was invisible until the nightly/preflight full sweep). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Qwen 3.5/3.6 support in dasLLAMA: the
qwen35/qwen35moehybrid architecture — 3:1 Gated DeltaNet (linear attention, recurrent state, no KV) : gated full attention (GQA head_dim 256, partial RoPE 64/256, sigmoid output gate), 248k vocab, MoE variant with 256 experts + shared expert. Unlocks the whole Qwen3.5 open lineup (0.8B/4B/9B/27B) plus Qwen3.6-35B-A3B / 27B.One PR, staged commits:
pre="qwen35": letter runs absorb combining marks ([\p{L}\p{M}]+);cpt_is_mark+marksflag inpretok_split; byte-exact vs llama.cppggml-vocab-qwen35corpus, existing corpora unchanged (6/6)qwen35— per-layer recurrent mask (recr_mask, zero KV vialayer_n_kv_heads), partial NEOX rope (direct + table leaves), 2×-wide gated Q, Session-resident conv/delta f32 state (forward-only, rewind-guarded), deltanet kernels matching ggml's float-accumulation order, prefix-cache refusal for hybrids. Bit-exact on Qwen3.5-0.8B-Q8_0: 40/40 vssimple_idson the frozen counting fixture, the high-entropy "Once upon a time" prompt, and the both-sides-q8_0-KV arm; full parity suite 42 tests / 0 failedqwen35moe— existing top-8 softmax+renorm router + sigmoid-gated shared expert; optionalfeed_forward_length(pure-MoE files), F32ssm_beta/ssm_alphafp32 matmul rail, modulo GQA-repeat (32 v-heads over 16 k-heads). Bit-exact on Qwen3.6-35B-A3B-UD-Q4_K_M: 40/40, fixture frozen running both MoE prefill paths (K-quant expert stacks on the native kq tier)gemm_f32, f32 state touched once per chunk instead of per token) behindset_deltanet_prefill_mode— chunked is the runtime default like llama.cpp, the parity fixtures pin recurrent (the bit-exact ggml float order) and every hybrid fixture runs an explicit chunked arm; batched causal conv (tap-major weights, float4 rows, history handled at both seams); decode delta rule threaded work-proportionally over v-heads. Newtest_deltanet.dasproves chunked==recurrent on a 150-token multi-chunk window (argmax + 8-token greedy lockstep across the state handoff into decode). Also: jobque worker affinity now off by default (DAS_JOBQUE_AFFINITY=1|2opts in).Perf at 16 threads on the 3990X, vs llama.cpp
-t 16(fdb1db877) on the same files — spreads are the placement lottery with affinity off;DAS_JOBQUE_AFFINITY=1lands the top of the range:The 35B prefill residual vs llama.cpp is the MoE expert GEMMs (
mm_gemm55% of the profile), not deltanet — that's kq-tier territory (#3444 follow-ups). Deferred beyond the arc: MTP self-spec decode, vision mmproj, prefix-cache state snapshots for hybrids, YaRN long-ctx.🤖 Generated with Claude Code