Model Request: Qwen3.5 family
Qwen has released the Qwen3.5 model family with state-of-the-art performance that surpasses GPT-5-mini and Qwen3-235B on many benchmarks.
Available Models
| Model |
Total Params |
Active Params |
Architecture |
| Qwen3.5-35B-A3B |
35B |
3B |
MoE (256 experts, 8+1 active) |
| Qwen3.5-27B |
27B |
27B |
Dense hybrid |
| Qwen3.5-122B-A10B |
122B |
10B |
MoE |
Why it matters for WebLLM
Qwen3.5-35B-A3B with only 3B active parameters would be ideal for browser-based inference. Performance exceeds Qwen3-235B-A22B on agent tasks, coding (SWE-bench 69.2%), and multilingual benchmarks while requiring minimal compute per token.
New Architecture (requires TVM compiler support)
Qwen3.5 introduces a hybrid architecture different from standard transformers:
-
Gated DeltaNet (linear attention) — 75% of layers
linear_conv_kernel_dim: 4, separate Q/K heads (16) and V heads (32), head_dim: 128- More memory-efficient than standard KV-cache
-
Gated Full Attention — 25% of layers
- GQA with
num_attention_heads: 16, num_key_value_heads: 2, head_dim: 256
- Partial rotary factor 0.25, interleaved mRoPE
-
Sparse MoE: 256 experts, 8 routed + 1 shared, moe_intermediate_size: 512
-
Multi-Token Prediction (MTP): mtp_num_hidden_layers: 1
-
Native multimodal (vision encoder built-in)
Config
- Architectures:
Qwen3_5MoeForConditionalGeneration / Qwen3_5ForConditionalGeneration
- Model type:
qwen3_5_moe / qwen3_5
- Layer pattern:
[linear_attention × 3, full_attention × 1] repeating
- HuggingFace: https://huggingface.co/Qwen/Qwen3.5-35B-A3B
Would love to see this in WebLLM — the 3B active MoE variant would be a game-changer for in-browser AI agents. Thank you for the amazing work! 🙏
Model Request: Qwen3.5 family
Qwen has released the Qwen3.5 model family with state-of-the-art performance that surpasses GPT-5-mini and Qwen3-235B on many benchmarks.
Available Models
Why it matters for WebLLM
Qwen3.5-35B-A3B with only 3B active parameters would be ideal for browser-based inference. Performance exceeds Qwen3-235B-A22B on agent tasks, coding (SWE-bench 69.2%), and multilingual benchmarks while requiring minimal compute per token.
New Architecture (requires TVM compiler support)
Qwen3.5 introduces a hybrid architecture different from standard transformers:
Gated DeltaNet (linear attention) — 75% of layers
linear_conv_kernel_dim: 4, separate Q/K heads (16) and V heads (32),head_dim: 128Gated Full Attention — 25% of layers
num_attention_heads: 16, num_key_value_heads: 2, head_dim: 256Sparse MoE: 256 experts, 8 routed + 1 shared,
moe_intermediate_size: 512Multi-Token Prediction (MTP):
mtp_num_hidden_layers: 1Native multimodal (vision encoder built-in)
Config
Qwen3_5MoeForConditionalGeneration/Qwen3_5ForConditionalGenerationqwen3_5_moe/qwen3_5[linear_attention × 3, full_attention × 1]repeatingWould love to see this in WebLLM — the 3B active MoE variant would be a game-changer for in-browser AI agents. Thank you for the amazing work! 🙏