Full SwiGLU MLP Block + Unified GPU Orchestration

Complete Layer 0 MLP Block

This release implements the complete SwiGLU Multi-Layer Perceptron block for Layer 0 of microsoft/bitnet-b1.58-2B-4T, running natively in the browser via WebGPU. The full pipeline:

embedding(2560) → gate_proj(6912) ─→ SiLU·mul(6912) → down_proj(2560)
                → up_proj(6912)   ─┘

All three ternary weight matrices are extracted, bit-packed (16 weights per u32), and served as static .bin files:

• gate_proj — 6912×2560 (4,320 KB)
• up_proj — 6912×2560 (4,320 KB)
• down_proj — 2560×6912 (4,320 KB)

Unified GPU Orchestration (Zero-Copy Pipeline)

The critical architectural optimization in this release: all four compute passes execute in a single WebGPU command encoder submission. Intermediate tensors (gate_out, up_out, silu_out) never leave VRAM — eliminating the GPU↔CPU "ping-pong" that was adding ~35ms of PCIe bus latency per step.

Before (v0.4.0 approach): 55.1ms — 4 separate submissions with CPU readback between each
After (unified): 7.2ms — single submission, single readback at the end

Benchmark Results (Warmed Cache)

Compute time for the full SwiGLU MLP block (52.7M ternary parameters, 4 compute passes):

Device	GPU	Setup	Compute	Total
MacBook Pro 14" (2023)	M2 Max 30-core	0.8ms	6.4ms	7.2ms
iPad 13"	M3 10-core	2.0ms	16.0ms	18.0ms
iPhone 14 Pro Max	A16 5-core	9.0ms	29.0ms	38.0ms

Bit-exact determinism — output values are identical across all three devices down to the last decimal place.

Real Semantic Embeddings (from v0.4.0)

Carried forward from the real-embeddings sprint:

• 16,384-word "Vocab-Slice" dictionary using sparse FP16 embeddings
• Successfully bypasses mobile WebKit out-of-memory (OOM) limits
• Browser Cache API (304 Not Modified) for instant reload

New Files

• extract_full_mlp.py — Python script to extract & pack all 3 MLP weight matrices from HuggingFace safetensors
• bitnet_layer_0_gate_proj.bin — packed gate_proj weights
• bitnet_layer_0_up_proj.bin — packed up_proj weights
• SHADER_SILU_MUL — new WGSL compute shader for SwiGLU activation fusion

What's Next

• LM Head — map the MLP output back to vocabulary logits so the model can "speak"
• RMSNorm — proper layer normalization for correct magnitudes
• Attention block — the other half of a transformer layer