feat: MLA absorption for DeepSeek V2/V3 — fuse low-rank Q/K/V into standard dense tensors

[mvkorobkov]

Summary

• gqa_attention_asym — new attention kernel in larql-inference that handles asymmetric qk_head_dim / v_head_dim (required for absorbed MLA tensors where Q/K use 192-dim heads but V uses 128-dim heads in DS-V3)
• MLA geometry fields in ModelConfig — qk_nope_head_dim, qk_rope_head_dim, v_head_dim parsed from config.json; DeepSeekArch exposes them via trait methods
• mla_absorb — new module in larql-vindex that fuses the four DS-V2/V3 low-rank attention projections (kv_a, kv_b, q_a, q_b) into standard dense Q/K/V weight matrices
• write_model_weights — F32 weight writer now accepts MLA architectures: detects full geometry, runs absorption per layer, writes absorbed Q/K/V under standard key names so the loader needs no MLA awareness

Why absorption

DS-V2/V3 stores attention as four low-rank matrices. Absorbing them into standard Q/K/V at extraction time means:

• Inference path is uniform — no special MLA forward pass at runtime
• Loader is unchanged — reads Q/K/V tensors as for any Llama/Mistral model
• One-time compute cost at extraction, not at every inference step

Correctness

Key details:

• kv_a rope-K is MQA (one shared row for all KV heads, not per-head) — replicated num_kv times when building absorbed K
• DS-V3 native per-head layout is [nope | rope]; LARQL convention is [rope | nope] — absorption reorders symmetrically for both Q and K
• Equivalence proven by absorbed_forward_matches_reference test: reference MLA forward pass vs absorbed path through gqa_attention_asym must agree within 1e-4 (f32 precision)

Test plan

• cargo test -p larql-inference -- gqa_attention_asym — 4 tests (shape, finite, sym-equivalence, causal)
• cargo test -p larql-vindex -- mla_absorb — 3 tests (forward equivalence, shapes, rope broadcast)
• cargo test -p larql-models — existing DS-V3 detection tests extended with new geometry accessors
• cargo test -p larql-vindex — 971 tests, 0 failures

[chrishayuk]

Hey @mvkorobkov — same situation as #103: branch is conflicting against current main (mergeStateStatus: DIRTY). The MLA absorption work itself is substantial (~1000 lines) and a clean rebase would be the right way to land it.

Could you rebase against current main? If you'd rather, I can also cherry-pick onto a fresh branch with your attribution preserved on the commits. Let me know which you prefer.

Once rebased, I'll do a proper review of the gqa_attention_asym kernel + the DeepSeek geometry plumbing.

[mvkorobkov]

Rebased onto current main (810f163). Branch now contains 5 focused MLA commits (656 insertions, 10 deletions):

• fix(gguf): map deepseek_v4/deepseekv4 arch string to DeepSeekV4Arch
• feat(gqa): add gqa_attention_asym for MLA-absorbed asymmetric head dims — relocated to larql-compute/src/attention/gqa.rs to match post-ADR-0022 layout
• feat(mla): add qk_nope/rope/v_head_dim fields for DS-V3 MLA absorption
• feat(vindex): MLA absorption — fuse DS-V3 low-rank Q/K/V into dense weight matrices (mla_absorb.rs, 327 lines + 3 tests)
• feat(vindex): wire MLA absorption into f32 weight writer

Dropped two commits that didn't belong here: Q3_K/Q5_K dequant (that's PR #103) and GGUF streaming for MoE extraction (separate feature, will open as its own PR if useful).

cargo check --workspace and the new gqa_attention_asym + mla_absorb tests pass (7 new tests, all green). Ready for the gqa_attention_asym + DeepSeek geometry review.