GPU embedding lookup writes garbage when invoked from HybridForwardPass

[pekkah]

Summary

The EmbedLookup / EmbedLookupQ4K Vulkan compute shaders produce garbage output when called from HybridForwardPass.Forward, but the same shaders produce correct output when called from GpuForwardPass.Forward.

Evidence

With instrumentation that downloads _gpuHidden immediately after the embedding dispatch:

[DBG] after-embed: nans=7 big=815 h[0..4]=-4.438109E-20,2.464776E-36,-1.299449E+08,1.798577E+24

Out of 2048 floats, 7 are NaN and 815 have absolute value > 1e6 - clearly not dequantized embedding values (which should be roughly in [-0.1, 0.1]).

A Clear before the embed lookup does zero the buffer (verified), and a Clear after the embed lookup also zeros it - so both shaders execute. The embed shader runs but writes wrong values. The same shader running from GpuForwardPass produces correct values for the same model.

What was ruled out

• Not a transfer/compute barrier issue (still happens with RecordComputeCopy everywhere)
• Not a buffer-aliasing bug (_buffers dictionary entries verified unique)
• Not an upload-ordering bug (moved embedding upload to after all layer uploads - same garbage)
• Not specific to the Q4_K shader (EmbedLookup produces identical garbage)
• Not a descriptor-set reuse issue (_embedLookupQ4KPipeline._reusableDs is updated only once per Forward; embedding is the only call)

Workaround in place

HybridForwardPass.ShouldKeepFixedWeightsOnCpu now always returns true, forcing CPU dequantization of the single embedding row per token. Cost is negligible (one row vs. all the layer compute).

TODO

Find the actual cause. Suspect something about descriptor-pool / pipeline state that differs between HybridForwardPass and GpuForwardPass constructor sequences.

[pekkah]

Status update (workaround retained, root cause still unknown):

Tried to remove the ShouldKeepFixedWeightsOnCpu = true workaround after fixing the descriptor-set reuse hazard in ComputePipeline.RecordWith (commit 67a1e68 — which did resolve #2's MoE NaN cascade). The existing HybridForwardPassTurboQuantRunsPastFp32Window test passed with the workaround off because it only checked float.IsFinite(logit), but the CLI smoke caught it immediately:

dotnet run --project src/SharpInference.Cli -c Release -- \
  -m models/SmolLM2-1.7B-Instruct-Q4_K_M.gguf -g 1 -p "The capital of France is" --temp 0
# Prefill: 36 tokens | Decode: 0 tokens   ← workaround off, GPU embed path

vs. all-GPU and CPU-only paths producing "Paris." correctly. So the DS hazard wasn't this issue's root cause.

Workaround restored with an updated TODO comment in HybridForwardPass.ShouldKeepFixedWeightsOnCpu. The forward-pass coherence tests were strengthened (argmax != EOS at first decode, multi-token decode produces variety) so the next attempt has a real safety net.

Root cause is now narrower: it's either the EmbedLookupQ4K shader misbehaving when invoked from HybridForwardPass.Forward (the original suspect) OR the Phase-5 final GPU norm + output projection writing wrong logits — both code paths only execute when the workaround is off. See #19 for the narrowed reproduction details.

[pekkah]

Superseded by #19, which captures the post-#2 narrowing (descriptor-set hazard fixed, bug specific to HybridForwardPass embed lookup). Closing here so the investigation lives in one place.