Help wanted: separating Motif-3 task behavior from 2.28 bp/e quantization loss

[hebo1221]

I released a directly converted Motif-3 GGUF that runs the full 314.7B-parameter
core on one 128 GB DGX Spark. The deployment result is reproducible, but the
next question is about attribution rather than fit or speed.

What is already established

• The artifact is an 83.56 GiB mixed IQ2_XXS GGUF at 2.2805 effective bits per
  element.
• The conversion path was official BF16 -> BF16 GGUF -> IQ2_XXS. It did not
  pass through a Q5 checkpoint or use requantization.
• On a fixed 49,152-token held-out comparison against BF16, it measured a
  1.411541 +/- 0.010497 perplexity ratio and 0.454130 +/- 0.004009 mean KL
  divergence. Both missed the preservation limits fixed before the run.
• The current GGUF contains the target core but not the native MTP head. That
  is a separate speculative-speed limitation, not an explanation for the
  target model's measured distribution shift.

So there is already direct evidence that this ultra-low-bit artifact loses
information relative to BF16. What is not yet established is how that loss maps
to downstream behavior, or which tensor groups are responsible for most of it.

Questions I would like to answer

1. On the same frozen task cases, which failures are already present in BF16
   or Q5, and which appear only in the 2.2805 bp/e artifact?
2. Which routed experts, shared tensors, attention tensors, or layers dominate
   the BF16-to-IQ2 KL increase?
3. Would an expert-aware mixed-precision build in roughly the 95-110 GB range
   produce a meaningfully better quality/size trade-off on a single Spark?
4. Can another DGX Spark reproduce the published throughput and memory result?

Contributions that would be especially useful

Paired BF16 or Q5 task evaluation

If you can run the official BF16 checkpoint or a traceable Q5 build, a paired
evaluation on the same frozen cases would let us separate base-model behavior
from quantization loss. Hardware does not need to be a DGX Spark for this arm.

Quantization-policy review

I would value concrete suggestions for expert-aware or layer-aware bit
allocation, especially methods that use teacher KL or causal tensor overlays
rather than activation magnitude alone. A larger artifact is acceptable if it
still fits safely on a 128 GB Spark.

Independent Spark reproduction

If you have a DGX Spark or comparable 128 GB NVIDIA system, please run the
included benchmark and submit the raw JSONL through the benchmark report form.
Results that disagree with mine are just as useful as matching results.

What to include with a result

Please include the model or shard digest, source revision, runtime repository
and commit, complete launch arguments, hardware configuration, prompt/output
token counts, and raw output where possible. A one-number result without the
configuration is difficult to compare.

• Project and evidence: https://github.com/hebo1221/motif3-dgx-spark
• Model: https://huggingface.co/jhkim55/Motif-3-Direct-IQ2-XXS-DGX-Spark
• Structured benchmark report: https://github.com/hebo1221/motif3-dgx-spark/issues/new?template=benchmark.yml

The goal is not to defend this quantization. It is to determine, with
reproducible evidence, which limitations belong to Motif-3 itself and which
belong to the single-Spark deployment trade-off.