Skip to content

Expose TensorStats validation for all formats via apr rosetta #175

@noahgift

Description

@noahgift

Summary

The TensorStats validation infrastructure exists in src/format/validation.rs but is only accessible for APR format files. GGUF and SafeTensors files cannot be validated via CLI.

Current State

Implemented in validation.rs:

pub struct TensorStats {
    pub mean: f32,
    pub std: f32,
    pub nan_count: usize,
    pub inf_count: usize,
    // ...
}

impl TensorStats {
    pub fn has_no_nan(&self) -> bool
    pub fn has_no_inf(&self) -> bool
    pub fn is_valid_layernorm_weight(&self) -> bool  // mean in [0.5, 3.0]
    pub fn is_valid_layernorm_bias(&self) -> bool    // mean in [-0.5, 0.5]
    pub fn is_valid_linear_weight(&self) -> bool     // |mean| < 0.1
}

CLI Limitations:

Command GGUF SafeTensors APR
apr validate --quality ⚠️ Partial
apr tensors --stats
apr rosetta inspect ✅ (no stats) ✅ (no stats) ✅ (no stats)

Requested Feature

Add apr rosetta validate or extend apr rosetta inspect --stats to:

  1. Compute tensor statistics for any format (GGUF, SafeTensors, APR)
  2. Check physics constraints per APR-SPEC section 10.9:
    • LayerNorm weights: mean in [0.5, 3.0]
    • LayerNorm bias: mean in [-0.5, 0.5]
    • Attention weights: |mean| < 0.1, std in [0.02, 0.10]
  3. Detect anomalies: NaN, Inf, all-zeros tensors
  4. Output validation report with pass/fail per tensor

Example Usage

# Validate any model format
apr rosetta validate model.gguf --stats

# Output:
# === Tensor Statistics Validation ===
# 
# Tensor                          Mean      Std    NaN   Inf   Status
# ─────────────────────────────────────────────────────────────────────
# blk.0.attn_norm.weight          1.02     0.15     0     0   ✅ PASS
# blk.0.attn_q.weight             0.001    0.04     0     0   ✅ PASS
# blk.0.ffn_norm.weight          11.03     0.18     0     0   ❌ FAIL (mean > 3.0)
# 
# Result: 337/339 tensors valid (2 anomalies detected)

Justification

Per APR-SPEC section 10.10:

"Better to fail early than produce a 'zombie' model."

Without cross-format validation, users cannot detect:

  • Corrupted downloads (bit flips)
  • Conversion errors (incorrect scaling)
  • Quantization failures (NaN injection)

References

  • APR-SPEC section 10.9: Expected Tensor Statistics
  • APR-SPEC section 10.10: Conversion Validation Requirements
  • src/format/validation.rs: Existing implementation

Filed from apr-model-qa-playbook certification process.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions