Skip to content

v1.7.1

Choose a tag to compare

@github-actions github-actions released this 12 Jun 22:51

Added

  • linalg.WrapInt8 / linalg.WrapInt4 — zero-copy constructors for
    already-quantized WeightMat weights (Experimental).
    The inverse of the
    Int8()/Int4() accessors: they wrap caller-owned, pre-quantized slices (int8
    • per-row scales; or packed int4 nibbles + per-group scales + group size)
      without copying or re-quantizing, aliasing the caller's backing arrays the
      same way WrapF32 does. This fills the gap that blocked the goinfer
      decoder.weightMatWeightMat migration: 1.7.0 shipped only quantize-from-f32
      constructors (QuantizeInt8/QuantizeInt4), but the decoder's .giw
      deserialization reads weights already quantized and zero-copy-aliases the int4
      nibbles straight off an mmap'd blob
      — re-quantizing from f32 would have
      regressed both that fast load and its OS-page-cache residency. The wrap path
      preserves both. Shape-validated (panics on a mismatched length, like
      QuantizeRowsInt8). Additive.