Skip to content

v3.10.0

Choose a tag to compare

@wpferrell wpferrell released this 19 May 02:25
· 50 commits to main since this release

v3.10.0 ships CLI improvements: new stat and diff subcommands, a fast verify --fast mode that runs in seconds even on multi-GB shards, a richer benchmark with per-layer-type breakdown, and --no-progress on the IO-heavy commands.

Added

  • bigsmall verify --fast — header-only integrity check (offsets, codec names, index consistency). Runs in seconds on multi-GB shards. --verbose adds counters on the pass path. Also importable programmatically as bigsmall.verify.verify_fast(path).

  • bigsmall stat <file.bs> — detailed per-tensor table (name, shape, dtype, codec, raw, cmp, ratio) plus summary footer with codec breakdown. Flags: --tensor <substring> filters, --sort {ratio,size,name}, --reverse.

  • bigsmall diff <a.bs> <b.bs> — three-way structural diff (identical / changed / only-in-A / only-in-B). Header-only — does not decompress. Exit 0 identical, 1 if any difference.

  • bigsmall benchmark enhancements — per-layer-type breakdown table (attn_qkv / attn_out / mlp_gate_up / mlp_down / norm / embedding / lm_head / other), peak-RSS measurement (via psutil), -o/--output flag, --no-detail to skip the breakdown.

  • --no-progress flag on compress, decompress, and benchmark. Forwarded to the existing progress= argument. Default unchanged.

Tests

  • 7 new tests in tests/test_opt_step7.py covering all four CLI features. 131 passed / 2 skipped (up from 124).

Compatibility

  • Zero format changes. All existing .bs files work with all new commands.
  • Existing CLI commands and verify (without --fast) unchanged.

Install: pip install bigsmall==3.10.0