Skip to content

mini-infer v0.2.7: Embedded Agent Memory Engine & INT4 Inference

Latest

Choose a tag to compare

@Prajwal7-2002 Prajwal7-2002 released this 27 Oct 14:23
· 14 commits to main since this release
e7a9f98

What's Changed in mini-infer v0.2.7

🧠 Embedded Agent Cognitive Memory Engine

  • Added mini_infer.memory.MemoryEngine with zero external heavy database dependencies.
  • Powered by persistent SQLite WAL mode with an in-memory inverted index supporting BM25 scoring and subword token hashing.
  • Four memory categories: core, semantic, episodic, and working with time-based importance decay and top-$k$ similarity queries.
  • New CLI commands:
    • mini-infer memory add "text" --category core --importance 0.9
    • mini-infer memory query "search terms" --limit 5
    • mini-infer memory list
    • mini-infer memory stats

⚡ Sub-Byte Weight Quantization Integration

  • Integrated quantbench-engine to support sub-byte model weights:
    • INT4: 4-bit asymmetric block quantization (reducing 1.5B model weight footprint to ~900MB).
    • INT8: 8-bit symmetric per-channel quantization.
    • Ternary: 1.58-bit {-1, 0, +1} weight packing.
    • 2:4 Structured Sparsity: 50% parameter sparsity with contiguous bitmask.
  • Seamless CLI flags for serve and chat:
    • mini-infer serve --quant int4 --kv-cache-dtype auto
    • mini-infer chat --quant int4

🛡️ CLI Polish & Upfront Validation

  • Added immediate upfront validation for --quant and --kv-cache-dtype CLI parameters. Typos fail fast with informative options before model weight loading.

🔒 Enterprise Governance & CI Hardening

  • Interactive GitHub Issue Forms for bug reports and feature requests.
  • PR templates with mandatory architectural invariant checklists.
  • Added SECURITY.md, CODE_OF_CONDUCT.md, and .pre-commit-config.yaml.
  • Hardened cross-platform matrix CI (Windows + Linux, Python 3.10, 3.11, 3.12).

Full Changelog: v0.2.5...v0.2.7