Skip to content

NppAI v0.2.0

Choose a tag to compare

View all tags
@deser1 deser1 released this 02 Oct 22:44
· 453 commits to main since this release
v0.2.0
9d9ca1a
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

NppAI v0.2.0

NppAI v0.2.0 focuses on reliability, testing, documentation, backend hardening, and optimized INT8 inference.

Added

  • Native CTest coverage for:
    • tensor math
    • model loading
    • tokenizer fallback
    • generation pipeline
  • Generation-pipeline fixture verifying BPE merging through public generation behavior.
  • Backend automated tests.
  • CI benchmark coverage.
  • Contributor documentation.
  • Architecture documentation.
  • Model-format documentation.
  • Testing documentation.
  • Security documentation.

Changed

  • Improved INT8 inference performance on x64 systems with AVX2/FMA.
  • Strengthened model-header validation.
  • Added minimum model-payload validation.
  • Improved backend request-size handling.
  • Moved expensive hashing work away from the backend event loop.
  • Added model-version-aware caching.
  • Improved CI and automated test coverage.

Testing

NppAI v0.2.0 introduces significantly broader automated testing for the native C++ inference engine.

CTest now validates core areas including tensor operations, model loading, tokenizer fallback behavior, and the complete generation pipeline.

The generation test also verifies BPE merging through the public generation API rather than testing only isolated internal functions.

Performance

INT8 inference received optimized x64 AVX2/FMA execution paths.

Benchmark coverage was added to CI to make performance behavior easier to track during future development.

Security & Reliability

  • Model headers receive additional validation before loading.
  • Minimum expected tensor payload size is validated.
  • Backend request sizes are constrained.
  • Expensive hashing operations are moved away from the event loop.
  • Model cache entries account for model versions.

Documentation

New project documentation covers:

  • architecture
  • contributing
  • model format
  • testing
  • security

These documents make the repository easier to understand, test, maintain, and contribute to.

Known Limitations

  • v0.2.0 requires AVX2/FMA for optimized x64 execution.
  • Runtime SIMD dispatch and scalar fallback are not yet available.
  • Model loading validates the minimum required payload but does not yet reject all trailing data.
  • The legacy model format does not contain SHA-256 payload integrity metadata.

These limitations are addressed or improved in NppAI v0.3.0.

Release

Version: v0.2.0

Release commit:

9d9ca1a758f87428bdd36826da497199d714ee21

Assets 2
Loading