Skip to content

Release v0.3.0

Latest
Latest

Choose a tag to compare

View all tags
@github-actions github-actions released this 17 Jul 18:11
v0.3.0
7d5e68b
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

LLiMa v0.3.0

LLiMa v0.3.0 updates the low-level GenAI runtime and model-compilation layer for Neat 0.3.0 and Platform 2.1.2.

This release adds speculative-decoding model generation, expands vision-model compilation, makes strided KV cache the standard path, improves Phi-4 generation quality, supports concurrent requests across served models, and makes installation and model-regression workflows more reliable.

Compatibility: Pair LLiMa v0.3.0 with the Neat 0.3.0 package set and Platform 2.1.2. Most application developers should use the higher-level Core and PyNeat APIs; use LLiMa directly for GenAI runtime, model compilation, or lower-level model integration.

Highlights

Speculative Decoding And EAGLE3 Support

  • Adds generation support for EAGLE3 draft models used by speculative decoding.
  • Adds the model, configuration, export, and runtime paths required to build draft-model artifacts alongside the main language model.
  • Expands regression coverage for draft-model generation and end-to-end compilation.

This gives model developers a supported path for preparing LLiMa models that use speculative decoding to accelerate token generation.

Expanded Vision-Model Compilation

  • Adds SiMa builder paths for Qwen and Gemma vision models.
  • Moves model generation to the current afe.ir.sima_builder.SimaBuilder interface.
  • Adds VLM reference configurations and compilation coverage for vision and multimodal model paths.

This expands the model-compilation foundation for Qwen- and Gemma-based VLM applications in the Neat 0.3.0 line.

Strided KV Cache By Default

  • Makes strided KV cache the default compile and runtime configuration.
  • Aligns generated model artifacts and runtime configuration on the same KV-cache layout.
  • Removes the need to enable the strided path separately for new builds.

Build environments must use the matching AFE release, and models should be regenerated with the Neat 0.3.0 toolchain.

Correct Phi-4 And LongRoPE Generation

  • Fixes degraded Phi-4 output that could collapse into repetitive sentences.
  • Corrects RoPE frequency generation for proportional and LongRoPE configurations.
  • Adds attention_factor handling to Python, C++, GGUF import, and generated cosine/sine tables.
  • Calculates the Phi LongRoPE fallback when the model configuration does not provide it.

The result is correct positional scaling and more stable generation for Phi-4 and other LongRoPE-based models.

Safer Concurrent GenAI Requests

  • Gives each request thread its own prepared MLA queue instead of sharing one process-wide queue.
  • Prevents simultaneous requests for different served models from overwriting shared queue state.
  • Preserves per-request batching before work is submitted to mlashmcomplex.

This improves reliability when applications serve multiple models or issue concurrent GenAI requests.

Cleaner SDK And CMake Integration

  • Removes build-machine and SDK-sysroot paths from the installed SimaLMM::sima_lmm_runtime CMake target.
  • Exports package-relative include directories so downstream projects can resolve their own DevKit or SDK dependencies.
  • Keeps the public API unchanged while making the installed development package more portable.

More Reliable Installation And Upgrades

  • Moves the LLiMa package line to version 0.3.0.
  • Accepts the Neat-compatible simaai-memory-lib package range for Platform 2.1.2 instead of requiring one exact Debian revision.
  • Allows Neat memory-library revisions such as 2.1.1+neat1 to satisfy LLiMa dependencies.
  • Keeps LLiMa aligned with the Internals and Core packages in the Neat 0.3.0 artifact set.

Reproducible Model-Compilation Inputs

  • Adds a Vulcan-backed cache for selected Hugging Face safetensors used in LLiMa compilation regression testing.
  • Records the resolved Hugging Face revision, file list, size, checksum, artifact location, and public artifact URL for every cached model.
  • Reduces repeated public Hub downloads and makes CI inputs stable and traceable.
  • Seeds coverage for Qwen3, LFM2, LFM2-VL, and Qwen3-VL model paths.

Clearer Interactive CLI Behavior

  • Removes the obsolete list command entry from interactive help and autocomplete.
  • Documents that interactive CLI chat history is enabled by default.
  • Fixes a small system-prompt issue in CLI mode.

Installation

Install LLiMa through the standard Neat package flow:

sima-cli neat install llima

For normal application development, installing the matching Core v0.3.0 bundle resolves the compatible LLiMa and Internals dependencies automatically.

Compatibility And Migration Notes

  • LLiMa package version: 0.3.0.
  • Compatible platform release: 2.1.2.
  • Use LLiMa from the same Neat 0.3.0 artifact set as Core and Internals.
  • Build and compilation environments require the AFE version that provides afe.ir.sima_builder.SimaBuilder.
  • Strided KV cache is the default and required compilation path for the new model-generation workflow.
  • Regenerate model artifacts with the matching Neat 0.3.0 compiler stack before deployment.
  • Public C++ and Python API signatures remain unchanged.

Validation

Release validation included:

  • LLiMa build, packaging, policy, and PR checks
  • Phi-4 and LongRoPE configuration and generation-path checks
  • Concurrent prepared-queue ownership checks
  • CMake export and downstream package-layout checks
  • Memory-library Debian dependency-range checks
  • Model and unit tests for the new model-generation paths
  • ONNX regression tests
  • VLM reference configurations
  • GGUF and end-to-end compilation tests
  • EAGLE3 draft-model generation coverage
  • Hugging Face cache dry-run permission checks and a real four-model publish with zero failures

Key Included Pull Requests

  • #39 — Portable CMake exports and concurrent request queueing
  • #36 — Reproducible Hugging Face safetensors cache
  • #56 — Phi-4 and LongRoPE generation correction
  • #59 — Neat-compatible memory-library dependency range
  • #61 — Interactive CLI help and history cleanup
  • #62 — LLiMa 0.3.0 package version
  • #63 — EAGLE3, vision builders, strided KV cache, and expanded tests
  • #70 — Final promotion to the 0.3 release branch

Full Changelog: v0.2.0...v0.3.0

Contributors

Thank you to everyone who contributed to this release:

Assets 2
Loading