Skip to content

Release v0.1.0

Choose a tag to compare

View all tags
@github-actions github-actions released this 13 Jun 03:28
v0.1.0
390496b
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Summary

LLiMa v0.1.0 is the first formal release of SiMa.ai's LLiMa runtime and Model Compiler tooling for generative AI workloads on Modalix. It packages the runtime, command-line tools, Python bindings, C++ development files, model compilation helpers, and documentation needed to compile, deploy, run, and evaluate LLM, VLM, and ASR models.

This release is intended for developers using the SiMa.ai GenAI stack on Modalix. It focuses on packaging the initial supported runtime, aligning model compilation and deployment workflows, and documenting the user-facing llima CLI and Model Compiler flows.

LLiMa is packaged and released together with the Neat Library. Install the matching Neat Library package set before using LLiMa; the Neat Library provides the higher-level APIs that most applications should use, while LLiMa supplies the GenAI runtime, model-management, and Model Compiler support used by those APIs.

Release Artifacts

Platform compatibility: this release is compatible with the 2.0.0 platform version.

The Modalix runtime is delivered as Debian packages:

  • sima-lmm-core: C++ runtime library, including libsima_lmm_runtime.so
  • sima-lmm-dev: public C++ headers and SimaLMM CMake package metadata
  • sima-lmm-cli: Python runtime package, nanobind extension, and the llima CLI entry point

The Model Compiler tooling is delivered through the sima-lmm Python package and optional dependency sets:

  • sima-lmm[sdk]: model compilation dependencies
  • sima-lmm[sdk_ext]: optional MoLE, benchmark, and evaluation dependencies

What's New

LLiMa Runtime And CLI

This release introduces the LLiMa runtime package set and the llima command-line workflow for Modalix.

  • Run compiled GenAI model directories with llima run.
  • Use CLI mode for interactive model validation.
  • Use web mode for OpenAI-compatible chat-completion style workflows.
  • Search, download, list, and remove precompiled models with the model manager commands.
  • Configure model storage with LLIMA_MODELS_PATH or use the default Modalix model directory.
  • Run LLM, VLM, and ASR model paths from one runtime package set.

Model Compilation And Deployment

The release adds Model Compiler-facing tools and documentation for preparing models for Modalix.

Supported input formats include:

  • Hugging Face safetensors model directories
  • GGUF LLM files
  • pre-quantized GPTQ/AWQ-style safetensor models where supported

The compilation and deployment flow now covers:

  • LLM and VLM compilation
  • GGUF conversion
  • Hugging Face transformer integration
  • model deployment to Modalix devices
  • LoRA deployment helpers
  • host-side benchmark and evaluation helpers
  • MoLE benchmarking support

Model And Architecture Support

This release brings in model/runtime support for current GenAI workloads, including language, vision-language, and speech paths.

Notable areas include:

  • Llama-family LLM support
  • Gemma-family LLM and VLM support
  • Qwen LLM and VLM support
  • Mistral-style tool-call output handling
  • Whisper ASR runtime components
  • LFM-family model support
  • PaliGemma and LLaVA-style VLM paths
  • per-layer language model handling
  • mixed-precision VLM compilation guidance

See docs/index.md in the repository for the full supported-model table and limitations.

Tool Calling And Chat API Behavior

LLiMa now supports native tool-call round trips for OpenAI/Ollama-compatible chat workflows.

This includes:

  • preserving incoming tool-call conversation history
  • forwarding native tools payloads through chat-template handling
  • parsing model-emitted tool calls into structured API responses
  • supporting multiple tool-call syntaxes across common model families
  • returning structured tool_calls in non-streaming and streaming responses

Runtime Fixes And Compatibility

Several runtime and packaging fixes are included in this release:

  • Corrected dispatcher API usage for the current runtime dependency stack.
  • Fixed runtime and packaging paths for the Modalix package layout.
  • Fixed Gemma-family sliding-attention buffer sizing during runtime execution.
  • Updated package naming and dependency integration for the current Neat Framework runtime packages.
  • Improved SDK/sysroot handling so repeated builds do not reinstall overlays when the required payloads are already present.
  • Added x86 SDK cross-build fixes for Ubuntu 24.04 based SDK environments.

Documentation

The LLiMa documentation has been restructured and converted to Markdown for external consumption.

Included documentation covers:

  • LLiMa overview and supported models
  • system requirements and setup
  • GenAI model compilation
  • model deployment
  • the llima CLI runtime workflow
  • MoLE benchmarking and evaluation

The documentation also updates user-facing terminology to align with Model Compiler and Neat Framework naming.

Build, Packaging, And Release Flow

This release aligns LLiMa with the current package and artifact publishing flow.

  • Adds manifest-driven package versioning for Debian, CMake, and Python package metadata.
  • Generates install metadata for package-based installation.
  • Builds and publishes sima-lmm-core, sima-lmm-dev, and sima-lmm-cli artifacts.
  • Adds branch and tag build support for release artifacts.
  • Removes the older tarball-only publishing path.
  • Adds a manual release workflow for release-managed builds.
  • Ensures tagged LLiMa releases use matching tagged runtime dependency artifacts rather than falling back to a development branch artifact.

Compatibility Notes

  • This release is compatible with platform version 2.0.0.
  • LLiMa is packaged with the Neat Library and should be installed alongside the matching Neat Library package set.
  • Applications should prefer the higher-level Neat Library APIs where available; LLiMa provides the underlying GenAI runtime and tooling.
  • This release targets Modalix and the SiMa.ai Neat Framework runtime stack.
  • Runtime models must be compiled with the matching Model Compiler flow before they are run with llima.
  • GGUF support is for LLMs; VLMs should use the Hugging Face safetensors compilation path.
  • Model support is limited to the architectures and constraints documented in docs/index.md.
  • Tag builds are expected to use matching tagged runtime dependency artifacts.

Validation Notes

Validation called out in the merged changes includes:

  • LLiMa package build validation
  • Modalix runtime validation for several VLMs
  • tool-call round-trip checks through the OpenAI-compatible chat path
  • SDK/sysroot build-path checks
  • package metadata generation checks
  • documentation review and terminology cleanup
  • workflow validation for release and artifact publishing

Before using this release in a production workflow, validate the target model path on the intended Modalix software image and confirm that the matching Neat Framework runtime packages are installed.

Notable Changes

  • Added the initial LLiMa package and runtime integration: #5, #11
  • Added model manifest resolution and model support updates: #6, #7
  • Added full tool-call round-trip support: #8
  • Updated package naming and runtime dependency integration: #12, #15
  • Changed license metadata to Apache 2.0: #13
  • Added CI/release artifact publishing flow: #14, #16, #21, #28, #31, #32
  • Aligned package versioning: #18
  • Corrected dispatcher API usage: #17
  • Restructured and cleaned up documentation: #23, #26
  • Ensured tag builds use matching dependency artifacts: #27
  • Fixed README workflow badge: #29

Full changelog: https://github.com/sima-neat/llima/commits/v0.1.0

Assets 2
Loading