Skip to content

offgrid 0.2.0

Choose a tag to compare

View all tags
@github-actions github-actions released this 11 Oct 11:49
· 18 commits to main since this release
v0.2.0
9c08954

offgrid is now installable from the package registries: the Rust crates are on crates.io and the Deno binding is on JSR, which downloads the native library for you. The library itself is unchanged from 0.1.0.

Install

Rust

cargo add offgrid

This builds llama.cpp from source the first time (a few minutes). It needs CMake and a C/C++ compiler, and for the default vulkan feature the Vulkan headers and glslc. Use default-features = false for a CPU-only build. libclang is not needed: the llama.cpp bindings ship with the crate.

OpenAI-compatible server

cargo install offgrid-server

The server is also in the prebuilt packages below.

Deno

import { Offgrid } from "jsr:@pinta365/offgrid";

using core = await Offgrid.load();

On first use, Offgrid.load() downloads this release's prebuilt package for your platform from GitHub (release-assets.githubusercontent.com), checks it against the SHA-256 embedded in the JSR package, and caches it. Later runs work offline. OFFGRID_LIB or Offgrid.open() skip the download entirely, for offline or bundled apps. Models are never downloaded. The package README covers the lookup order, the cache location and the permissions.

What changed

  • Crates on crates.io: offgrid-sys (bindings, builds llama.cpp), offgrid (the safe Rust API) and offgrid-server. The pinned llama.cpp sources ship inside offgrid-sys, limited to the parts the build uses (4.4 MB).
  • No libclang needed: the bindings are committed for Linux and Windows instead of being generated at build time. The bindgen feature regenerates them, and is needed on other platforms.
  • Deno binding on JSR: new Offgrid.load() with a verified, cached download of the native library, plus downloadLibrary() to pre-fill the cache and OFFGRID_CACHE to move it.
  • Windows package as .tar.gz too, alongside the .zip. The Deno binding downloads the .tar.gz.
  • For contributors: the llama.cpp submodule moved to crates/offgrid-sys/llama.cpp. Existing clones need git submodule update --init.

Downloads

Package Platform
offgrid-0.2.0-linux-x86_64.tar.gz Linux x86_64, glibc 2.28+
offgrid-0.2.0-windows-x86_64.zip / .tar.gz Windows x86_64 (same contents)

Each package is a relocatable folder with liboffgrid / offgrid.dll, the llama.cpp libraries, the Vulkan and CPU backends, offgrid-server and include/offgrid.h. See the 0.1.0 release for the full feature list, tested hardware and models.

Full changelog: v0.1.0...v0.2.0

Assets 5
Loading