offgrid 0.2.1

Documentation release for the Deno package. The native library and the Rust crates are unchanged from 0.2.0; the crates are republished as 0.2.1 so all versions stay in sync.

Deno (jsr:@pinta365/offgrid)

• The JSR page now shows the full README: a feature overview, a deno add quick start, where to get models, how the native library is downloaded and cached, the permissions needed, and offline/bundled use.
• Every exported type and method is documented, so the JSR docs tab and editor hovers explain each option.
• Handle, the base class of Model, ChatSession and Embedder, is now exported.

Install

deno add jsr:@pinta365/offgrid       # Deno
cargo add offgrid                    # Rust
cargo install offgrid-server         # OpenAI-compatible server

Prebuilt Linux and Windows packages are attached below. See the 0.2.0 release for the registry setup and the 0.1.0 release for the full feature list.

Full changelog: v0.2.0...v0.2.1

offgrid 0.2.0

offgrid is now installable from the package registries: the Rust crates are on crates.io and the Deno binding is on JSR, which downloads the native library for you. The library itself is unchanged from 0.1.0.

Install

Rust

cargo add offgrid

This builds llama.cpp from source the first time (a few minutes). It needs CMake and a C/C++ compiler, and for the default vulkan feature the Vulkan headers and glslc. Use default-features = false for a CPU-only build. libclang is not needed: the llama.cpp bindings ship with the crate.

OpenAI-compatible server

cargo install offgrid-server

The server is also in the prebuilt packages below.

Deno

import { Offgrid } from "jsr:@pinta365/offgrid";

using core = await Offgrid.load();

On first use, Offgrid.load() downloads this release's prebuilt package for your platform from GitHub (release-assets.githubusercontent.com), checks it against the SHA-256 embedded in the JSR package, and caches it. Later runs work offline. OFFGRID_LIB or Offgrid.open() skip the download entirely, for offline or bundled apps. Models are never downloaded. The package README covers the lookup order, the cache location and the permissions.

What changed

• Crates on crates.io: offgrid-sys (bindings, builds llama.cpp), offgrid (the safe Rust API) and offgrid-server. The pinned llama.cpp sources ship inside offgrid-sys, limited to the parts the build uses (4.4 MB).
• No libclang needed: the bindings are committed for Linux and Windows instead of being generated at build time. The bindgen feature regenerates them, and is needed on other platforms.
• Deno binding on JSR: new Offgrid.load() with a verified, cached download of the native library, plus downloadLibrary() to pre-fill the cache and OFFGRID_CACHE to move it.
• Windows package as .tar.gz too, alongside the .zip. The Deno binding downloads the .tar.gz.
• For contributors: the llama.cpp submodule moved to crates/offgrid-sys/llama.cpp. Existing clones need git submodule update --init.

Downloads

Package	Platform
offgrid-0.2.0-linux-x86_64.tar.gz	Linux x86_64, glibc 2.28+
offgrid-0.2.0-windows-x86_64.zip / .tar.gz	Windows x86_64 (same contents)

Each package is a relocatable folder with liboffgrid / offgrid.dll, the llama.cpp libraries, the Vulkan and CPU backends, offgrid-server and include/offgrid.h. See the 0.1.0 release for the full feature list, tested hardware and models.

Full changelog: v0.1.0...v0.2.0

offgrid 0.1.0

First release of offgrid, a small native library that adds local, offline AI to an application: chat, tool calling, structured JSON output, embeddings, and image and audio input. Link the library, ship a GGUF model next to your app, and that's it. There's no daemon, no cloud, and no network access at runtime.

It is written in Rust on top of llama.cpp, runs on any GPU through Vulkan, and falls back to the CPU automatically.

What's in 0.1.0

• Chat: streaming, stop strings, sampling options and seeds. Requests use the OpenAI format; the session reuses its KV cache, so follow-up turns only process new tokens.
• Chat templates: each model's own Jinja template, rendered the Hugging Face way, with llama.cpp's built-in formats as a fallback.
• Structured output: response_format (json_object / json_schema) compiled into a grammar, so the output is valid by construction.
• Tool calling: OpenAI tools / tool_choice with each model's native call format (Hermes/Qwen, Llama 3.x, Mistral, Qwen3-Coder, generic JSON), plus a system-prompt fallback for templates without tool support (Gemma 3).
• Reasoning: <think> blocks are split into reasoning_content and streamed separately.
• Vision and audio: images (PNG, JPEG, …) and audio (WAV, MP3, FLAC) as OpenAI content parts, with encoded media reused across turns.
• Embeddings: batched, L2-normalized, float or base64.
• Cancellation from any thread, scoped to a single request.
• offgrid-server: OpenAI-compatible HTTP server (/v1/chat/completions with SSE, /v1/embeddings, /v1/models) with a session pool, an optional API key and graceful shutdown.

Use it from Rust (the offgrid crate), from C/C++ or anything with a C FFI (offgrid.h, JSON in and out), from Deno (bindings/deno), or over HTTP with any OpenAI client.

Downloads

Package	Platform	Contents
offgrid-0.1.0-linux-x86_64.tar.gz	Linux x86_64, glibc 2.28+ (Ubuntu 20.04+, Debian 10+, RHEL 8+)	lib/liboffgrid.so, llama.cpp/ggml libraries, Vulkan and CPU backends, bin/offgrid-server, include/offgrid.h
offgrid-0.1.0-windows-x86_64.zip	Windows x86_64 (tested on Windows 11)	bin/offgrid.dll, llama.cpp/ggml DLLs, Vulkan and CPU backends, bin/offgrid-server.exe, include/offgrid.h, MSVC and MinGW import libraries

Each package is a relocatable folder: keep it together and it runs from anywhere. It includes 14 CPU variants (the best one is picked at load time), and Vulkan is optional: without a GPU or Vulkan driver it runs on the CPU. The only runtime dependencies are the system C library and, for the GPU, the graphics driver's Vulkan loader.

Quick start

tar xzf offgrid-0.1.0-linux-x86_64.tar.gz
offgrid-0.1.0-linux-x86_64/bin/offgrid-server --model Qwen3-4B-Instruct-2507-Q4_K_M.gguf

Then point any OpenAI client at http://127.0.0.1:8080/v1. On Windows, run bin\offgrid-server.exe the same way. Rust projects can depend on the tag directly:

offgrid = { git = "https://github.com/Pinta365/offgrid", tag = "v0.1.0" }

See the README for the C and Deno APIs and for building from source.

Tested

• Linux: RX 9070 XT (Vulkan) and CPU, full test suite.
• Windows 11: RTX 3060 Laptop GPU (Vulkan) and CPU, both the release package and a native MSVC build with the full test suite. Laptops with an AMD iGPU plus a second GPU are handled (AMD's switchable-graphics Vulkan layer is disabled for the process).
• Models: Qwen3-4B-Instruct-2507, Llama 3.2 3B, Gemma 3 4B, Qwen3-0.6B (reasoning), Qwen3-Embedding-0.6B, Qwen2.5-VL-3B, Qwen2.5-Omni-3B and Ultravox 0.5. Qwen3-4B Q4_K_M generates about 150–175 tokens/s on the RX 9070 XT and about 43 tokens/s on the RTX 3060 Laptop.

Known limitations

• Linux and Windows on x86_64 only; macOS and ARM are not built yet.
• The server serves one chat model and one embedding model, with no per-request model switching and no legacy /v1/completions.
• Tool calls stream as one complete fragment each, not argument by argument.
• Embeddings are text-only, and rerankers aren't supported.

The full list is in the README.