Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions deployment/on-device/sdk/overview.mdx
Original file line number Diff line number Diff line change
@@ -1,39 +1,40 @@
---
title: "Overview"
sidebarTitle: "LEAP SDK"
---

The **Leap SDK** is Liquid AI's official on-device inference SDK and the **only SDK with first-class support for [Liquid Foundation Models](https://www.liquid.ai/blog/liquid-foundation-models-our-first-series-of-generative-ai-models) (LFMs)** — LFM2, LFM2.5 (text, thinking, JP, VL), and LFM2.5-Audio. "First-class" means every published Liquid checkpoint is supported, validated, and shipped through this SDK on day-one — the same team that trains the models ships the engine, sampler defaults, chat templates, and tool-call parsers that run them. There is no separate adapter layer, no community port, no upstream-rebase lag.

Check warning on line 6 in deployment/on-device/sdk/overview.mdx

View check run for this annotation

Mintlify / Mintlify Validation (liquidai-main) - vale-spellcheck

deployment/on-device/sdk/overview.mdx#L6

Did you really mean 'LFMs'?

Check warning on line 6 in deployment/on-device/sdk/overview.mdx

View check run for this annotation

Mintlify / Mintlify Validation (liquidai) - vale-spellcheck

deployment/on-device/sdk/overview.mdx#L6

Did you really mean 'LFMs'?

It's also a Kotlin Multiplatform library: the same `ModelRunner` / `Conversation` / `MessageResponse` API runs on iOS, macOS, Android, JVM desktop, Linux native, Windows native, and (preview) wasmJs. The Swift surface is generated through Kotlin/Native + SKIE and ships as XCFrameworks; the Android/JVM surface ships as Maven Central artifacts. Both call shapes are identical — only the language and packaging differ.

Check warning on line 8 in deployment/on-device/sdk/overview.mdx

View check run for this annotation

Mintlify / Mintlify Validation (liquidai-main) - vale-spellcheck

deployment/on-device/sdk/overview.mdx#L8

Did you really mean 'Multiplatform'?

Check warning on line 8 in deployment/on-device/sdk/overview.mdx

View check run for this annotation

Mintlify / Mintlify Validation (liquidai-main) - vale-spellcheck

deployment/on-device/sdk/overview.mdx#L8

Did you really mean 'wasmJs'?

Check warning on line 8 in deployment/on-device/sdk/overview.mdx

View check run for this annotation

Mintlify / Mintlify Validation (liquidai-main) - vale-spellcheck

deployment/on-device/sdk/overview.mdx#L8

Did you really mean 'XCFrameworks'?

Check warning on line 8 in deployment/on-device/sdk/overview.mdx

View check run for this annotation

Mintlify / Mintlify Validation (liquidai) - vale-spellcheck

deployment/on-device/sdk/overview.mdx#L8

Did you really mean 'Multiplatform'?

Check warning on line 8 in deployment/on-device/sdk/overview.mdx

View check run for this annotation

Mintlify / Mintlify Validation (liquidai) - vale-spellcheck

deployment/on-device/sdk/overview.mdx#L8

Did you really mean 'wasmJs'?

Check warning on line 8 in deployment/on-device/sdk/overview.mdx

View check run for this annotation

Mintlify / Mintlify Validation (liquidai) - vale-spellcheck

deployment/on-device/sdk/overview.mdx#L8

Did you really mean 'XCFrameworks'?

<Card title="Ready to install?" icon="rocket" href="/deployment/on-device/sdk/quick-start">
Jump to the **Quick Start** — install via SPM or Gradle, load a model, stream a response.

Check warning on line 11 in deployment/on-device/sdk/overview.mdx

View check run for this annotation

Mintlify / Mintlify Validation (liquidai-main) - vale-spellcheck

deployment/on-device/sdk/overview.mdx#L11

Did you really mean 'Gradle'?

Check warning on line 11 in deployment/on-device/sdk/overview.mdx

View check run for this annotation

Mintlify / Mintlify Validation (liquidai) - vale-spellcheck

deployment/on-device/sdk/overview.mdx#L11

Did you really mean 'Gradle'?
</Card>

## What "first-class support for Liquid models" gets you

- **Day-one model coverage.** New LFM checkpoints land in the SDK release that announces them — no waiting for a generic runtime to catch up to a new architecture, no manual quant conversion, no template-mismatch debugging. The [LEAP Model Library](https://leap.liquid.ai/models) is the canonical distribution path and the SDK pulls directly from it.
- **Per-checkpoint validated defaults.** The sampling parameters baked into each model's bundle manifest (`sampling_parameters` under `generation_time_parameters` in each `<Quant>.json` on [LiquidAI/LeapBundles](https://huggingface.co/LiquidAI/LeapBundles)) are the values the training team validated for that exact checkpoint. The SDK applies them automatically — no `temperature=0.7` placeholder retuning, no token-stream artifacts from the wrong `min_p` / `repetition_penalty`.
- **LFM-native special tokens and chat templates.** The shipped engine knows how to filter LFM control tokens before they reach your stream, applies the right chat template per checkpoint, and parses LFM's hermes and pythonic function-call dialects out of the box. Generic SDKs treat these as opaque text and surface raw tokens; Leap surfaces typed `MessageResponse.FunctionCalls` with parsed argument maps.

Check warning on line 18 in deployment/on-device/sdk/overview.mdx

View check run for this annotation

Mintlify / Mintlify Validation (liquidai-main) - vale-spellcheck

deployment/on-device/sdk/overview.mdx#L18

Did you really mean 'LFM's'?

Check warning on line 18 in deployment/on-device/sdk/overview.mdx

View check run for this annotation

Mintlify / Mintlify Validation (liquidai-main) - vale-spellcheck

deployment/on-device/sdk/overview.mdx#L18

Did you really mean 'hermes'?

Check warning on line 18 in deployment/on-device/sdk/overview.mdx

View check run for this annotation

Mintlify / Mintlify Validation (liquidai-main) - vale-spellcheck

deployment/on-device/sdk/overview.mdx#L18

Did you really mean 'pythonic'?

Check warning on line 18 in deployment/on-device/sdk/overview.mdx

View check run for this annotation

Mintlify / Mintlify Validation (liquidai-main) - vale-spellcheck

deployment/on-device/sdk/overview.mdx#L18

Did you really mean 'SDKs'?

Check warning on line 18 in deployment/on-device/sdk/overview.mdx

View check run for this annotation

Mintlify / Mintlify Validation (liquidai) - vale-spellcheck

deployment/on-device/sdk/overview.mdx#L18

Did you really mean 'LFM's'?

Check warning on line 18 in deployment/on-device/sdk/overview.mdx

View check run for this annotation

Mintlify / Mintlify Validation (liquidai) - vale-spellcheck

deployment/on-device/sdk/overview.mdx#L18

Did you really mean 'hermes'?

Check warning on line 18 in deployment/on-device/sdk/overview.mdx

View check run for this annotation

Mintlify / Mintlify Validation (liquidai) - vale-spellcheck

deployment/on-device/sdk/overview.mdx#L18

Did you really mean 'pythonic'?

Check warning on line 18 in deployment/on-device/sdk/overview.mdx

View check run for this annotation

Mintlify / Mintlify Validation (liquidai) - vale-spellcheck

deployment/on-device/sdk/overview.mdx#L18

Did you really mean 'SDKs'?
- **Multimodal LFMs in one API.** Vision (LFM2-VL family) and audio (LFM2.5-Audio) plug into the same `ChatMessage` / `ChatMessageContent` types you already use for text. Image inputs travel as JPEG bytes; audio travels as WAV blobs (or raw float32 PCM on Kotlin via `AudioPcmF32`). Output `MessageResponse.AudioSample` streams float32 PCM frames for audio-out checkpoints. No separate runtime per modality.

Check warning on line 19 in deployment/on-device/sdk/overview.mdx

View check run for this annotation

Mintlify / Mintlify Validation (liquidai-main) - vale-spellcheck

deployment/on-device/sdk/overview.mdx#L19

Did you really mean 'Multimodal'?

Check warning on line 19 in deployment/on-device/sdk/overview.mdx

View check run for this annotation

Mintlify / Mintlify Validation (liquidai-main) - vale-spellcheck

deployment/on-device/sdk/overview.mdx#L19

Did you really mean 'LFMs'?

Check warning on line 19 in deployment/on-device/sdk/overview.mdx

View check run for this annotation

Mintlify / Mintlify Validation (liquidai) - vale-spellcheck

deployment/on-device/sdk/overview.mdx#L19

Did you really mean 'Multimodal'?

Check warning on line 19 in deployment/on-device/sdk/overview.mdx

View check run for this annotation

Mintlify / Mintlify Validation (liquidai) - vale-spellcheck

deployment/on-device/sdk/overview.mdx#L19

Did you really mean 'LFMs'?
- **Constrained generation, end-to-end.** Kotlin annotations (`@Generatable` / `@Guide` on `@Serializable` data classes) and Swift macros (`@Generatable` / `@Guide` synthesizing `jsonSchema()` at compile time) produce JSON Schemas the engine enforces at decode time. The model's output is guaranteed to parse into your type.
- **One-call model fetching from the LEAP Model Library.** `LeapModelDownloader.loadModel(modelName:, quantizationType:)` resolves a manifest, downloads the right GGUF + matching `mmproj`/audio-decoder companion files for the checkpoint, caches them on disk, and hands back a `ModelRunner` — one call, no manual path wiring, no companion-file detection. Background-safe on iOS (`URLSessionConfiguration.background(withIdentifier:)`), WorkManager-backed on Android (survives app restarts).

## Other features

- **On-device by default.** No cloud round-trip, no per-token cost, full privacy, full offline operation.
- **KV cache reuse for fast multi-turn.** Bounded-LRU disk + memory `CacheOptions` skip the prefill step for shared prompt prefixes — TTFT on a long system prompt or RAG preamble drops from seconds to under a hundred milliseconds on cache hits. Disabled by default; opt in with `LiquidCacheOptions.enabled(path:)` / `ModelLoadingOptions.cacheOptions(path = ...)`.

Check warning on line 26 in deployment/on-device/sdk/overview.mdx

View check run for this annotation

Mintlify / Mintlify Validation (liquidai-main) - vale-spellcheck

deployment/on-device/sdk/overview.mdx#L26

Did you really mean 'prefill'?

Check warning on line 26 in deployment/on-device/sdk/overview.mdx

View check run for this annotation

Mintlify / Mintlify Validation (liquidai) - vale-spellcheck

deployment/on-device/sdk/overview.mdx#L26

Did you really mean 'prefill'?
- **Memory-mapped weight loading.** `use_mmap=true` is the default since v0.10.4. Model weights are file-backed, not anonymous RSS — iOS jetsam and Android LMK score the app much lower under memory pressure, cold load returns as soon as the file is mapped, and warm reloads stream from the kernel page cache.
- **Hybrid on-device + cloud routing.** `leap-openai-client` ships in the same release as an opt-in OpenAI-compatible chat-completions client (OpenAI, OpenRouter, vLLM, llama-server). One binary, two code paths — route small/fast prompts on-device, fall back to a cloud model for hard ones, share the same `ChatMessage` types.
- **Drop-in voice assistant UI.** `leap-ui` ships a Compose Multiplatform voice widget — animated orb, mic button, status label, state machine — that pairs with `VoiceConversation` to wire LFM2.5-Audio into a working voice experience without writing the recording-and-playback plumbing yourself.

Check warning on line 29 in deployment/on-device/sdk/overview.mdx

View check run for this annotation

Mintlify / Mintlify Validation (liquidai-main) - vale-spellcheck

deployment/on-device/sdk/overview.mdx#L29

Did you really mean 'Multiplatform'?

Check warning on line 29 in deployment/on-device/sdk/overview.mdx

View check run for this annotation

Mintlify / Mintlify Validation (liquidai) - vale-spellcheck

deployment/on-device/sdk/overview.mdx#L29

Did you really mean 'Multiplatform'?

## Where to go next

- [Quick Start](/deployment/on-device/sdk/quick-start) — install and run your first generation.
- [Model Loading](/deployment/on-device/sdk/model-loading) — manifest-based downloads, sideloaded GGUFs, `ModelLoadingOptions` reference, KV cache configuration.

Check warning on line 34 in deployment/on-device/sdk/overview.mdx

View check run for this annotation

Mintlify / Mintlify Validation (liquidai-main) - vale-spellcheck

deployment/on-device/sdk/overview.mdx#L34

Did you really mean 'sideloaded'?

Check warning on line 34 in deployment/on-device/sdk/overview.mdx

View check run for this annotation

Mintlify / Mintlify Validation (liquidai-main) - vale-spellcheck

deployment/on-device/sdk/overview.mdx#L34

Did you really mean 'GGUFs'?

Check warning on line 34 in deployment/on-device/sdk/overview.mdx

View check run for this annotation

Mintlify / Mintlify Validation (liquidai) - vale-spellcheck

deployment/on-device/sdk/overview.mdx#L34

Did you really mean 'sideloaded'?

Check warning on line 34 in deployment/on-device/sdk/overview.mdx

View check run for this annotation

Mintlify / Mintlify Validation (liquidai) - vale-spellcheck

deployment/on-device/sdk/overview.mdx#L34

Did you really mean 'GGUFs'?
- [Conversation & Generation](/deployment/on-device/sdk/conversation-generation) — the streaming generation API.
- [Constrained Generation](/deployment/on-device/sdk/constrained-generation) — `@Generatable` types and JSON-schema-enforced output.
- [Function Calling](/deployment/on-device/sdk/function-calling) — tool definitions, function-call parsing, hermes / pythonic dialects.

Check warning on line 37 in deployment/on-device/sdk/overview.mdx

View check run for this annotation

Mintlify / Mintlify Validation (liquidai-main) - vale-spellcheck

deployment/on-device/sdk/overview.mdx#L37

Did you really mean 'hermes'?

Check warning on line 37 in deployment/on-device/sdk/overview.mdx

View check run for this annotation

Mintlify / Mintlify Validation (liquidai-main) - vale-spellcheck

deployment/on-device/sdk/overview.mdx#L37

Did you really mean 'pythonic'?

Check warning on line 37 in deployment/on-device/sdk/overview.mdx

View check run for this annotation

Mintlify / Mintlify Validation (liquidai) - vale-spellcheck

deployment/on-device/sdk/overview.mdx#L37

Did you really mean 'hermes'?

Check warning on line 37 in deployment/on-device/sdk/overview.mdx

View check run for this annotation

Mintlify / Mintlify Validation (liquidai) - vale-spellcheck

deployment/on-device/sdk/overview.mdx#L37

Did you really mean 'pythonic'?
- [Voice Assistant Widget](/deployment/on-device/sdk/voice-assistant) — drop-in Compose Multiplatform voice UI.

Check warning on line 38 in deployment/on-device/sdk/overview.mdx

View check run for this annotation

Mintlify / Mintlify Validation (liquidai-main) - vale-spellcheck

deployment/on-device/sdk/overview.mdx#L38

Did you really mean 'Multiplatform'?

Check warning on line 38 in deployment/on-device/sdk/overview.mdx

View check run for this annotation

Mintlify / Mintlify Validation (liquidai) - vale-spellcheck

deployment/on-device/sdk/overview.mdx#L38

Did you really mean 'Multiplatform'?
- [OpenAI-Compatible Client](/deployment/on-device/sdk/openai-client) — hybrid on-device + cloud routing.
- [Migrating from 0.9.x?](/deployment/on-device/leap-sdk-changelog#0-9-x-0-10-x-kotlin-multiplatform-unification) — the unification story and drop-in replacements for legacy `Leap.load(...)` / `LiquidEngine(...)` call sites.
1 change: 1 addition & 0 deletions docs.json
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,7 @@
"group": "Edge Inference",
"icon": "mobile",
"pages": [
"deployment/on-device/sdk/overview",
"deployment/on-device/llama-cpp",
"deployment/on-device/lm-studio",
"deployment/on-device/mlx",
Expand Down