Repository navigation
Releases
Releases · sharar-project/hibana
Hibana v0.1.0
The first release of Hibana, an inference engine for the NVIDIA GB10 (DGX Spark). It serves open models on your own Spark through OpenAI- and Anthropic-compatible APIs, so Claude Code, opencode, aider or any OpenAI client can use them.
Features
- Speculative decoding with each model's own drafter (MTP, DFlash2 or LiLiCorr)
- Prefix caching, with an optional SSD tier for long agent sessions
- Tensor parallel across two Sparks linked by their ConnectX ports
- Image input on Nemotron Super and Bolt
- Tool calls and constrained output on both APIs
Models
| Model | Command | Sparks | Input |
|---|---|---|---|
| Qwen3.8-27B | hibana serve 27b |
1 or 2 | text |
| Qwen3.8-Flash-Next (125B MoE) | hibana serve flashnext |
1 or 2 | text |
| Nemotron 3.5 Super (120B MoE, 12B active) | hibana serve nemotron |
1 | text, images |
| Nemotron 3.5 Bolt (74B MoE, 9B active) | hibana serve bolt |
1 | text, images |
Quick start
git clone https://github.com/sharar-project/hibana && cd hibana
uv sync
uv run hibana serve nemotron # http://127.0.0.1:30005
uv run hibana chat # in a second terminalThe first launch downloads the model and builds the GPU kernels; later launches start in under a minute.
The default fast profile is faster and slightly lossy; --exact gives the model's own outputs and --throughput is tuned for many concurrent requests (serving profiles).
Requirements
- An NVIDIA GB10 (DGX Spark) with DGX OS: driver 580 or newer and CUDA 13
- uv
- Disk for the model: 25 GB (Bolt) to 186 GB (Flash-Next)
Known limitations
- The Qwen models take text only.
- Nemotron's KV cache is FP8 only.
If you use Hibana in research, please cite the paper.
Apache 2.0 (third-party notices).