Skip to content

Releases

Hibana v0.1.0

Choose a tag to compare

@shadowpa0327 shadowpa0327 released this 13 Nov 23:19
d474b74

The first release of Hibana, an inference engine for the NVIDIA GB10 (DGX Spark). It serves open models on your own Spark through OpenAI- and Anthropic-compatible APIs, so Claude Code, opencode, aider or any OpenAI client can use them.

Paper · Blog post

Features

  • Speculative decoding with each model's own drafter (MTP, DFlash2 or LiLiCorr)
  • Prefix caching, with an optional SSD tier for long agent sessions
  • Tensor parallel across two Sparks linked by their ConnectX ports
  • Image input on Nemotron Super and Bolt
  • Tool calls and constrained output on both APIs

Models

Model Command Sparks Input
Qwen3.8-27B hibana serve 27b 1 or 2 text
Qwen3.8-Flash-Next (125B MoE) hibana serve flashnext 1 or 2 text
Nemotron 3.5 Super (120B MoE, 12B active) hibana serve nemotron 1 text, images
Nemotron 3.5 Bolt (74B MoE, 9B active) hibana serve bolt 1 text, images

Quick start

git clone https://github.com/sharar-project/hibana && cd hibana
uv sync
uv run hibana serve nemotron     # http://127.0.0.1:30005
uv run hibana chat               # in a second terminal

The first launch downloads the model and builds the GPU kernels; later launches start in under a minute.

The default fast profile is faster and slightly lossy; --exact gives the model's own outputs and --throughput is tuned for many concurrent requests (serving profiles).

Requirements

  • An NVIDIA GB10 (DGX Spark) with DGX OS: driver 580 or newer and CUDA 13
  • uv
  • Disk for the model: 25 GB (Bolt) to 186 GB (Flash-Next)

Known limitations

  • The Qwen models take text only.
  • Nemotron's KV cache is FP8 only.

If you use Hibana in research, please cite the paper.

Apache 2.0 (third-party notices).