refft-ai-hub-models

Per-model packages for running models on Qualcomm Hexagon NPUs through the refft-hexagon runtime.

Each hexagon package is self-contained: scripts/install.sh creates a uv env and pushes refft-hexagon; scripts/export.sh packs weights and programs into one .refft file; cli / serve / perf run refft-hexagon --model on the board. Official TTFT / TPOT / TPS come from [metrics.genie.*].

The reference implementation is qwen3_4b/hexagon. The short onboard contract is docs/ADDING_A_MODEL.md. Worked examples (quantize → debug → RFTB → official reports → Hugging Face):

• docs/ADAPTING_QWEN3_4B.md — dense GQA reference, including 1×HTP / 2×HTP
• docs/ADAPTING_LFM2_5_2_6B.md — hybrid short-conv port that copied this package

Layout

src/refft_ai_hub_models/models/<model_id>/
  README.md                 pointer at the hardware package
  spec.yaml                 model introduction only (architecture, license, tags)
  <hardware>/               one package per compute backend (hexagon|…)
    README.md               one-page flow
    info.yaml               hardware card: runtime, platforms, chipsets
    pyproject.toml          uv project
    <pkg>/                  VISA modeling via refft.hexagon.ops
    scripts/                install / serve / cli / perf
    reports/runs/*.json     official generate metrics (generated)
    quantization/           optional AIMET recipes

There is no <model>/scripts dispatcher. Pick a hardware directory and run its scripts in place. The reference walk-through is qwen3_4b/hexagon/README.md.

spec.yaml never describes a single chip or runtime. info.yaml always sits under a hardware directory.

Models

Model	Id	Status
Qwen3-0.6B	qwen3_0_6b	Measured on Dragonwing IQ-9075 (QCS9075) — 1×HTP pin-all
Qwen3-1.7B	qwen3_1_7b	Measured on Dragonwing IQ-9075 (QCS9075) — 1×HTP pin-all
Qwen3-4B-Instruct-2507	qwen3_4b	Measured on Dragonwing IQ-9075 (QCS9075) — reference
LFM2.5-2.6B	lfm2_5_2_6b	Measured on SM8850 and IQ-9075 — hybrid short-conv + GQA
LFM2.5-VL-3B	lfm2_5_vl_3b	Measured on SM8850 (text-first) — same hybrid LM as 2.6B
LFM2.5-8B-A1B	lfm2_5_8b_a1b	Measured on Dragonwing IQ-9075 (QCS9075) — hybrid MoE, 1×/2×HTP
SmolVLM2-500M-Video-Instruct	smolvlm2	Measured on SM8850 and IQ-9075 — vision+text, pin-all RFTB

Quick start

cd src/refft_ai_hub_models/models/qwen3_4b/hexagon
scripts/install.sh                          # uv venv + push runtime
scripts/export.sh --push                    # pack weights+programs → one .refft
scripts/cli.sh --prompt "Who are you?"      # refft-hexagon cli --model
scripts/perf.sh                             # C=64/128 prefill + Who are you? TTFT/TPS
scripts/serve.sh start                      # refft-hexagon serve --model :8080

Script	Role
install.sh	Local .venv + refft wheel + push refft-hexagon
export.sh	Single-file RFTB (--push copies it to the board)
cli.sh	One-shot refft-hexagon cli --model
serve.sh	start / stop / restart / status / logs
perf.sh	Official genie KPIs → reports/runs/ + perf.yaml
debug.sh	vm + host Python client (not official numbers)
prefill.sh	Per-op VISA debug against the vm

Override the board with SSH_HOST. Flags, env vars, and the debug path are in hexagon/README.md.

Rules every hexagon package follows

1. Do not export REFFT_* model knobs from shipped scripts. Unset defaults are the device-calibrated optimum.
2. Official numbers come from [metrics.genie.query.kpis] on refft-hexagon cli --model. Init is not TTFT.
3. Modeling emits only through refft.hexagon.ops.
4. The uv env lives under the hardware package (hexagon/.venv).
5. Hexagon DSP is exclusive. Ask before killing another process that holds it. cli / serve / perf stop our vm; debug.sh vm start stops our product serve.

Adding a model

See docs/ADDING_A_MODEL.md. Copy qwen3_4b/hexagon, not another tree.