-
Notifications
You must be signed in to change notification settings - Fork 0
bonsai ternary 27b dgx.stub
id: bonsai-ternary-27b-dgx name: Ternary Bonsai 27B on DGX Spark repo_path: ~/Documents/sw30labs/repos/bonsai-ternary-27b-dgx remote_url: https://github.com/sw30labs/bonsai-ternary-27b-dgx.git primary_language: Shell + Python framework: llama.cpp (PrismML CUDA fork) + Textual TUI category: local-inference-dgx stacks: [dgx, cli] ingested: 2026-07-22 last_commit_date: 2026-07-20
Local chat stack for the Prism ML Ternary Bonsai 27B model on NVIDIA DGX Spark (GB10 / CUDA). Builds the PrismML llama.cpp fork (custom ternary kernels), pulls the ~7.2 GB Ternary-Bonsai-27B-Q2_0.gguf into the shared HF cache, serves it with llama-server, and fronts it with a Textual chat TUI that renders the thinking stream.
Script-driven, no Python package: scripts/setup.sh (one-shot CUDA build + model download), download_model.sh, run_server.sh (OpenAI-compatible API + web UI on :8080), chat.sh (TUI, auto-starts llama-server if needed), stop_server.sh. Weights are never stored in the repo — they live in ~/models/huggingface, the same shared HF cache used by the other Desktop DGX projects. TUI supports /clear, /new, and a thinking-stream pane.
PrismML llama.cpp fork (ternary CUDA kernels), Textual (chat TUI in tui/), Hugging Face hub cache. No pyproject — shell scripts plus a small Python TUI.
No CI workflows. Manual scripts; TODO-iphone-android.md tracks a mobile-client idea.
- bonsai-image-ternary-4b-mlx-2bit — Apple Silicon sibling in the Prism ML ternary family (image generation vs chat)
- nemotron-parse-spark — same DGX Spark host conventions
- Tags: [cuda, llm, local-inference, cli]