Switch Alpaka

Switch-Alpaka v0.1 — Initial Alpha Release

Run local LLMs natively on your Nintendo Switch. Fully offline, CPU-only inference via a cross-compiled llama.cpp, console-based UI.

This is a very early, experimental release — expect rough edges.

Features
🧠 Load any GGUF model dropped into sdmc:/switch/llm/models/
💬 Multiple saved chats per model, persisted as plain text on the SD card
🔄 Multi-turn conversation context (not just single-shot Q&A)
📴 100% offline — no internet connection needed at runtime
Installation
Copy switch-llm-console.nro to sdmc:/switch/
Copy a .gguf model (see recommendations below) to sdmc:/switch/llm/models/
Launch via the Homebrew Menu

See the README for build-from-source instructions and details on the llama.cpp patch required for Horizon OS.

Recommended models to start with
Qwen2.5-0.5B-Instruct (Q4_K_M) — ~380 MB, fastest
Gemma-2-2B-it (Q4_K_M) — ~1.6 GB, better quality

Stick to small, Q4-quantized models — expect ~1–3 tokens/second on the Switch's Tegra X1, CPU-only.

Known limitations
No GPU acceleration (Horizon OS has no CUDA driver; see README for details)
No incremental KV-cache reuse — full conversation history is re-processed every turn, so long chats slow down and can eventually exceed the context window
Console-only UI — an SDL2 graphical UI was attempted but hit unresolved runtime crashes (contributions very welcome, see Known limitations in the README)
Tested primarily on Switch V1 (Erista); Mariko/V2 behavior not fully verified
What's next
Chat rename/delete
Context window trimming for long conversations
A working graphical UI (help wanted!)
Credits

Built on top of llama.cpp (MIT) and devkitPro/libnx for the Horizon OS homebrew toolchain.