Architecture & Shortcuts

Configuration & Hardware Guide

Configuration System

Defaults are loaded from classpath resources inside src/main/resources/systemprompts/. You can override configuration and prompts at runtime by placing custom files in ./systemprompts/ or an ./application.config file next to the binary.

Example application.config setups:

Managed llama-server (GGUF)

backend.type=managed-llama-server
backend.llama.command=llama-server
backend.llama.modelPath=./models/Gemma4-26B-A4B-QAT.gguf
backend.llama.port=0
backend.llama.startupTimeoutSeconds=120
backend.llama.arguments=--ctx-size 32768 --ctx-checkpoints 32 --parallel 2 --threads 6 --threads-batch 6 --batch-size 1024 --ubatch-size 512 --kv-offload --cache-type-k q8_0 --cache-type-v q8_0 --flash-attn on --reasoning off