Architecture & Shortcuts

Configuration & Hardware Guide

Configuration System

Defaults are loaded from classpath resources inside src/main/resources/systemprompts/. You can override configuration and prompts at runtime by placing custom files in ./systemprompts/ or an ./application.config file next to the binary.

Example application.config setups:

Managed llama-server (GGUF)

backend.type=managed-llama-server
backend.llama.command=llama-server
backend.llama.modelPath=./models/Gemma4-26B-A4B-QAT.gguf
backend.llama.port=0
backend.llama.startupTimeoutSeconds=120
backend.llama.arguments=--ctx-size 32768 --ctx-checkpoints 32 --parallel 2 --threads 6 --threads-batch 6 --batch-size 1024 --ubatch-size 512 --kv-offload --cache-type-k q8_0 --cache-type-v q8_0 --flash-attn on --reasoning off

Managed mlx_vlm.server (Apple Silicon MLX)

backend.type=managed-mlx-server
backend.mlx.command=python3
backend.mlx.modelPath=./model/gemma-4-26b-a4b-it-4bit
backend.mlx.port=0
backend.mlx.startupTimeoutSeconds=180
backend.mlx.arguments=-m mlx_vlm.server --max-kv-size 32768 --max-tokens 32768