Repository navigation
v0.5.5
v0.5.5
Lightweight CPU Inference & Deployment Resilience
All working stable platform
![]()
⚡
- Qwen2.5:0.5b Integration: Replaced the default CPU profile model (
llama3.2:1b) withqwen2.5:0.5b, reducing local memory footprint and accelerating response times on non-GPU setups. - Synced Orchestration: Updated LiteLLM routing rules and department default models to seamlessly target the new
qwen2.5backend.
🛡️
- Non-blocking Model Pulls: Configured timeouts for model preload init-containers, ensuring cluster services (ArgoCD, LiteLLM, Traefik) start cleanly even if internet bandwidth is limited.
- Bootstrap Job Backoff: Increased retry threshold for initialization jobs.
📁
- Open WebUI Persistence: Included PVC template for persistent user data across restarts.
- Script Executables: Updated execution bits for all environment setup and teardown scripts.
- Architecture Media Overhaul: Refreshed system diagrams and updated main repository README with new project banner.