v0.1.3
Fixes
- gpu: enable Flash Attention (
-fa) for the embed server, the one-shot path, and the install probe. The 8192-token attention compute buffer is otherwise O(n²) (~3.7 GB on GPU → ~5.7 GB total), which OOMs cards <6 GB and forces CPU fallback.-famakes it O(n) (~0.8 GB → ~2.9 GB total), so 4 GB GPUs now run on GPU at the full 8192 context. Embeddings are numerically identical (verified cosine 1.0 vs non-FA) and throughput improves (~26 → ~32 msg/s on an RTX 2060). The probe must mirror runtime, or it OOMs on a 4 GB card and falsely rejects a working GPU.
Builds on v0.1.2 (bin/ CUDA staging, real-device probe, embed resilience, codex array output). CUDA backend asset is byte-identical (llama.cpp b5300).