0.1.0-alpha.13 — CUDA OOM falls back to CPU instead of crashing
Targeted hotfix on top of 0.1.0a12. Surfaced by a live install on a machine with another GPU-resident model.
Fixed
route()no longer crashes on CUDA OOM. sentence-transformers' default device pick is CUDA when a GPU is visible. If the GPU is already full (e.g. a local LLM in Ollama owns the VRAM),SentenceTransformerraisedtorch.AcceleratorError: CUDA error: out of memoryand the whole call failed before the first prompt could even be encoded._Runtime.__init__now catches GPU-init failures broadly (matchesout of memory,cuda,cudaerror,no cudain the exception message), prints a one-line stderr notice, and retries the model load withdevice='cpu'.
Added
MIND_NERVE_DEVICEenv var — set tocputo force CPU unconditionally even when a GPU is visible. Useful for hosts sharing the GPU with other tenants.
Behavior on this machine after the fix
$ mind-nerve route 'deploy the staging build to production' --top-k 5
mind-nerve: GPU init failed (AcceleratorError), falling back to CPU
{ "query": ..., "routes": [
{ "name": "deploy", "score": 0.7157 },
{ "name": "Deploy to Render", "score": 0.6703 },
{ "name": "build-and-deployment-rules", "score": 0.6569 },
...
] }