Skip to content

v0.1.2

Choose a tag to compare

View all tags
@TheSylvester TheSylvester released this 31 May 08:46
v0.1.2
1e4bbe5

Fixes — GPU embedding backfill

  • installer (gpu): stage libggml-cuda.so into bin/ beside the executable (not a separate bin-cuda/). llama.cpp's GGML_BACKEND_DL loader discovers backends by scanning the executable's own directory, not LD_LIBRARY_PATH — so the CUDA backend never loaded and embedding silently ran on CPU (while still printing offloaded N/N layers to GPU). Matches the Windows path.
  • installer (gpu probe): require a real CUDA device marker (ggml_cuda_init / loaded CUDA backend / CUDA0 buffer) instead of the offloaded N/N string, which prints even on CPU and caused the installer to falsely persist mode=gpu.
  • embed: per-item fallback so a single message exceeding the 8192-token physical batch can't 500 the whole batch and wedge the backfill.
  • codex: handle array-valued function_call_output (image/screenshot results) without crashing the session ingest (output.match is not a function).

Net effect on an RTX 2060: embedding goes from ~5 msg/s (silent CPU, stalling) to ~26 msg/s at 80–96% GPU utilization.

CUDA backend asset is byte-identical to v0.1.0/v0.1.1 (llama.cpp b5300).

Assets 3
Loading