Skip to content

v0.1.16 — Fix Ollama embedding 500 errors

Choose a tag to compare

View all tags
@ssdavidai ssdavidai released this 23 Feb 12:34
v0.1.16
45d782d

Fix: Ollama embedding context overflow

Root cause: MAX_EMBEDDING_CHARS was set to 32,000 characters, but nomic-embed-text has an 8,192-token context window (~12,200 char hard limit). Any vault record exceeding this limit caused an HTTP 500 ("the input length exceeds the context length"), resulting in ~60% failure rate across the vault.

Changes

  • Reduced MAX_EMBEDDING_CHARS from 32,000 to 8,000 — leaves comfortable margin for non-English text and special tokens
  • Added HTTP error response body logging — embedder.embed_retry events now include detail field with the server's error message for faster diagnostics

Result

  • Surveyor Ollama embedding success rate: ~40% → 100%
  • All four workers (curator, janitor, distiller, surveyor) confirmed healthy and processing
Assets 4
Loading