Skip to content

v3.3.2 — Graceful handling of oversized embedding chunks

Choose a tag to compare

@brian-lai brian-lai released this 28 Feb 02:29
· 49 commits to main since this release
2bb3166

Bug Fix

Embedding batch failures no longer cause total data loss.

Problem

When codetect index --force encountered a single chunk exceeding the Ollama model's context window (e.g. nomic-embed-text with 8192 tokens), the error cascaded through three layers:

  1. OllamaClient.EmbedBatchWithContext returned nil, error — discarding already-computed embeddings
  2. Pipeline.embedNewChunks propagated the error — discarding the entire batch
  3. Indexer.processBatch returned error — skipping the whole file batch and reporting chunks_created=0, chunks_embedded=0

Fix

Graceful degradation — skip individual failing chunks instead of failing entire batches:

  • OllamaClient.EmbedBatchWithContext: logs a warning and skips texts that fail, returning nil slots. Only errors if ALL texts fail or context is cancelled
  • LiteLLMClient.Embed: falls back to individual retries when the batch API call fails, skipping individual failures
  • Pipeline.embedNewChunks: filters out nil entries from partial results, storing only successful embeddings
  • Pipeline.EmbedChunks: tracks skipped chunks in EmbedResult.Errors

Tests

  • TestEmbedChunksPartialFailure — 1/5 chunks fails → 4 embedded, 1 error
  • TestEmbedChunksAllFail — all chunks fail → error returned
  • TestEmbedChunksMixedBatches — failures spread across batches → successful chunks preserved