Improve embedding backends with error handling, batching, and consistent naming

[calebevans]

Summary

• Rename *Vectorizer classes to *Embedder for consistency with the Embedder protocol (TransformerEmbedder, LlamaCppEmbedder, RemoteEmbedder)
• Rename create_vectorizer to create_embedder
• Create shared normalize_embeddings() utility in embedding/normalize.py for vectorized L2 normalization
• Add batching with tqdm progress to LlamaCppEmbedder (previously processed one window at a time)
• Add error handling for llama.cpp inference failures with context about which window failed
• Replace print() with logging.info() in LlamaCppEmbedder model download
• Add model loading error handling in TransformerEmbedder with user-friendly messages
• Narrow except Exception: pass in truncation warning to specific exception types with debug logging
• Move yield outside try/except in RemoteEmbedder to prevent consumer exceptions from being swallowed
• Add malformed API response detection in RemoteEmbedder
• Add explicit backend validation in create_embedder factory (raises ValueError for unknown backends)
• Remove per-batch torch.cuda.empty_cache() in TransformerEmbedder
• Fix CI GHCR auth: use repository_owner instead of actor for consistent container registry authentication

Summary by CodeRabbit

• New Features

  • Improved embedding normalization and quality through L2 normalization across all backends.
  • Enhanced error handling with clearer error messages for failed API calls and configuration issues.
  • Expanded logging functionality for better visibility into embedding operations.

• Tests

  • Expanded test coverage for improved error handling and edge cases.

[coderabbitai]

Caution

Review failed

Pull request was closed or merged during review

📝 Walkthrough

Walkthrough

This PR systematically refactors the embedding generation API from a vectorizer-based interface to an embedder-based interface. The factory function is renamed, all three embedding backends (remote, llama-cpp, transformer) are refactored with updated class names and improved batch processing logic, a new shared L2 normalization utility is introduced, and all application code, benchmarks, tests, and CI workflows are updated to use the new API surface.

Changes

Embedder API Refactoring

Layer / File(s)	Summary
Embedding factory and module exports src/cordon/embedding/__init__.py	Factory function renamed from create_vectorizer to create_embedder; backend selection logic updated to recognize "remote", "llama-cpp", and "sentence-transformers" with explicit ValueError for unknown backends; __all__ exports updated.
L2 normalization utility src/cordon/embedding/normalize.py, tests/test_normalize.py	New normalize_embeddings function provides L2 normalization for single vectors and 2D batches with epsilon clamping; comprehensive test suite covers normalization, zero vectors, and already-normalized inputs.
LlamaCpp embedder src/cordon/embedding/llama_cpp.py, tests/test_llama_cpp.py	Class renamed to LlamaCppEmbedder; embed_windows converted to batch processing with normalize_embeddings integration; download logging uses logger.info instead of print; test fixtures and coverage updated with logging validation and normalization checks.
Remote embedder src/cordon/embedding/remote.py, tests/test_remote.py	Class renamed to RemoteEmbedder; batch result collection with normalize_embeddings; response validation raises RuntimeError on malformed payloads; timeout errors include timeout duration; test suite reorganized with configuration and embedding test classes; new test for malformed API responses.
Transformer embedder src/cordon/embedding/transformer.py, tests/test_transformer.py	Class renamed to TransformerEmbedder; model loading wrapped in try/except raising RuntimeError; embed_windows yields via zip for batch emission; truncation-warning exception handling narrowed with debug logging; test classes updated to reflect embedder API.
Pipeline and benchmark integration src/cordon/pipeline.py, benchmark/evaluate.py, benchmark/visualize.py	SemanticLogAnalyzer instantiates create_embedder and uses self._embedder; benchmark scripts updated to use new embedder API for intermediate outputs and visualization pipelines.
CI and workflow updates .github/workflows/ci.yml	Docker login steps in container-build and container-manifest jobs now use github.repository_owner instead of github.actor.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

• calebevans/cordon#15: The PR refactors the existing remote backend from RemoteVectorizer/create_vectorizer to RemoteEmbedder/create_embedder, building on the LiteLLM-based remote API functionality.

• calebevans/cordon#34: Both PRs modify SemanticLogAnalyzer in src/cordon/pipeline.py to alter how embeddings are constructed during analysis, affecting the embedding handoff into scoring.

Suggested reviewers

• waynesun09

Poem

🐰 The vectorizer's tale now ends,
Embedder API transcends,
Three backends unified with care,
L2 norms float through the air,
From llama-cpp to remote and far,
Refactored code shines like a star! ✨

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main changes: renaming vectorizer classes to embedder, adding error handling, implementing batching, and ensuring consistent naming across the embedding backends.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch refactor/embedding-backends

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[sonarqubecloud]

Quality Gate failed

Failed conditions
5.9% Duplication on New Code (required ≤ 3%)

See analysis details on SonarQube Cloud