fix(llm): drop local/* in-process embedding models from generation fallbacks

[guangyu-reflexio]

Summary

Filters local/* in-process embedding models out of text-generation fallback lists so they can never be handed to litellm's completion fallback ladder.

Problem

local/* models (e.g. local/nomic-embed-text-v1.5) are in-process ONNX embedding models — _litellm_embedding.py routes them to NomicEmbedder/LocalEmbedder and they have no litellm completion route. If such a model ends up in a text-generation fallback_models list, _build_completion_params passes it straight into litellm's fallbacks, and when the primary model fails, litellm tries the embedding model via async_completion_with_fallbacks → get_llm_provider → BadRequestError: LLM Provider NOT provided. You passed model=local/nomic-embed-text-v1.5.

This surfaced in production as Sentry PYTHON-FASTAPI-CV (~1,290 events). The existing filter at _build_completion_params only deduped the primary model (m != actual_model); there was no guard against non-generation model names.

Fix

_build_completion_params now drops local/* entries alongside the existing primary-model dedup:

fallback_models = [
    m for m in fallback_models_raw
    if m != actual_model and not m.startswith("local/")
]

local/ is the sole in-process embedding prefix (embedding_service.py), and no generation model uses it, so this is safe and targeted — a stray embedding model in the fallback list is now skipped instead of crashing the call.

Test

Added test_local_embedding_model_is_dropped_from_fallbacks: a local/nomic-embed-text-v1.5 + gpt-5-mini fallback list resolves to ["gpt-5-mini"]. All existing fallback tests still pass (10 passed).

Summary by CodeRabbit

• Bug Fixes

  • Improved text-generation fallback handling by excluding local/* embedding models from the fallback ladder, avoiding provider routing errors when mixed with valid generation models.

• Tests

  • Added unit coverage to confirm local/* models are filtered out while valid fallbacks are preserved.
  • Added a regression/contract test to ensure the local provider is embedding-only and excluded from generation-capable provider defaults.

[coderabbitai]

Important

Review skipped

No new commits to review since the last review.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 8cb55b53-5f6d-4abf-a8d2-efa4f896ddcd

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

📝 Walkthrough

Walkthrough

The completion fallback builder now excludes local/* embedding models while retaining valid non-local fallbacks. Tests also verify that the local provider remains embedding-only.

Changes

Completion fallback filtering

Layer / File(s)	Summary
Provider capability contract tests/server/llm/test_model_defaults.py	Tests assert that the local provider has no generation default and is excluded from generation-capable providers.
Filter and validate fallback models reflexio/server/llm/_litellm_text_generation.py, tests/server/llm/test_litellm_client_unit.py	_build_completion_params removes local/* entries from fallback models, and a unit test confirms valid fallbacks are preserved.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

• ReflexioAI/reflexio#28: Establishes the local provider’s embedding-only capability contract.
• ReflexioAI/reflexio#124: Changes LiteLLM completion fallback construction in the same path.
• ReflexioAI/reflexio#176: Updates the LiteLLM fallback-ladder construction and forwarding path.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main change: excluding local/* in-process embedding models from generation fallbacks.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/local-embedding-fallback-filter

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[guangyu-reflexio]

@coderabbitai review

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.