fix(llm): silence litellm 'Provider List' stderr banner#142
Conversation
There was a problem hiding this comment.
Pull request overview
This PR reduces operator-facing stderr noise by silencing LiteLLM’s raw print() “Provider List” banner when provider resolution fails, ensuring it doesn’t leak past ZettelForge’s structlog setup during recall().
Changes:
- Added a
_get_litellm()import helper that setslitellm.suppress_debug_info = Truebefore calls tolitellm.completion(). - Updated
LiteLLMProvider.generate()to use_get_litellm()instead of importing LiteLLM inline. - Added a regression test asserting the suppress flag is flipped on
generate().
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
src/zettelforge/llm_providers/litellm_provider.py |
Routes LiteLLM import through _get_litellm() and sets suppress_debug_info to silence the stderr banner. |
tests/test_llm_providers.py |
Adds regression coverage to ensure generate() enables litellm.suppress_debug_info. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
LiteLLM's get_llm_provider helper writes a coloured "Provider List: ..." banner to stderr via raw print() whenever it cannot resolve a provider from a model name (litellm/litellm_core_utils/get_llm_provider_logic.py line 466). The banner bypasses Python logging entirely, so it leaks past ZettelForge's structlog setup and pollutes stderr for every operator running recall() — the background LLM-NER enrichment fires litellm via entity_indexer.extract_llm even when the user is just doing pip-only remember()/recall() without a working LLM. The first user trying the README's 30-second hello world sees ~40 of these banners during a single recall. Fix: route the lazy litellm import through a small _get_litellm() helper that sets litellm.suppress_debug_info = True after import. suppress_debug_info is the documented escape hatch for exactly this banner — it gates only the print, not exception raising, so real errors continue to propagate via litellm's exception types and our own structured logger reports them through llm_call_exception as before. Idempotent (safe to set on every generate() call). Module-level side effect on litellm is bounded to suppressing one print path that no caller relies on for control flow. Tests: 12 LiteLLMProvider tests pass (was 11 + new test_generate_silences_litellm_debug_banner). The test mocks the litellm module, asserts the suppress flag flips True after a generate() call, and would catch any future regression that drops the helper or its side effect. Out of scope: the env-dependent test_import_error_raised_when_sdk_missing already failed in litellm-installed environments before this change. CI runs without litellm, so it passes there. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
f056e8d to
862f4f8
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 862f4f8846
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| prev_suppress = getattr(litellm, "suppress_debug_info", False) | ||
| litellm.suppress_debug_info = True |
There was a problem hiding this comment.
Guard suppress_debug_info mutation with a lock
litellm.suppress_debug_info is a process-global module flag, but this code snapshots and restores it without synchronization; with concurrent generate() calls, one thread can restore an outdated value while another call is still in litellm.completion(). In this repo that is realistic because providers are shared singletons (src/zettelforge/llm_providers/registry.py notes singleton instances) and LLM work runs on a background enrichment thread (src/zettelforge/memory_manager.py), so this race can both re-enable the stderr banner mid-call and leave the global flag in the wrong final state after both calls complete.
Useful? React with 👍 / 👎.
Summary
LiteLLM's
get_llm_providerhelper writes a colouredProvider List: https://docs.litellm.ai/docs/providersbanner to stderr via rawprint()whenever it cannot resolve a provider from a model name (litellm/litellm_core_utils/get_llm_provider_logic.py:466). The banner bypasses Python logging entirely, so it leaks past ZettelForge's structlog setup.This affects every operator running
recall()— the background LLM-NER enrichment fires litellm viaentity_indexer.extract_llmeven when the user is only doing pip-onlyremember()/recall()without a working LLM. The first user trying the README's 30-second hello world (R1, just merged) sees ~40 of these banners during a single recall — exactly the wrong first impression now that the no-Ollama path is being marketed.Reproduction (before this PR)
Fix
Route the lazy
litellmimport through a new_get_litellm()helper that setslitellm.suppress_debug_info = Trueafter import.suppress_debug_infois the documented escape hatch for exactly this banner — it gates only the print statement, not exception raising, so:litellm.exceptions.*.llm_call_exceptionstructured-log event still fires with the same fields.After this PR:
Verified end-to-end with a fd-2-capturing test harness — zero
Provider Listbytes leak through.Tests
LiteLLMProvidertests pass (was 11 + newtest_generate_silences_litellm_debug_banner).litellmmodule, asserts the suppress flag flipsTrueafter agenerate()call, and would catch any future regression that drops the helper or its side effect.Out of scope
test_import_error_raised_when_sdk_missingalready failed in litellm-installed environments before this change. CI runs without litellm, so it passes there. Out of scope for this PR.LiteLLM Logging Details, etc.) goes through Python logging and is already managed by structlog level config — not in scope.🤖 Generated with Claude Code