fix: enable LLM retry without fallbacks and surface SDK retries - #7
Merged
ngoclam9415 merged 1 commit intoApr 19, 2026
Merged
Conversation
Bug: An Azure APIConnectionError exited the STAR loop after ~7.5min without any visible retry, despite the LLMCaller having retry logic. Three converging bugs were fixed: 1. LLMCaller retry was opt-in via fallback_providers, so the primary provider had no retry path. Now retry/backoff always runs; failover stays gated on fallback_providers. 2. _is_transient_error did not classify "Connection error." as transient (no matching keyword) and could not see openai.APIConnectionError buried in __cause__. Added "connection" to the keyword list and an exception-class-name walk over the __cause__ chain. 3. STAR loop exited on any exception. Added one bounded retry per iteration for transient LLM errors (sync + async paths), with full tracebacks via exc_info=True on all error sites. Observability: OpenAI/Azure/Moonshot providers now use a logging httpx.AsyncClient that emits "LLM HTTP request" per attempt, surfacing SDK-internal retries that were previously silent. New structured events: llm_retries_exhausted, STAR transient retry warnings, and an APIConnectionError-specific log line with traceback. Tests: updated test_no_fallbacks_exception_propagates (split into transient/permanent variants), added regression tests for the connection-error classification and __cause__ chain detection. Full unit + regression suites pass.
TheVinhLuong102
deleted the
fix/llm-retry-connection-error-and-star-resilience
branch
May 9, 2026 03:04
This was referenced Jul 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes a bug where an Azure
APIConnectionErrorexited the STAR loop after ~7.5 minutes with no visible retry attempts, despite theLLMCallerhaving retry logic. Three converging bugs were responsible:LLMCallerretry was opt-in viafallback_providers— primary provider had no retry path. Now retry/backoff always runs; failover stays gated onfallback_providers._is_transient_errordid not classify"Connection error."as transient (no matching keyword) and could not seeopenai.APIConnectionErrorburied in__cause__. Added"connection"to_TRANSIENT_KEYWORDSand a__cause__-chain walk that detects OpenAI SDK exception class names.exc_info=Trueon all error sites.Observability improvements
make_logging_http_clientwired into OpenAI/Azure/Moonshot providers — emitsLLM HTTP requestper attempt, surfacing SDK-internal retries that were previously silent (those took ~7.5 min in the original incident).llm_retries_exhausted, STAR transient retry warnings,OpenAI-compatible API connection errorwith traceback.error_type+exc_info=True.Files changed
dana/core/llm/llm_caller.py— always-retry path, expanded transient detection, publicis_transient_llm_errordana/common/llm/providers/openai_compatible_base.py— APIConnectionError handling, logging http client helperdana/common/llm/providers/{azure,openai,moonshot}.py— wire logging http clientdana/core/agent/base_star_agent.py— STAR transient retry (sync+async), tracebacksdana/core/agent/star_agent_streaming.py— traceback + transient flag in error logtests/unit/core/test_llm_caller_failover.py— regression tests for connection-error classification,__cause__walk, no-fallback retryTest plan
pytest tests/unit/core/test_llm_caller_failover.py— 24 passedpytest tests/unit— 1055 passed, 20 skippedpytest tests/regression— 44 passed, 18 skippedtests/live/llm/test_multi_modal.pyagainst real API keys — all providers (anthropic, openai, azure, gemini, gemini_openai, anthropic_like, moonshot) returned OK on every supported modality