Fix speech-to-text fallbacks #11663
Closed
Fix speech-to-text fallbacks #11663
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…s and examples. Update Router class to handle fallbacks more effectively, ensuring proper function calls without infinite recursion. Add tests for transcription fallbacks, covering both same-provider and cross-provider scenarios.
…usage and fallback configurations. Replace "whisper" with "whisper-1" in code examples, enhance fallback implementation details, and streamline Router and proxy configurations for better clarity and usability.
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
colesmcintosh
changed the title
….py. Add async_transcription_with_fallbacks and transcription_with_fallbacks functions to handle model fallbacks during transcription, enhancing error handling and preventing infinite recursion. Update atranscription and transcription functions to utilize these new fallback mechanisms.
…nts for model and file parameters, improving code clarity and type safety. Include litellm_logging_obj in the function call to support logging during transcription attempts.
…ling logic. This simplifies the atranscription and transcription functions, delegating fallback management to dedicated utilities, enhancing code clarity and maintainability.
…to streamline the codebase. This change focuses on enhancing clarity and maintainability by delegating fallback handling to dedicated utilities, following recent refactoring efforts.
- Implemented `transcription` and `_transcription` methods to handle audio file input and model selection. - Added example usage in the docstring for clarity. - Enhanced error handling and logging for better debugging and tracking of calls.
- Introduced a new test `test_router_sync_transcription_fallbacks` to verify that the Router's synchronous transcription method correctly handles fallbacks between primary and backup models. - The test simulates an API error for the primary model and checks that the backup model is used successfully, ensuring robust error handling in the transcription process.
|
closing as the fallbacks work already |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Title
Fix speech-to-text fallbacks
Relevant issues
Fixes issue where speech-to-text (/audio/transcriptions) fallbacks were not working
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unitType
Bug Fix
Changes
Root Cause
Fixed critical infinite recursion bug in fallback system that was preventing speech-to-text fallbacks from working:
Infinite Recursion Bug: In
litellm/router_utils/fallback_event_handlers.pyline 139, the fallback handler was callingasync_function_with_fallbacksinstead oforiginal_function, creating infinite loops.Parameter Loss Bug: In
litellm/router.pyline 3598,kwargs.pop("original_function")was permanently removing the function reference needed for fallbacks.Parameter Conflict Bug:
original_functionwas being passed as both positional and keyword arguments causing "multiple values" errors.Fixes Applied
original_functioninstead ofasync_function_with_fallbacks.pop("original_function")to.get("original_function")to maintain parameteroriginal_functionfrom kwargs before calling to avoid conflictsTesting
tests/test_litellm/test_router.py:test_router_transcription_fallbacks()for same-provider fallbackstest_router_transcription_fallbacks_cross_provider()for cross-provider fallbacksDocumentation
docs/my-website/docs/audio_transcription.mdwith:litellm.transcription()mock_testing_fallbacks=trueImpact
The fix resolves the core issue preventing audio transcription fallbacks while maintaining all existing functionality.