Refactor/all in one openai #43
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Consolidate requirements files: combine server and all requirements - Update various package versions, including: - numpy (1.26.4 -> 2.0.0) - openai (1.30.5 -> 1.35.3) - google-generativeai (0.5.4 -> 0.7.0) - sqlalchemy (2.0.30 -> 2.0.31) - pytest (8.2.1 -> 8.2.2) - Remove unused dependencies (e.g., accelerate, flask, flask-restful) - Adjust package extras and their dependencies - Clean up redundant or obsolete entries in poetry.lock This update simplifies the project's dependency management and brings core libraries up to their latest versions for improved functionality and security.
- Remove `main.py` and split functionality into `app.py` and `build.py` - Simplify `ServerBaseSettings` in `config.py`: - Remove `APP_TYPE`, `DEFAULT_PORT`, and optional `PORT` - Set default `PORT` to 8680 - Delete `LlmSettings` and `AgentSettings` classes - Update `create_app` and `run_app` functions in `build.py` - Add health check endpoint - Implement `app_lifespan` context manager - Remove model registration and discovery logic
- Rename main CLI group from `app` to `languru_cli` - Remove separate `agent` and `llm` groups, replace with single `server` group - Simplify `server_run` command, removing complex port and action handling - Remove `AppType` enum and related logic - Update imports and function calls to reflect new structure - Streamline server settings by using `ServerBaseSettings`
- Introduce OrganizationType enum with OPENAI, AZURE, ANTHROPIC, GOOGLE, GROQ, PERPLEXITY, and VOYAGE - Implement to_org_type function for converting string to OrganizationType - Create organization_type_aliases mapping for easy reference - Add OrganizationNotFound exception - Remove unused Enum import from server/config.py
- Change APP_NAME from "languru" to "languru-server" in config.py - Introduce new file deps/openai_clients.py with OpenaiClients class: - Implement client initialization for various AI providers - Add depends_openai_client method for dynamic client selection - Create _org_to_openai_client helper method for organization-based client mapping
- Introduce new CredentialsNotProvided exception - Replace ValueError with CredentialsNotProvided in API clients: - Anthropic - Google - Groq - Perplexity - Voyage - Refactor OpenaiClients initialization: - Implement lazy loading for API clients - Add error handling for client initialization - Improve logging for uninitialized clients - Update client retrieval logic in depends_openai_client - Enhance error handling in _org_to_openai_client method
…ructor - Import OpenaiClients in languru/server/build.py - Initialize OpenaiClients in create_app function - Update OpenaiClients constructor to accept *args and **kwargs
- Remove separate handling for LLM and agent types - Streamline chat completion request processing - Integrate OpenAI client dependency - Simplify stream and non-stream response handling - Remove unused imports and settings - Add OpenAI client initialization in openai_clients.py
- Remove separate handlers for LLM and agent app types - Simplify request handling by using a single OpenAI client - Remove unused imports and redundant code - Update function signatures to include OpenAI client parameter - Adjust API endpoints to use the new OpenAI client dependency
- Simplify chat completion handler to use OpenAI client directly - Refactor text completion handler for better code organization - Remove unused imports and simplify dependencies - Unify handling of normal and streaming requests for completions - Update router to include OpenAI client dependency
- Remove separate handling for LLM and Agent app types - Integrate OpenAI client as a dependency in API handlers - Simplify request handling logic in embeddings, images generations, edits, and variations - Remove unused imports and redundant code - Update function signatures to include OpenAI client parameter - Streamline error handling and model selection process
- Remove app type-specific logic from handlers - Introduce OpenAI client dependency - Simplify model and moderation request handling - Remove unused imports and functions - Raise 501 Not Implemented for unimplemented features - Update function signatures to include OpenAI client - Remove model registration endpoint
- Create new file `languru/types/models.py` - Define `MODELS_OPENAI` tuple with 28 OpenAI model names - Add `MODELS_AZURE_OPENAI` tuple using a generator expression to create Azure versions of OpenAI model names
- Introduce `depends_openai_client_chat_completion_request` function to handle organization type and chat completion request dependencies - Refactor `chat_completions` endpoint to use the new dependency function - Implement `org_from_model` method in `OpenaiClients` class to determine organization type based on model name - Update `org_to_openai_client` method (formerly `_org_to_openai_client`) - Add `default_openai_client` method to `OpenaiClients` class - Simplify test cases for chat completions - Update Azure OpenAI client initialization with specific API version - Import model constants (MODELS_AZURE_OPENAI, MODELS_OPENAI) - Remove unused code and imports
- Reorganize imports and remove unused TYPE_CHECKING - Add support for multiple AI providers (Anthropic, Google, Groq, Perplexity, Voyage) - Implement models() method to return supported models for each provider - Update ModelsHandler and RetrieveModelHandler to use new models() method - Add new model lists for various AI providers in types/models.py - Refactor error handling for model retrieval - Update API endpoints to use organization_type instead of openai_client
- Replace OpenAI's NotFoundError with custom ModelNotFound exception - Simplify test suite for model API endpoints - Update OpenAI clients to use custom exception - Remove unused imports and clean up code
- Introduce `depends_openai_client_completion_request` function for better dependency injection - Update `TextCompletionHandler` to use the new dependency - Simplify `text_completions` endpoint by leveraging the new dependency - Refactor test cases to use a module-scoped test client fixture - Remove environment-specific tests and consolidate into unified test cases - Update imports and remove unused code
- Enhance embeddings endpoint with organization type handling - Add dependency for OpenAI client and embedding request - Simplify test structure for embeddings and completions - Remove unnecessary environment and model discovery mocks - Update imports and type hints for better code clarity
- Introduce `depends_openai_client_model` and `depends_openai_client_images_generations_request` functions - Update image generation, editing, and variation endpoints to use new dependency functions - Simplify test cases by removing separate LLM and agent app tests - Replace individual test fixtures with a single `test_client` fixture - Remove unused imports and simplify code structure - Update type hints and error handling in image-related API functions
- Introduce `depends_openai_client_model` and `depends_openai_client_audio_speech_request` functions - Update audio endpoints to use new dependency functions - Refactor `AudioSpeechHandler` to use `dummy_generator_func` - Simplify test structure by removing separate LLM and agent tests - Add `test_app_audio_translations` function - Implement `dummy_generator_func` in `common.py` - Update imports and remove unused imports - Adjust error handling and type annotations
- Update audio, images, and moderations handlers to use typed Depends - Introduce new dependency function for moderation requests - Adjust type hints for OpenAI client and request objects - Remove redundant imports and simplify test setup - Add 'text-moderation-latest' and 'text-moderation-stable' to MODELS_OPENAI This change improves type safety and readability across the codebase.
- Remove separate LLM and agent app tests - Introduce a module-scoped test_client fixture - Simplify health check test to use the new fixture - Remove unnecessary imports and type checking
- Add root endpoint ("/") to the FastAPI app
- Update CLI entry point in pyproject.toml from "app" to "languru_cli"
- Retain existing "/health" endpoint
- Rename 'languru-agent' to 'languru-server' - Remove individual LLM service configurations (OpenAI, Google, PPLX, Groq, Anthropic) - Update Dockerfiles to install 'languru[all]' instead of specific extras - Remove separate Dockerfiles for Google and Anthropic - Update logging configuration to use dynamic logger name - Replace OrganizationNotFound exception with CredentialsNotProvided in OpenAI clients - Minor code cleanup and import optimizations
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #43 +/- ##
==========================================
- Coverage 80.04% 79.90% -0.14%
==========================================
Files 73 62 -11
Lines 3382 2125 -1257
==========================================
- Hits 2707 1698 -1009
+ Misses 675 427 -248 ☔ View full report in Codecov by Sentry. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Server Refactoring
APP_TYPEdistinction betweenllmandagent, simplifying the server architecture.servercommand, replacing separateagentandllmcommands.languru/server/app.py, removing the lifespan context manager and simplifying the app creation.OpenAI Client Integration
OpenaiClientsclass to manage different OpenAI-compatible clients (OpenAI, Azure, Anthropic, Google, Groq, Perplexity, Voyage).API Endpoints
llmandagenttypes.Model Management
OpenaiClientsclass for model information.Configuration
ServerBaseSettingsto removeAPP_TYPEand related settings.Testing
llmandagentenvironments.OpenaiClientsclass and simplified API calls.Cleanup
languru/action/directory.OpenaiClientsclass.Docker
-E allflag instead of separate installations for different components.