Correctly passing API key to moderation #36
Conversation
There was a problem hiding this comment.
Pull Request Overview
This PR refactors the moderation guardrail's client selection logic to simplify how it chooses between the context client and fallback OpenAI client. The previous implementation attempted to detect if the context client pointed to the official OpenAI API by inspecting base URLs; the new approach attempts to use any context client first and falls back to the default OpenAI client if a NotFoundError is raised (indicating the moderation endpoint doesn't exist on third-party providers).
Key changes:
- Simplified client selection logic using try-catch instead of URL inspection
- Added
NotFoundErrorexception handling for graceful fallback to OpenAI client - Added test coverage for both context client usage and third-party provider fallback scenarios
Reviewed Changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| src/guardrails/checks/text/moderation.py | Refactored moderation function to use try-catch pattern for client selection instead of URL-based detection; added NotFoundError import and exception handling for fallback logic |
| tests/unit/checks/test_moderation.py | Added two new tests to verify context client usage and fallback behavior when moderation endpoint is unavailable |
| tests/conftest.py | Added stub NotFoundError exception to support testing the new fallback logic |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull Request Overview
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| # Try the context client first, fall back if moderation endpoint doesn't exist | ||
| if client is not None: | ||
| try: | ||
| candidate = getattr(context, "guardrail_llm", None) | ||
| if not isinstance(candidate, AsyncOpenAI): | ||
| return None | ||
|
|
||
| # Attempt to discover the effective base URL in a best-effort way | ||
| base_url = getattr(candidate, "base_url", None) | ||
| if base_url is None: | ||
| inner = getattr(candidate, "_client", None) | ||
| base_url = getattr(inner, "base_url", None) or getattr(inner, "_base_url", None) | ||
|
|
||
| # Reuse only when clearly the official OpenAI endpoint | ||
| if base_url is None: | ||
| return candidate | ||
| if isinstance(base_url, str) and "api.openai.com" in base_url: | ||
| return candidate | ||
| return None | ||
| except Exception: | ||
| return None | ||
|
|
||
| client = _maybe_reuse_openai_client_from_ctx(ctx) or _get_moderation_client() | ||
| resp = await client.moderations.create( | ||
| model="omni-moderation-latest", | ||
| input=data, | ||
| ) | ||
| resp = await _call_moderation_api(client, data) | ||
| except NotFoundError as e: | ||
| # Moderation endpoint doesn't exist on this provider (e.g., third-party) | ||
| # Fall back to the OpenAI client | ||
| logger.debug( | ||
| "Moderation endpoint not available on context client, falling back to OpenAI: %s", | ||
| e, | ||
| ) | ||
| client = _get_moderation_client() | ||
| resp = await _call_moderation_api(client, data) | ||
| else: | ||
| # No context client, use fallback | ||
| client = _get_moderation_client() | ||
| resp = await _call_moderation_api(client, data) |
There was a problem hiding this comment.
Code duplication: Lines 187-188 and 191-192 duplicate the same logic (getting fallback client and calling moderation API). Consider refactoring to avoid this duplication. For example, you could set client = _get_moderation_client() once and call resp = await _call_moderation_api(client, data) after the if-else block.
gabor-openai
changed the title
Fixes this reported BUG:
api_keypassed in via client initialization is not properly passed tomoderationcheckThe fix:
guardrail_llm, if the moderation endpoint does not exist with that client, fall back to creating a compatible OpenAI client viaenv variableOpenAI API keyis required for the Moderation checkFuture PR to allow configuring an
api_keyspecific to the moderation check via the config file (will only be needed for users who are using non-openai models as their base client).