Skip to content

Built-in tools should fail loudly on providers/platforms that don't support them #367

Description

@cpsievert

Built-in tools currently fail silently (or with cryptic provider errors) in two situations where a loud, early error would save users real confusion.

1. Providers with only the generic ToolBuiltIn fallback send {}

ToolWebSearch / ToolWebFetch initialize with definition={} (chatlas/_tools_builtin.py:113) — the real definition is only produced when a provider calls get_definition(provider_name). Providers that special-case these classes (openai, anthropic, google) do that; providers that only have the generic fallback pass the raw definition through:

# chatlas/_provider_openai_completions.py:206
if isinstance(tool, ToolBuiltIn):
    tool_schemas.append(tool.definition)   # {} for ToolWebSearch

So ChatOpenAICompletions() (or any ChatOpenAI-compatible generic provider) with tool_web_search() registered sends an empty tool spec — at best a confusing API 400, at worst silently ignored.

Proposal: when the generic fallback encounters a ToolBuiltIn subclass whose definition is empty (i.e., a provider-agnostic tool this provider has no mapping for), raise a clear ValueError naming the tool and provider instead of sending {}.

2. Hosted platforms that don't support the upstream provider's server tools

AnthropicBedrockProvider inherits AnthropicProvider's tool translation unchanged, but per Anthropic's tool docs, Amazon Bedrock supports essentially none of the Anthropic server tools (no web search, web fetch, or code execution), and Vertex AI supports only the basic web search variant. Worth verifying current behavior on Bedrock and, if it fails, raising/warning at register_tool() or request-build time rather than letting the platform reject the request.

This matters more as the built-in tool lineup grows (code execution, MCP connector, ...): each new tool multiplies the silent-gap surface.

Metadata

Metadata

Assignees

No one assigned

    Labels

    ai-triage:needs-reviewNeeds human review before the workflow takes further action.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions