feat: add provider-agnostic tool_web_search() and tool_web_fetch()

CHANGELOG.md

@@ -11,11 +11,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### New features
 
-
-* `ChatOpenAI()`, `ChatAnthropic()`, and `ChatGoogle()` gain a new `reasoning` parameter to easily opt-into, and fully customize, reasoning capabilities. (#202) 
+* `ChatOpenAI()`, `ChatAnthropic()`, and `ChatGoogle()` gain a new `reasoning` parameter to easily opt-into, and fully customize, reasoning capabilities. (#202)
     * A new `ContentThinking` content type was added and captures the "thinking" portion of a reasoning model. (#192)
 * Added support for built-in provider tools via a new `ToolBuiltIn` class. This enables provider-specific functionality like OpenAI's image generation to be registered and used as tools. Built-in tools pass raw provider definitions directly to the API rather than wrapping Python functions. (#214)
 * `ChatGoogle()` gains basic support for image generation. (#214)
+* New `tool_web_search()` and `tool_web_fetch()` functions provide provider-agnostic access to built-in web search and URL fetch tools:
+    * `tool_web_search()` is supported by OpenAI, Claude (Anthropic), and Google (Gemini).
+    * `tool_web_fetch()` is supported by Claude (requires beta header) and Google.
+    * New content types `ContentToolRequestSearch`, `ContentToolResponseSearch`, `ContentToolRequestFetch`, and `ContentToolResponseFetch` capture web tool interactions.
 * `ChatOpenAI()` and `ChatAzureOpenAI()` gain a new `service_tier` parameter to request a specific service tier (e.g., `"flex"` for slower/cheaper or `"priority"` for faster/more expensive). (#204)
 * `Chat` and `Turn` now have a `_repr_markdown_` method and an overall improved `repr()` experience. (#245)
 

CLAUDE.md

@@ -51,6 +51,14 @@ The project uses `uv` for package management and Make for common tasks:
 4. **Content-Based Messaging**: All communication uses structured `Content` objects rather than raw strings
 5. **Tool Integration**: Seamless function calling with automatic JSON schema generation from Python type hints
 
+### Typing Best Practices
+
+This project prioritizes strong typing that leverages provider SDK types directly:
+
+- **Use provider SDK types**: Import and use types from `openai.types`, `anthropic.types`, `google.genai.types`, etc. rather than creating custom TypedDicts or dataclasses that mirror them. This ensures compatibility with SDK updates and provides better IDE support.
+- **Use `@overload` for provider-specific returns**: When a method returns different types based on a provider argument, use `@overload` with `Literal` types to give callers precise return type information.
+- **Explore SDK types interactively**: Use `python -c "from <sdk>.types import <Type>; print(<Type>.__annotations__)"` to inspect available fields and nested types when implementing provider-specific features.
+
 ### Testing Structure
 
 - Tests are organized by component (e.g., `test_provider_openai.py`, `test_tools.py`)

chatlas/__init__.py

@@ -35,6 +35,7 @@
 from ._provider_snowflake import ChatSnowflake
 from ._tokens import token_usage
 from ._tools import Tool, ToolBuiltIn, ToolRejectError
+from ._tools_builtin import tool_web_fetch, tool_web_search
 from ._turn import AssistantTurn, SystemTurn, Turn, UserTurn
 
 try:
@@ -86,6 +87,8 @@
     "Tool",
     "ToolBuiltIn",
     "ToolRejectError",
+    "tool_web_fetch",
+    "tool_web_search",
     "Turn",
     "UserTurn",
     "SystemTurn",

chatlas/_content.py

@@ -132,6 +132,10 @@ def from_tool(cls, tool: "Tool | ToolBuiltIn") -> "ToolInfo":
     "json",
     "pdf",
     "thinking",
+    "web_search_request",
+    "web_search_results",
+    "web_fetch_request",
+    "web_fetch_results",
 ]
 """
 A discriminated union of all content types.
@@ -622,6 +626,103 @@ def tagify(self):
         return HTML(html)
 
 
+class ContentToolRequestSearch(Content):
+    """
+    A web search request from the model.
+
+    This content type represents the model's request to search the web.
+    It's automatically generated when a built-in web search tool is used.
+
+    Parameters
+    ----------
+    query
+        The search query.
+    extra
+        The raw provider-specific response data.
+    """
+
+    query: str
+    extra: Optional[dict[str, Any]] = None
+
+    content_type: ContentTypeEnum = "web_search_request"
+
+    def __str__(self):
+        return f"[web search request]: {self.query!r}"
+
+
+class ContentToolResponseSearch(Content):
+    """
+    Web search results from the model.
+
+    This content type represents the results of a web search.
+    It's automatically generated when a built-in web search tool returns results.
+
+    Parameters
+    ----------
+    urls
+        The URLs returned by the search.
+    extra
+        The raw provider-specific response data.
+    """
+
+    urls: list[str]
+    extra: Optional[dict[str, Any]] = None
+
+    content_type: ContentTypeEnum = "web_search_results"
+
+    def __str__(self):
+        url_list = "\n".join(f"* {url}" for url in self.urls)
+        return f"[web search results]:\n{url_list}"
+
+
+class ContentToolRequestFetch(Content):
+    """
+    A web fetch request from the model.
+
+    This content type represents the model's request to fetch a URL.
+    It's automatically generated when a built-in web fetch tool is used.
+
+    Parameters
+    ----------
+    url
+        The URL to fetch.
+    extra
+        The raw provider-specific response data.
+    """
+
+    url: str
+    extra: Optional[dict[str, Any]] = None
+
+    content_type: ContentTypeEnum = "web_fetch_request"
+
+    def __str__(self):
+        return f"[web fetch request]: {self.url}"
+
+
+class ContentToolResponseFetch(Content):
+    """
+    Web fetch results from the model.
+
+    This content type represents the results of fetching a URL.
+    It's automatically generated when a built-in web fetch tool returns results.
+
+    Parameters
+    ----------
+    url
+        The URL that was fetched.
+    extra
+        The raw provider-specific response data.
+    """
+
+    url: str
+    extra: Optional[dict[str, Any]] = None
+
+    content_type: ContentTypeEnum = "web_fetch_results"
+
+    def __str__(self):
+        return f"[web fetch result]: {self.url}"
+
+
 ContentUnion = Union[
     ContentText,
     ContentImageRemote,
@@ -631,6 +732,10 @@ def tagify(self):
     ContentJson,
     ContentPDF,
     ContentThinking,
+    ContentToolRequestSearch,
+    ContentToolResponseSearch,
+    ContentToolRequestFetch,
+    ContentToolResponseFetch,
 ]
 
 
@@ -661,6 +766,14 @@ def create_content(data: dict[str, Any]) -> ContentUnion:
         return ContentPDF.model_validate(data)
     elif ct == "thinking":
         return ContentThinking.model_validate(data)
+    elif ct == "web_search_request":
+        return ContentToolRequestSearch.model_validate(data)
+    elif ct == "web_search_results":
+        return ContentToolResponseSearch.model_validate(data)
+    elif ct == "web_fetch_request":
+        return ContentToolRequestFetch.model_validate(data)
+    elif ct == "web_fetch_results":
+        return ContentToolResponseFetch.model_validate(data)
     else:
         raise ValueError(f"Unknown content type: {ct}")
 

chatlas/_provider_anthropic.py

@@ -27,6 +27,10 @@
     ContentText,
     ContentThinking,
     ContentToolRequest,
+    ContentToolRequestFetch,
+    ContentToolRequestSearch,
+    ContentToolResponseFetch,
+    ContentToolResponseSearch,
     ContentToolResult,
 )
 from ._logging import log_model_default
@@ -39,6 +43,7 @@
 )
 from ._tokens import get_price_info
 from ._tools import Tool, ToolBuiltIn, basemodel_to_param_schema
+from ._tools_builtin import ToolWebFetch, ToolWebSearch
 from ._turn import AssistantTurn, SystemTurn, Turn, UserTurn, user_turn
 from ._utils import split_http_client_kwargs
 
@@ -494,6 +499,11 @@ def stream_merge_chunks(self, completion, chunk):
             elif chunk.delta.type == "signature_delta":
                 this_content = cast("ThinkingBlock", this_content)
                 this_content.signature += chunk.delta.signature
+            elif chunk.delta.type == "citations_delta":
+                # https://docs.claude.com/en/docs/build-with-claude/citations#streaming-support
+                # Accumulate citations on the content block
+                if hasattr(this_content, "citations"):
+                    this_content.citations.append(chunk.delta.citation)  # type: ignore
         elif chunk.type == "content_block_stop":
             this_content = completion.content[chunk.index]
             if this_content.type == "tool_use" and isinstance(this_content.input, str):
@@ -695,11 +705,28 @@ def _as_content_block(content: Content) -> "ContentBlockParam":
                 "thinking": content.thinking,
                 "signature": extra.get("signature", ""),
             }
+        elif isinstance(
+            content,
+            (
+                ContentToolRequestSearch,
+                ContentToolResponseSearch,
+                ContentToolRequestFetch,
+                ContentToolResponseFetch,
+            ),
+        ):
+            # extra contains the full original content block param
+            return cast("ContentBlockParam", content.extra)
 
         raise ValueError(f"Unknown content type: {type(content)}")
 
     @staticmethod
     def _anthropic_tool_schema(tool: "Tool | ToolBuiltIn") -> "ToolUnionParam":
+        if isinstance(tool, ToolWebSearch):
+            return tool.get_definition("anthropic")
+        if isinstance(tool, ToolWebFetch):
+            # N.B. seems the return type here (BetaWebFetchTool20250910Param) is
+            # not a member of ToolUnionParam since it's still in beta?
+            return tool.get_definition("anthropic")  # type: ignore
         if isinstance(tool, ToolBuiltIn):
             return tool.definition  # type: ignore
 
@@ -757,6 +784,76 @@ def _as_turn(self, completion: Message, has_data_model=False) -> AssistantTurn:
                         extra={"signature": content.signature},
                     )
                 )
+            elif content.type == "server_tool_use":
+                # Unfortunately, content.model_dump() includes fields like "url"
+                # that aren't acceptable as API input, so we manually construct
+                # the extra dict
+                if isinstance(content.input, str):
+                    input_data = orjson.loads(content.input)
+                else:
+                    input_data = content.input
+
+                extra = {
+                    "type": content.type,
+                    "id": content.id,
+                    "name": content.name,
+                    "input": input_data,
+                }
+                # https://docs.claude.com/en/docs/agents-and-tools/tool-use/web-search-tool#response
+                if content.name == "web_search":
+                    contents.append(
+                        ContentToolRequestSearch(
+                            query=str(input_data.get("query", "")),
+                            extra=extra,
+                        )
+                    )
+                # https://docs.claude.com/en/docs/agents-and-tools/tool-use/web-fetch-tool#response
+                elif content.name == "web_fetch":
+                    # N.B. type checker thinks this is unreachable due to
+                    # ToolUnionParam not including BetaWebFetchTool20250910Param
+                    # yet
+                    contents.append(
+                        ContentToolRequestFetch(
+                            url=str(input_data.get("url", "")),
+                            extra=extra,
+                        )
+                    )
+                else:
+                    raise ValueError(f"Unknown server tool: {content.name}")
+            elif content.type == "web_search_tool_result":
+                # https://docs.claude.com/en/docs/agents-and-tools/tool-use/web-search-tool#response
+                urls: list[str] = []
+                if isinstance(content.content, list):
+                    urls = [x.url for x in content.content]
+                contents.append(
+                    ContentToolResponseSearch(
+                        urls=urls,
+                        extra=content.model_dump(),
+                    )
+                )
+            elif content.type == "web_fetch_tool_result":
+                # N.B. type checker thinks this is unreachable due to
+                # ToolUnionParam not including BetaWebFetchTool20250910Param
+                # yet. Also, at run-time, the SDK is currently giving non-sense
+                # of type(content) == TextBlock, but it doesn't even fit that
+                # shape?!? Anyway, content.content has a dict with the content
+                # we want.
+                content_fetch = cast("dict", getattr(content, "content", {}))
+                if not content_fetch:
+                    raise ValueError(
+                        "web_fetch_tool_result content is empty. Please report this issue."
+                    )
+                extra = {
+                    "type": "web_fetch_tool_result",
+                    "tool_use_id": content.tool_use_id,  # type: ignore
+                    "content": content_fetch,
+                }
+                contents.append(
+                    ContentToolResponseFetch(
+                        url=content_fetch.get("url", "failed"),
+                        extra=extra,
+                    )
+                )
 
         return AssistantTurn(
             contents,

chatlas/_provider_google.py

@@ -22,6 +22,7 @@
 from ._provider import ModelInfo, Provider, StandardModelParamNames, StandardModelParams
 from ._tokens import get_price_info
 from ._tools import Tool, ToolBuiltIn
+from ._tools_builtin import ToolWebFetch, ToolWebSearch
 from ._turn import AssistantTurn, SystemTurn, Turn, UserTurn, user_turn
 
 if TYPE_CHECKING:
@@ -295,7 +296,11 @@ def _chat_perform_args(
         data_model: Optional[type[BaseModel]] = None,
         kwargs: Optional["SubmitInputArgs"] = None,
     ) -> "SubmitInputArgs":
-        from google.genai.types import FunctionDeclaration, GenerateContentConfig
+        from google.genai.types import (
+            FunctionDeclaration,
+            GenerateContentConfig,
+            ToolListUnion,
+        )
         from google.genai.types import Tool as GoogleTool
 
         kwargs_full: "SubmitInputArgs" = {
@@ -319,20 +324,30 @@ def _chat_perform_args(
             config.response_mime_type = "application/json"
 
         if tools:
-            config.tools = [
-                GoogleTool(
-                    function_declarations=[
-                        FunctionDeclaration.from_callable(
-                            client=self._client._api_client,
-                            callable=tool.func,
-                        )
-                        for tool in tools.values()
-                        # TODO: to support built-in tools, we may need a way to make
-                        # tool names (e.g., google_search to google.genai.types.GoogleSearch())
-                        if isinstance(tool, Tool)
-                    ]
-                )
-            ]
+            google_tools: ToolListUnion = []
+            for tool in tools.values():
+                if isinstance(tool, ToolWebSearch):
+                    gtool = GoogleTool(google_search=tool.get_definition("google"))
+                    google_tools.append(gtool)
+                elif isinstance(tool, ToolWebFetch):
+                    gtool = GoogleTool(url_context=tool.get_definition("google"))
+                    google_tools.append(gtool)
+                elif isinstance(tool, ToolBuiltIn):
+                    gtool = GoogleTool.model_validate(tool.definition)
+                    google_tools.append(gtool)
+                else:
+                    gtool = GoogleTool(
+                        function_declarations=[
+                            FunctionDeclaration.from_callable(
+                                client=self._client._api_client,
+                                callable=tool.func,
+                            )
+                        ]
+                    )
+                    google_tools.append(gtool)
+
+            if google_tools:
+                config.tools = google_tools
 
         kwargs_full["config"] = config