feat: add Firecrawl web search tools#7764
Conversation
There was a problem hiding this comment.
Hey - I've left some high level feedback:
- The Firecrawl search and scrape helpers duplicate the same API key/header/ClientSession setup logic; consider extracting a shared internal helper to reduce duplication and keep future changes (e.g., base URL or headers) in one place.
- Both
_firecrawl_searchand_firecrawl_scraperaise a genericExceptionfor HTTP errors; using more specific exception types (e.g., a custom web-search error orRuntimeError/ValueError) would make it easier for callers to distinguish between configuration, network, and API-level failures.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- The Firecrawl search and scrape helpers duplicate the same API key/header/ClientSession setup logic; consider extracting a shared internal helper to reduce duplication and keep future changes (e.g., base URL or headers) in one place.
- Both `_firecrawl_search` and `_firecrawl_scrape` raise a generic `Exception` for HTTP errors; using more specific exception types (e.g., a custom web-search error or `RuntimeError`/`ValueError`) would make it easier for callers to distinguish between configuration, network, and API-level failures.Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
There was a problem hiding this comment.
Code Review
This pull request integrates Firecrawl as a new web search and page extraction provider, adding the FirecrawlWebSearchTool and FirecrawlExtractWebPageTool. The changes span the core agent logic, configuration defaults, tool implementations, dashboard UI, and unit tests. Review feedback identifies a critical bug in the Firecrawl API response parsing that would lead to an AttributeError, the inclusion of unsupported search parameters (tbs), potential TypeErrors when handling null arguments, and opportunities to improve performance and code reuse by refactoring HTTP session management.
Soulter
left a comment
There was a problem hiding this comment.
按原来那样创建新的 aiohttp.session 实例会更好一些,防止资源泄漏以及不使用websearch功能的用户可以避免创建模块级 session 实例。
…improved session management as it was
… add corresponding tests
…improved error handling and session management
…use default limit in payload
Closes #7761
添加 Firecrawl 作为内置网页搜索提供商。
本 PR 在内置 Function Tool 网页搜索层接入 Firecrawl,并对齐现有 Tavily 的使用方式。用户可以在网页搜索提供商中选择 Firecrawl,用于网页搜索和指定 URL 页面内容提取。
Modifications / 改动点
新增
web_search_firecrawl内置网页搜索工具,使用 Firecrawl/v2/search。新增
firecrawl_extract_web_page内置页面内容提取工具,使用 Firecrawl/v2/scrape。新增
provider_settings.websearch_firecrawl_key配置项,支持多个 Firecrawl API Key 轮询。在内置网页搜索提供商配置选项中加入
firecrawl。更新 Agent 网页搜索工具注入逻辑,选择
firecrawl时同时注册搜索工具和页面提取工具。更新 Dashboard 配置元数据翻译,补充 Firecrawl API Key 文案。
更新旧版 ChatUI 网页搜索结果解析逻辑,支持识别
web_search_firecrawl。新增单元测试,覆盖 Firecrawl 工具注册、配置迁移、搜索参数映射、页面提取输出和 Agent 工具注入。
This is NOT a breaking change. / 这不是一个破坏性变更。
Screenshots or Test Results / 运行截图或测试结果
日志1




日志2
配置栏
使用截图
Checklist / 检查清单
😊 If there are new features added in the PR, I have discussed it with the authors through issues/emails, etc.
/ 如果 PR 中有新加入的功能,已经通过 Issue / 邮件等方式和作者讨论过
我在此issue提到了该功能[Feature]关于网页搜索的自定义引擎问题 #7761
👀 My changes have been well-tested, and "Verification Steps" and "Screenshots" have been provided above.
/ 我的更改经过了良好的测试,并已在上方提供了“验证步骤”和“运行截图”。
🤓 I have ensured that no new dependencies are introduced, OR if new dependencies are introduced, they have been added to the appropriate locations in
requirements.txtandpyproject.toml./ 我确保没有引入新依赖库,或者引入了新依赖库的同时将其添加到
requirements.txt和pyproject.toml文件相应位置。😮 My changes do not introduce malicious code.
/ 我的更改没有引入恶意代码。
Summary by Sourcery
Integrate Firecrawl as a built-in web search provider alongside existing engines, including both search and page-extraction capabilities, configuration, and agent/dashboard wiring.
New Features:
web_search_firecrawlbacked by the Firecrawl Search API.firecrawl_extract_web_pagebacked by the Firecrawl scrape API.firecrawlas a selectable web search provider in chat configuration with support for multiple API keys viaprovider_settings.websearch_firecrawl_key.Enhancements:
websearch_firecrawl_keyto support list-based key rotation and align with other providers.Tests: