v2.43.18 - Web Search Integration

Features

Web Search Integration

Add web search capabilities to Spider's RemoteMultimodalEngine with support for multiple search providers.

Supported Providers

• Serper (search_serper) - Google SERP API
• Brave (search_brave) - Privacy-focused search
• Bing (search_bing) - Microsoft Bing Web Search
• Tavily (search_tavily) - AI-optimized search

New Methods

• search() - Search the web and return structured results
• search_and_extract() - Search + fetch pages + LLM extraction
• research() - Search + extract + synthesize findings into summary

Setup

Cargo.toml

[dependencies]
spider = { version = "2.43.18", features = ["search_serper"] }

Configuration

use spider::configuration::{SearchConfig, SearchProviderType};
use spider::features::automation::RemoteMultimodalEngine;

let mut engine = RemoteMultimodalEngine::new(api_url, model, None);
engine.with_search_config(Some(
    SearchConfig::new(SearchProviderType::Serper, "your-api-key")
        // Optional: custom API endpoint
        .with_api_url("https://custom.api.com/search")
));

// Simple search
let results = engine.search("rust web crawler", None, None).await?;

// Search + extract
let data = engine.search_and_extract(
    "best rust frameworks",
    "Extract name and description",
    None,
    None,
).await?;

// Research with synthesis
use spider::features::automation::ResearchOptions;
let research = engine.research(
    "How do async runtimes work?",
    ResearchOptions::new().with_max_pages(5).with_synthesis(),
    None,
).await?;
println!("Summary: {}", research.summary.unwrap());

Custom API Endpoints

All providers support custom API URLs for self-hosted or alternative endpoints:

SearchConfig::new(SearchProviderType::Brave, "api-key")
    .with_api_url("https://my-brave-proxy.example.com/search")

Full Changelog

v2.43.17...v2.43.18