v2.42.0

WebDriver Support

Full W3C WebDriver protocol support via thirtyfour crate for Selenium Grid, remote browsers, and cross-browser testing.

use spider::website::Website;
use spider::features::webdriver_common::{WebDriverConfig, WebDriverBrowser};

let mut website = Website::new("https://example.com")
    .with_webdriver(
        WebDriverConfig::new()
            .with_server_url("http://localhost:4444")
            .with_browser(WebDriverBrowser::Chrome)
            .with_headless(true)
    )
    .build()
    .unwrap();

website.crawl().await;

Features

• Browser Support: Chrome, Firefox, Edge via WebDriver protocol
• Remote Connections: Selenium Grid, BrowserStack, SauceLabs compatible
• Stealth Mode: Automation marker cleanup when webdriver_stealth enabled
  • Hides navigator.webdriver
  • Spoofs navigator.deviceMemory
  • Cleans CDP markers (cdc_, $cdc_)
  • Removes Selenium markers
• Automation Scripts: Reuses existing chrome automation configuration

Feature Flags

[dependencies]
spider = { version = "2.42", features = ["webdriver"] }
# With stealth:
spider = { version = "2.42", features = ["webdriver", "webdriver_stealth"] }

New Files

• spider/src/features/webdriver.rs
• spider/src/features/webdriver_common.rs
• spider/src/features/webdriver_args.rs
• examples/webdriver.rs
• examples/webdriver_remote.rs