Skip to content

feat(v1.1): migrate to wreq for TLS fingerprint impersonation (Layer 2 WAF Evasion) #12

Description

@XaviCode1000

Problem

The current HTTP client uses reqwest with rustls-tls-native-roots. This emits a rustls TLS fingerprint (JA3), not a Chrome fingerprint. Any WAF with TLS fingerprinting (Cloudflare, DataDome, Akamai) detects the scraper immediately.

Current state (v1.0.6): Capa 1 implemented — WAF challenge detection in HTTP 200 responses via signature matching. This catches ~70% of false-200 pages but does NOT prevent detection.

Proposed Solution

Migrate from reqwest to wreq (731K downloads, 718 stars, hard fork of reqwest). Uses BoringSSL for real TLS/JA3/JA4 fingerprint impersonation.

use wreq::Client;
use wreq_util::Emulation;
let client = Client::builder().emulation(Emulation::Chrome131).build()?;

Scope

  • Replace reqwest with wreq + wreq-util
  • Rewrite reqwest_middleware → wreq native tower middleware
  • Replace reqwest_retry → manual backoff or wreq middleware
  • Verify BoringSSL doesn't conflict with openssl-sys
  • Update all 267 tests

Risks

  • BoringSSL vs openssl-sys symbol conflict
  • reqwest-middleware/retry tightly coupled to reqwest
  • MSRV 1.85+ (project is 1.88+, OK)

Acceptance Criteria

  • wreq replaces reqwest for all HTTP operations
  • TLS fingerprint matches Chrome 131+
  • All 267+ tests pass
  • clippy clean
  • CAPTCHA detection still works as fallback

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions