Skip to content

docs_site fetcher bypasses SSRF protections on docs and llms.txt URLs #94

@chaliy

Description

@chaliy

Summary

DocsSiteFetcher uses a standalone reqwest::Client for both docs-page fetches and direct llms.txt fetches. That client never applies FetchKit's DNS pinning / private-IP validation or manual redirect policy.

Impact

This breaks the default SSRF posture for any URL that lands in DocsSiteFetcher, including direct /llms.txt requests.

Confirmed locally:

  • Tool::default() successfully fetched http://127.0.0.1:<port>/llms.txt
  • Tool::default() also fetched http://docs.127.0.0.1.nip.io:<port>/

Ordinary default-fetcher requests to loopback are supposed to fail with BlockedUrl, so this is a real policy bypass rather than an expected dev-mode behavior.

Affected code

  • crates/fetchkit/src/fetchers/docs_site.rs:107-174

Repro outline

  1. Start a local HTTP server on 127.0.0.1
  2. Serve plain text on /llms.txt or /
  3. Call Tool::default().execute(...) with one of:
    • http://127.0.0.1:<port>/llms.txt
    • http://docs.127.0.0.1.nip.io:<port>/
  4. Observe a successful response instead of BlockedUrl

Expected fix

DocsSiteFetcher should reuse the shared hardened transport behavior instead of dialing its own unrestricted client.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions