Skip to content

fix(fetchers): enforce fetch options on YouTube secondary requests#117

Merged
chaliy merged 1 commit into
mainfrom
2026-05-17-fix-youtube-fetcher-outbound-url-policies
May 17, 2026
Merged

fix(fetchers): enforce fetch options on YouTube secondary requests#117
chaliy merged 1 commit into
mainfrom
2026-05-17-fix-youtube-fetcher-outbound-url-policies

Conversation

@chaliy
Copy link
Copy Markdown
Contributor

@chaliy chaliy commented May 17, 2026

Motivation

  • The YouTube fetcher validated the caller-provided URL but performed secondary outbound requests to www.youtube.com without applying FetchOptions checks, creating an egress-policy / SSRF bypass when short URLs like youtu.be were allowed.

Description

  • Call options.validate_url on the oEmbed endpoint before issuing the oEmbed request and on the timedtext transcript URL before requesting captions.
  • Disable automatic redirects on the dedicated YouTube reqwest client by using .redirect(reqwest::redirect::Policy::none()) to avoid unvalidated redirect hops.
  • Update fetch_transcript signature to accept &FetchOptions and validate the timedtext URL prior to sending the request.
  • Add a regression test test_fetch_blocked_secondary_host asserting that a youtu.be input is rejected when .youtube.com is in blocked_hosts.

Testing

  • Ran formatter with cargo fmt --all which completed successfully.
  • Ran unit tests for the crate with cargo test -p fetchkit (targeted YouTube tests executed) and all relevant tests passed (16 passed; 0 failed).
  • The added regression test fetchers::youtube::tests::test_fetch_blocked_secondary_host passed as part of the crate test run.

Codex Task

@chaliy chaliy force-pushed the 2026-05-17-fix-youtube-fetcher-outbound-url-policies branch from 5a17211 to fab77b1 Compare May 17, 2026 18:09
@chaliy chaliy merged commit a3dbfdc into main May 17, 2026
11 checks passed
@chaliy chaliy deleted the 2026-05-17-fix-youtube-fetcher-outbound-url-policies branch May 17, 2026 18:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant