Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remediate File Downloader cookie leak #32445

Merged
merged 13 commits into from
Jul 18, 2023
Merged

Remediate File Downloader cookie leak #32445

merged 13 commits into from
Jul 18, 2023

Conversation

dirkf
Copy link
Contributor

@dirkf dirkf commented Jul 18, 2023

Boilerplate: mixed code, improvement ## Please follow the guide below
  • You will be asked some questions, please read them carefully and answer honestly
  • Put an x into all the boxes [ ] relevant to your pull request (like that [x])
  • Use Preview tab to see how your pull request will actually look like

Before submitting a pull request make sure you have:

In order to be accepted and merged into youtube-dl each piece of code must be in public domain or released under Unlicense. Check one of the following options:

  • I am the original author of this code and I am willing to release it under Unlicense, except for code from yt-dlp for which this or the below has been asserted
  • I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)

What is the purpose of your pull request?

  • Bug fix
  • Improvement
  • New extractor
  • New feature

Description of your pull request and other information

During file downloads, youtube-dl or the external downloaders that it invokes may leak cookies on HTTP redirects to a different host, or leak them when the host for download fragments differs from their parent manifest's host.

This PR implements various remediations to avoid this behaviour:

  • removing the Cookie header upon HTTP redirects
  • having native downloaders calculate their own Cookie header from the Cookiejar
  • using each external downloader's built-in support for cookies, instead of passing them as header arguments; or if the external downloader does not have proper cookie support, disabling HTTP redirection (axel only)
  • processing cookies passed as HTTP headers to limit their scope (--add-header "Cookie:..." is scoped to input URL domain only)
  • storing cookies in a separate cookies field of the info dict instead of http_headers so as not to lose their scope.

These remediations match similar ones committed to yt-dlp:
yt-dlp/yt-dlp@1ceb657
yt-dlp/yt-dlp@f8b4bcc
yt-dlp/yt-dlp@3121512
yt-dlp/yt-dlp@6c5211c
yt-dlp/yt-dlp@42ded0a

Thanks to yt-dlp devs involved in GHSA-v8mc-9377-rwjj, especially those referenced in commits here.

@dirkf dirkf merged commit 47214e4 into ytdl-org:master Jul 18, 2023
20 checks passed
dirkf referenced this pull request in ytdl-org/ytdl-nightly Jul 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants