Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ie/Boosty] include auth token in API call if available #9522

Merged
merged 10 commits into from
May 8, 2024

Conversation

RasmusAntons
Copy link
Contributor

Fix extracting video URLs from paywalled posts on Boosty

For paywalled posts (posts that require a subscription level), Boosty's API only returns the video URLs if the access token is set. This change attempts to extract the token from the provided cookies and sets the authorization header for the API call accordingly.

Fixes #9401

Template

Before submitting a pull request make sure you have:

In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under Unlicense. Check all of the following options that apply:

  • I am the original author of this code and I am willing to release it under Unlicense
  • I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)

What is the purpose of your pull request?

@seproDev seproDev added the site-enhancement Feature request for some website label Mar 25, 2024
yt_dlp/extractor/boosty.py Outdated Show resolved Hide resolved
@un-def
Copy link
Contributor

un-def commented Apr 17, 2024

I've just checked and can confirm that this works 👍

yt_dlp/extractor/boosty.py Outdated Show resolved Hide resolved
Copy link
Member

@pukkandan pukkandan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

yt_dlp/extractor/boosty.py Outdated Show resolved Hide resolved
yt_dlp/extractor/boosty.py Outdated Show resolved Hide resolved
yt_dlp/extractor/boosty.py Outdated Show resolved Hide resolved
@DocFreeman
Copy link

I've had no luck with this, tested on Windows and MacOS.

@bashonly
Copy link
Member

bashonly commented May 8, 2024

@DocFreeman you pulled the branch and ran from source? can you show a verbose log please?

@DocFreeman
Copy link

DocFreeman commented May 8, 2024

@DocFreeman you pulled the branch and ran from source? can you show a verbose log please?

Yes, for MacOS I did that. For Windows I just pulled a nightly build to see if it could partially parse it at least but it could not.

/Users/******/Downloads/yt-dlp-boosty/yt-dlp -v https://boosty.to/******media/video/f578be90-4a0d-4a88-bb31-e9844b613df3 [debug] Command-line config: ['-v', 'https://boosty.to/*****/media/video/f578be90-4a0d-4a88-bb31-e9844b613df3'] [debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8 [debug] yt-dlp version stable@2024.04.09 from yt-dlp/yt-dlp [ff0779267] (zip) [debug] Python 3.12.3 (CPython arm64 64bit) - macOS-14.2.1-arm64-arm-64bit (OpenSSL 3.0.13 30 Jan 2024) [debug] exe versions: none [debug] Optional libraries: Cryptodome-3.20.0, brotli-1.1.0, certifi-2024.02.02, mutagen-1.47.0, requests-2.31.0, sqlite3-3.45.1, urllib3-2.2.1, websockets-12.0 [debug] Proxy map: {} [debug] Request Handlers: urllib, requests, websockets [debug] Plugin directories: ['/Users/******/.config/yt-dlp/plugins/boosty/yt_dlp_plugins'] [debug] Loaded 1810 extractors [generic] Extracting URL: https://boosty.to/*****/media/video/f578be90-4a0d-4a88-bb31-e9844b613df3 [generic] f578be90-4a0d-4a88-bb31-e9844b613df3: Downloading webpage [redirect] Following redirect to https://boosty.to/ [generic] Extracting URL: https://boosty.to/ [generic] boosty: Downloading webpage WARNING: [generic] Falling back on generic information extractor [generic] boosty: Extracting information [debug] Looking for embeds ERROR: Unsupported URL: https://boosty.to/ Traceback (most recent call last): File "/Users/******/Downloads/yt-dlp-boosty/yt-dlp/yt_dlp/YoutubeDL.py", line 1606, in wrapper return func(self, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/******/Downloads/yt-dlp-boosty/yt-dlp/yt_dlp/YoutubeDL.py", line 1741, in __extract_info ie_result = ie.extract(url) ^^^^^^^^^^^^^^^ File "/Users/******/Downloads/yt-dlp-boosty/yt-dlp/yt_dlp/extractor/common.py", line 734, in extract ie_result = self._real_extract(url) ^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/******/Downloads/yt-dlp-boosty/yt-dlp/yt_dlp/extractor/generic.py", line 2514, in _real_extract raise UnsupportedError(url) yt_dlp.utils.UnsupportedError: Unsupported URL: https://boosty.to/

@bashonly
Copy link
Member

bashonly commented May 8, 2024

  1. /media/video/ URLs are not supported (yet) by the extractor
  2. that is the stable unix/zip release, it does not have the PR's patch applied
  3. you're not passing cookies to yt-dlp

This PR specifically only fixes paywalled /posts/ URLs when the user passes cookies to yt-dlp

@bashonly bashonly merged commit 145dc6f into yt-dlp:master May 8, 2024
6 checks passed
@shilin-da
Copy link

It's working! Thanks a lot! 🤝

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
site-enhancement Feature request for some website
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Boosty] No videos found
7 participants