Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ie/soundcloud] Add formats extractor-arg #10004

Merged
merged 3 commits into from
May 22, 2024

Conversation

bashonly
Copy link
Member

@bashonly bashonly commented May 22, 2024

The Soundcloud extractor needs to make an API call for every format available in order to extract the actual format URL (and other format metadata such as bitrate).

This is already a problem since this format endpoint has a rate-limit, and users frequently hit the rate-limit when extracting a playlist/batch of songs.

Ideally, lazy info dict would solve this problem. But now Soundcloud has introduced encrypted-hls (AES-128 HLS) formats, which are just dupes of the hls formats we already extract, so now every song makes almost double the requests to the rate-limited API endpoint.

Since lazy info dict still has a ways to go before merge, and since the hls-aes formats have severely compounded the problem, the best solution is to implement an extractor-arg to allow the user to configure which formats should actually be extracted.

hls-aes formats are excluded by default since they are dupes, but now users will be to request them if Soundcloud stops offering the unencrypted hls formats in the future.

Note: extracting the original/source quality download formats does not count towards the API rate-limit (it doesn't use the problematic endpoint) and therefore is not imapcted by this extractor-arg

Template

Before submitting a pull request make sure you have:

In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under Unlicense. Check all of the following options that apply:

  • I am the original author of this code and I am willing to release it under Unlicense
  • I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence): co-authored by @Grub4K

What is the purpose of your pull request?

  • Fix or improvement to an extractor (Make sure to add/update tests)

Authored by: bashonly, Grub4K
@bashonly bashonly added the site-enhancement Feature request for some website label May 22, 2024
@bashonly bashonly requested a review from Grub4K May 22, 2024 19:53
Authored by: bashonly
Authored by: bashonly
@bashonly bashonly merged commit beaf832 into yt-dlp:master May 22, 2024
6 checks passed
@bashonly bashonly deleted the feat/soundcloud-formats branch May 22, 2024 22:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
site-enhancement Feature request for some website
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants