Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

option to list video URLs in playlist; want to parallelize #16881

Closed
fsiler opened this issue Jul 1, 2018 · 4 comments
Closed

option to list video URLs in playlist; want to parallelize #16881

fsiler opened this issue Jul 1, 2018 · 4 comments

Comments

@fsiler
Copy link

@fsiler fsiler commented Jul 1, 2018

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2018.06.25. If it's not, read this FAQ entry and update. Issues with outdated version will be rejected.

  • I've verified and I assure that I'm running youtube-dl 2018.06.25

Before submitting an issue make sure you have:

  • At least skimmed through the README, most notably the FAQ and BUGS sections
  • Searched the bugtracker for similar issues including closed ones
  • Checked that provided video/audio/playlist URLs (if any) are alive and playable in a browser

What is the purpose of your issue?

  • Bug report (encountered problems with youtube-dl)
  • Site support request (request for adding support for a new site)
  • Feature request (request for a new functionality)
  • Question
  • Other

Description of your issue, suggested solution and other information

Explanation of your issue in arbitrary form goes here. Please make sure the description is worded well enough to be understood. Provide as much context and examples as possible.
If work on your issue requires account credentials please provide them or explain how one can obtain them.

Can youtube-dl provide simply a list of video URLs given a playlist? I want to be able to do something like

youtube-dl --dump-playlist <playlist-URL>|parallel -j3 youtube-dl

@Hrxn
Copy link

@Hrxn Hrxn commented Jul 1, 2018

Info should be included with --dump-json <playlist URL>..
You can then process with something like jq.

But I'm not sure if that approach is really practical, because ultimately download rates are limited by the server side, and if they detect any usual spikes in traffic this might end up in a temporary IP ban. You know, due to automated DDoS mitigation techniques etc.

I'm not saying that it can't work, just that it might not.

@fsiler
Copy link
Author

@fsiler fsiler commented Jul 1, 2018

In my case, I'm adding subtitles and doing other processing to downloaded videos, so it would be much faster in parallel because right now network sits idle quite a bit of the time.

This does indeed dump JSON, but fairly slowly- looks like maybe youtube-dl is doing deep introspection, which isn't required for my use case. I only need the youtube ID for this application.

Filtering through jq '.webpage_url_basename' gives me what I need to know- just wondered if there's a lighter-weight, faster version of this.

@dstftw
Copy link
Collaborator

@dstftw dstftw commented Jul 1, 2018

--flat-playlist.

@dstftw dstftw closed this Jul 1, 2018
@fsiler
Copy link
Author

@fsiler fsiler commented Jul 5, 2018

Thanks. Can confirm that

youtube-dl $PLAYLIST --dump-json --flat-playlist|jq -r ".id"|parallel -j2 --retry-failed youtube-dl -- :::

works very well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.