Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

youtube-dl hangs on large playlist even with --no-playlist #11979

Closed
dban0001 opened this issue Feb 5, 2017 · 1 comment
Closed

youtube-dl hangs on large playlist even with --no-playlist #11979

dban0001 opened this issue Feb 5, 2017 · 1 comment

Comments

@dban0001
Copy link

@dban0001 dban0001 commented Feb 5, 2017

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2017.02.04.1. If it's not read this FAQ entry and update. Issues with outdated version will be rejected.

  • I've verified and I assure that I'm running youtube-dl 2017.02.04.1

Before submitting an issue make sure you have:

  • At least skimmed through README and most notably FAQ and BUGS sections
  • Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

  • Bug report (encountered problems with youtube-dl)
  • Site support request (request for adding support for a new site)
  • Feature request (request for a new functionality)
  • Question
  • Other

I have a small program which calls youtube-dl. I always want users which use the program to download single files only, thus I use the --no-playlist flag. However, when somebody pastes a large file with multiple videos in it, youtube-dl begins to hang with 100% cpu usage. On the link below, this even takes multiple hours so you have to kill the process.
How can I prevent youtube-dl from doing this? Is there something like a timeout? Obviously, --socket-timeout won't work here.
Of course I could set a timeout in my program and then kill youtube-dl if it runs more than X seconds, but I first wanted to ask whether I can simply do this with youtube-dl.

youtube-dl "http://www.porntrex.com/maps/videositemap.xml" --no-playlist -v
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'http://www.porntrex.com/maps/videositemap.xml', u'--no-playlist', u'-v']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2017.02.04.1
[debug] Python version 2.7.12 - Linux-3.14.32-xxxx-grs-ipv6-64-x86_64-with-Ubuntu-16.04-xenial
[debug] exe versions: ffmpeg N-82999-g72d6101, ffprobe N-82999-g72d6101, rtmpdump 2.4
[debug] Proxy map: {}
[generic] videositemap: Requesting header
WARNING: Falling back on generic information extractor.
[generic] videositemap: Downloading webpage
# hanging now a bit
[generic] videositemap: Extracting information
# now begins to use 100% for hours.
@dstftw dstftw closed this in 2aec725 Feb 5, 2017
@dstftw
Copy link
Collaborator

@dstftw dstftw commented Feb 5, 2017

It's been a sub-optimal regular expression for parsing HTML5 embeds. Not it's fixed and takes about 25 seconds to match embeds with generic extractor in this 40 MB file.
--socket-timeout has nothing to do with data processing and there is no other timeout to limit processing time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.