Why is youtube-dl so slow to get urls compared to alternatives?

[mattjis]

I'm using --skip-download and still it takes an average of 5 seconds to get urls (youtube, soundcloud etc..)

Here's the command I'm using:
time youtube-dl --ignore-config --ignore-errors --skip-download --geo-bypass --youtube-skip-dash-manifest --write-info-json --restrict-filenames --max-downloads 1 --format "worst" --output "%(autonumber)s" "https://www.youtube.com/watch?v=NF9aJoxqBxo"

Output:

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'--verbose', u'--ignore-config', u'--ignore-errors', u'--skip-download', u'--geo-bypass', u'--youtube-skip-dash-manifest', u'--write-info-json', u'--restrict-filenames', u'--max-downloads', u'1', u'--format', u'worst', u'--output', u'%(autonumber)s', u'https://www.youtube.com/watch?v=NF9aJoxqBxo']
-output', u'%(autonumber)s', u'https://www.youtube.com/watch?v=NF9aJoxqBxo']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2017.05.14
[debug] Python version 2.7.12 - Linux-4.4.0-78-generic-x86_64-with-Ubuntu-16.04-xenial
[debug] exe versions: ffmpeg 2.8.11-0ubuntu0.16.04.1, ffprobe 2.8.11-0ubuntu0.16.04.1
[debug] Proxy map: {}
[youtube] NF9aJoxqBxo: Downloading webpage
[youtube] NF9aJoxqBxo: Extracting video information
[info] Writing video description metadata as JSON to: 00001.info.json

real    0m4.832s
user    0m4.496s
sys     0m0.076s

After this line shows up: [youtube] NF9aJoxqBxo: Downloading webpage the rest happens almost instantly. What's happening before this point (nothing is shown in the console)?

I downloaded an alternative I found on github built specifically for PHP (https://github.com/Athlon1600/youtube-downloader) and it takes an average of 0.2 seconds to get the urls... (It doesn't get the info which is unfortunate but even manually doing that myself only adds an extra 0.2 seconds - could even do a multi curl and get it all done in the original 0.2 seconds).

5 seconds compared to 0.2 is a huge difference

Is there anything I can do to improve the speed or is this as good as it gets? :(

[yan12125]

Here's what happened on my machine:

$ time youtube-dl --ignore-config --ignore-errors --skip-download --geo-bypass --youtube-skip-dash-manifest --write-info-json --restrict-filenames --max-downloads 1 --format "worst" --output "%(autonumber)s" "https://www.youtube.com/watch?v=NF9aJoxqBxo"
[youtube] NF9aJoxqBxo: Downloading webpage
[youtube] NF9aJoxqBxo: Extracting video information
[info] Writing video description metadata as JSON to: 00001.info.json
youtube-dl --ignore-config --ignore-errors --skip-download --geo-bypass     1  0.56s user 0.01s system 57% cpu 0.996 total

There are two factors that affect running time:

1. Loading Python modules
2. Network status

From your description most likely 1. is the key factor. There are some ways to make it faster - use the experimental lazy extractor feature (#8497), or create an API server like https://youtube-dl-api-server.readthedocs.io/en/latest/

[mattjis]

@yan12125 Thanks for the response I do appreciate it! So I just tested out lazy extractor which made it about 1/3 faster which is nice, still not ideal though :(

What are the benefits of creating this API server? They don't seem to have an about on their page explaining why anyone would want to use it. Will installing it help me get my ideal: under 1 second timespan?

Also with the lazy extractor: will the usual youtube-dl -U keep it up to date or will I have to create something myself that detects new versions and performs those commands to make the updated youtube-dl

[yan12125]

From your descriptions I guess startup time is a primary factor. An API server needs to be started only once so the average running time can be reduced.

Also with the lazy extractor: will the usual youtube-dl -U keep it up to date or will I have to create something myself that detects new versions and performs those commands to make the updated youtube-dl

The lazy-extractor variant is still considered experimental and not included in official releases, so you'll need to repeat those steps for each new version.

[bryankhan]

try the -4 option to force ipv4

[yausername]

<edited by @yan12125: SPAM>