URL's generated with youtube-dl resulting in 403 error?

[Elite]

My php script to merge 1080p was working fine until today and used wget to download url's generated by youtube-dl and merged them (as youtube-dl does not delete source files after merge) , today this functionality is broken and both wget and curl fails to download the streams with 403 Forbidden error from Youtube.

Strangely, YTDL is able to download the files. I tried changing user-agent in curl as below but it still results in 403.

> User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/533.2 (KHTML, like Gecko) Chrome/5.0.342.3 Safari/533.2
> Host: r3---sn-aig7knek.googlevideo.com
> Accept: */*
>
* additional stuff not fine transfer.c:1037: 0 0
* HTTP 1.1 or later with persistent connection, pipelining supported
< HTTP/1.1 403 Forbidden
< Last-Modified: Wed, 02 May 2007 10:26:10 GMT
< Content-Type: text/plain
< Connection: close
< X-Content-Type-Options: nosniff
< Date: Sat, 01 Feb 2014 14:37:24 GMT
< Server: gvs 1.0
<
{ [data not shown]
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
* Closing connection #0

I would also like to request that youtube-dl deletes dash files after conversion (#1612 (comment)) which will specifically come handy in situations and usage scenario (scripted downloading on mass level) like this.

[jaimeMF]

Do all urls fail, or only some? are you requesting dash formats or the normal formats? . Could you post how you are calling youtube-dl and curl?
I have tested with this script and it works fine:

#!/bin/sh

URL="BaW_jenozKc"
URL="UxxajLWwzqY"

VID=$(youtube-dl -g $URL)

#curl -o final.mp4 "$VID"
wget -O final.mp4 "$VID"

[Elite]

This is the complete PHP test script which extracts the DASH stream URL's and prints them on browser which I copy paste in terminal to test what is failing:

<?php
ini_set('display_errors', TRUE);
ini_set('display_startup_errors', TRUE);
apc_clear_cache();

$id = 'bkx9kCdaaMg';
$video = exec('youtube-dl -g -f 264/137 http://www.youtube.com/watch?v=' . $id);
$audio = exec('youtube-dl -g -f 141/172/140/171/139 http://www.youtube.com/watch?v=' . $id);


echo('wget -O test.audio ' . '"' . $audio . '"');
echo '<br/><br/>';
echo('wget -O test.video ' . '"' . $video . '"');
echo '<br/><br/>';

?>

The above outputs this:

wget -O test.audio "http://r3---sn-aig7knek.googlevideo.com/videoplayback?gir=yes&ip=xx.xxx.xx.xxx&key=yt5&fexp=924614%2C902543%2C924616%2C936910%2C936913&mv=m&burst=40&sparams=algorithm%2Cburst%2Cclen%2Cdur%2Cfactor%2Cgcr%2Cgir%2Cid%2Cip%2Cipbits%2Citag%2Clmt%2Csource%2Cupn%2Cexpire&ms=au&sver=3&clen=3190667&id=6e4c7d90275a68c8&dur=200.852&gcr=gb&factor=1.25&lmt=1390856987411284&algorithm=throttle-factor&itag=140&source=youtube&mt=1391318958&ipbits=0&upn=ZFAiW4V55-8&expire=1391343999&signature=E203F6C457D85FBAE46F9A5EB4BFB02F885.F586D9A16A1AE6080468341BEE168BDACAEC0DADADA&ratebypass=yes"

wget -O test.video "https://r3---sn-aig7knek.googlevideo.com/videoplayback?source=youtube&expire=1391343999&sver=3&dur=200.793&gir=yes&mt=1391318958&ms=au&id=6e4c7d90275a68c8&key=yt5&clen=102968468&lmt=1390856962390962&factor=1.25&itag=137&gcr=gb&pcm2fr=yes&upn=yQPjpRnHmyI&burst=40&fexp=935637%2C909715%2C933209%2C932273%2C914083%2C942400%2C936910%2C936913&mv=m&sparams=algorithm%2Cburst%2Cclen%2Cdur%2Cfactor%2Cgcr%2Cgir%2Cid%2Cip%2Cipbits%2Citag%2Clmt%2Cpcm2fr%2Csource%2Cupn%2Cexpire&ipbits=0&ip=xx.xxx.xx.xxx&algorithm=throttle-factor&signature=B7DA4D646B094BDEF46F337FCE2970BCDF3.676252BBCCAB94B9DA93E1BAB6A5E59FCB848EF5F5F&ratebypass=yes"

Which results in the below 403 on terminal -

root@mybox:~# wget -O test.audio "http://r3---sn-aig7knek.googlevideo.com/videoplayback?gir=yes&ip=xx.xxx.xxx.xxx&key=yt5&fexp=924614%2C902543%2C924616%2C936910%2C936913&mv=m&burst=40&sparams=algorithm%2Cburst%2Cclen%2Cdur%2Cfactor%2Cgcr%2Cgir%2Cid%2Cip%2Cipbits%2Citag%2Clmt%2Csource%2Cupn%2Cexpire&ms=au&sver=3&clen=3190667&id=6e4c7d90275a68c8&dur=200.852&gcr=gb&factor=1.25&lmt=1390856987411284&algorithm=throttle-factor&itag=140&source=youtube&mt=1391318958&ipbits=0&upn=ZFAiW4V55-8&expire=1391343999&signature=E203F6C457D85FBAE46F9A5EB4BFB02F885.F586D9A16A1AE6080468341BEE168BDACAEC0DADADA&ratebypass=yes"
--2014-02-02 05:30:56--  http://r3---sn-aig7knek.googlevideo.com/videoplayback?gir=yes&ip=xx.xxx.xxx.xxx&key=yt5&fexp=924614%2C902543%2C924616%2C936910%2C936913&mv=m&burst=40&sparams=algorithm%2Cburst%2Cclen%2Cdur%2Cfactor%2Cgcr%2Cgir%2Cid%2Cip%2Cipbits%2Citag%2Clmt%2Csource%2Cupn%2Cexpire&ms=au&sver=3&clen=3190667&id=6e4c7d90275a68c8&dur=200.852&gcr=gb&factor=1.25&lmt=1390856987411284&algorithm=throttle-factor&itag=140&source=youtube&mt=1391318958&ipbits=0&upn=ZFAiW4V55-8&expire=1391343999&signature=E203F6C457D85FBAE46F9A5EB4BFB02F885.F586D9A16A1AE6080468341BEE168BDACAEC0DADADA&ratebypass=yes
Resolving r3---sn-aig7knek.googlevideo.com (r3---sn-aig7knek.googlevideo.com)... 74.125.105.8, 2a00:1450:4009:14::8
Connecting to r3---sn-aig7knek.googlevideo.com (r3---sn-aig7knek.googlevideo.com)|74.125.105.8|:80... connected.
HTTP request sent, awaiting response... 403 Forbidden
2014-02-02 05:30:56 ERROR 403: Forbidden.

[ghost]

You must use youtube-dl and wget on the same machine... otherwise it won't work.

[Elite]

@pulpe They are on the same machine and the script was working fine but stopped working suddenly. After hours of testing I finally found the problem and the solution which worked was to clear the youtube-dl cache folder.

@jaimeMF I am not sure what caused the issue but clearing the cache fixed the problem.

[kieranoreilly]

Hi I am having the same problem. Did you end up getting a fix?

[Elite]

After hours of testing I finally found the problem and the solution which worked was to clear the youtube-dl cache folder.

[DrupaListo-com]

my testing shows using axel as external downloader gives that 403 error while using aria2c works OK!
There is sth wrong with axel or the youtube-dl glue code for axel...

[yan12125]

axel is broken with YouTube, see #10429 (comment)