Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

YouTube: -j --flat-playlist no longer gets titles #6699

Closed
alphapapa opened this issue Aug 28, 2015 · 6 comments
Closed

YouTube: -j --flat-playlist no longer gets titles #6699

alphapapa opened this issue Aug 28, 2015 · 6 comments
Labels

Comments

@alphapapa
Copy link
Contributor

@alphapapa alphapapa commented Aug 28, 2015

$ youtube-dl --verbose -j --flat-playlist https://www.youtube.com/user/rhettandlink2
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'--verbose', u'-j', u'--flat-playlist', u'https://www.youtube.com/user/rhettandlink2']
[debug] Encodings: locale UTF-8, fs UTF-8, out None, pref UTF-8
[debug] youtube-dl version 2015.08.28
[debug] Python version 2.7.6 - Linux-3.16.0-46-generic-x86_64-with-Ubuntu-14.04-trusty
[debug] exe versions: avconv 11-6, avprobe 11-6, rtmpdump 2.4
[debug] Proxy map: {}
{"url": "GE6ipRyomnA", "_type": "url", "ie_key": "Youtube", "id": "GE6ipRyomnA"}
{"url": "fx3jLYHP8oE", "_type": "url", "ie_key": "Youtube", "id": "fx3jLYHP8oE"}
...

Earlier this year I reported a similar issue, #4971. I guess YouTube has changed something on their end again. Hopefully it's still possible to get the video titles without fetching each video's page. It seems like it should be possible, because if I leave off -j, I can see it fetching the playlist pages, and those include the video titles (at least, the HTML pages do), but I guess youtube-dl just isn't parsing them out.

If I leave off --flat-playlist, I can get JSON with the fulltitle element, but the JSON also includes everything, while all I want is the ID and title.

Thanks for your help.

@alphapapa
Copy link
Contributor Author

@alphapapa alphapapa commented Aug 28, 2015

I tried to debug this myself, but I got lost. I'm far from a Python expert, but I think it's harder than it ought to be. I documented my attempt in issue #6701.

@jaimeMF jaimeMF added the request label Aug 29, 2015
@alphapapa
Copy link
Contributor Author

@alphapapa alphapapa commented Sep 8, 2015

To try to fix my script, I tried to get youtube-dl to just output a list of video IDs for a playlist. First I updated to 2015.09.03. But now it seems that it's impossible to get a plain list of video IDs without youtube-dl making 3 GET requests for every video in the playlist:

$ youtube-dl --print-traffic --verbose -o "%(id)s" -j https://www.youtube.com/user/rhettandlink2 | grep GET
send: u'GET /user/rhettandlink2/videos?view=57 HTTP/1.1\r\nAccept-Language: en-us,en;q=0.5\r\nAccept-Encoding: gzip, deflate\r\nConnection: close\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/20.0 (Chrome)\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nHost: www.youtube.com\r\n\r\n'
send: u'GET /playlist?list=UU4PooiX37Pld1T8J5SYT-SQ HTTP/1.1\r\nAccept-Language: en-us,en;q=0.5\r\nAccept-Encoding: gzip, deflate\r\nConnection: close\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/20.0 (Chrome)\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nHost: www.youtube.com\r\nCookie: YSC=IGlhTGGwZjk; VISITOR_INFO1_LIVE=n_63LNnPOjQ\r\n\r\n'
send: u'GET /browse_ajax?action_continuation=1&continuation=4qmFsgIkEhpWTFVVNFBvb2lYMzdQbGQxVDhKNVNZVC1TURoGQ0dVJTNE HTTP/1.1\r\nAccept-Language: en-us,en;q=0.5\r\nAccept-Encoding: gzip, deflate\r\nConnection: close\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/20.0 (Chrome)\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nHost: youtube.com\r\nCookie: YSC=IGlhTGGwZjk; VISITOR_INFO1_LIVE=n_63LNnPOjQ\r\n\r\n'
send: u'GET /browse_ajax?action_continuation=1&continuation=4qmFsgIkEhpWTFVVNFBvb2lYMzdQbGQxVDhKNVNZVC1TURoGQ0dVJTNE HTTP/1.1\r\nAccept-Language: en-us,en;q=0.5\r\nAccept-Encoding: gzip, deflate\r\nConnection: close\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/20.0 (Chrome)\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nHost: www.youtube.com\r\nCookie: YSC=IGlhTGGwZjk; VISITOR_INFO1_LIVE=n_63LNnPOjQ\r\n\r\n'
send: u'GET /browse_ajax?action_continuation=1&continuation=4qmFsgIiEhpWTFVVNFBvb2lYMzdQbGQxVDhKNVNZVC1TURoEQ01rQg%253D%253D HTTP/1.1\r\nAccept-Language: en-us,en;q=0.5\r\nAccept-Encoding: gzip, deflate\r\nConnection: close\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/20.0 (Chrome)\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nHost: youtube.com\r\nCookie: YSC=IGlhTGGwZjk; VISITOR_INFO1_LIVE=n_63LNnPOjQ\r\n\r\n'
send: u'GET /browse_ajax?action_continuation=1&continuation=4qmFsgIiEhpWTFVVNFBvb2lYMzdQbGQxVDhKNVNZVC1TURoEQ01rQg%253D%253D HTTP/1.1\r\nAccept-Language: en-us,en;q=0.5\r\nAccept-Encoding: gzip, deflate\r\nConnection: close\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/20.0 (Chrome)\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nHost: www.youtube.com\r\nCookie: YSC=IGlhTGGwZjk; VISITOR_INFO1_LIVE=n_63LNnPOjQ\r\n\r\n'
send: u'GET /browse_ajax?action_continuation=1&continuation=4qmFsgIiEhpWTFVVNFBvb2lYMzdQbGQxVDhKNVNZVC1TURoEQ0swQw%253D%253D HTTP/1.1\r\nAccept-Language: en-us,en;q=0.5\r\nAccept-Encoding: gzip, deflate\r\nConnection: close\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/20.0 (Chrome)\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nHost: youtube.com\r\nCookie: YSC=IGlhTGGwZjk; VISITOR_INFO1_LIVE=n_63LNnPOjQ\r\n\r\n'
send: u'GET /browse_ajax?action_continuation=1&continuation=4qmFsgIiEhpWTFVVNFBvb2lYMzdQbGQxVDhKNVNZVC1TURoEQ0swQw%253D%253D HTTP/1.1\r\nAccept-Language: en-us,en;q=0.5\r\nAccept-Encoding: gzip, deflate\r\nConnection: close\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/20.0 (Chrome)\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nHost: www.youtube.com\r\nCookie: YSC=IGlhTGGwZjk; VISITOR_INFO1_LIVE=n_63LNnPOjQ\r\n\r\n'
send: u'GET /browse_ajax?action_continuation=1&continuation=4qmFsgIiEhpWTFVVNFBvb2lYMzdQbGQxVDhKNVNZVC1TURoEQ0pFRA%253D%253D HTTP/1.1\r\nAccept-Language: en-us,en;q=0.5\r\nAccept-Encoding: gzip, deflate\r\nConnection: close\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/20.0 (Chrome)\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nHost: youtube.com\r\nCookie: YSC=IGlhTGGwZjk; VISITOR_INFO1_LIVE=n_63LNnPOjQ\r\n\r\n'
send: u'GET /browse_ajax?action_continuation=1&continuation=4qmFsgIiEhpWTFVVNFBvb2lYMzdQbGQxVDhKNVNZVC1TURoEQ0pFRA%253D%253D HTTP/1.1\r\nAccept-Language: en-us,en;q=0.5\r\nAccept-Encoding: gzip, deflate\r\nConnection: close\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/20.0 (Chrome)\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nHost: www.youtube.com\r\nCookie: YSC=IGlhTGGwZjk; VISITOR_INFO1_LIVE=n_63LNnPOjQ\r\n\r\n'
send: u'GET /browse_ajax?action_continuation=1&continuation=4qmFsgIiEhpWTFVVNFBvb2lYMzdQbGQxVDhKNVNZVC1TURoEQ1BVRA%253D%253D HTTP/1.1\r\nAccept-Language: en-us,en;q=0.5\r\nAccept-Encoding: gzip, deflate\r\nConnection: close\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/20.0 (Chrome)\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nHost: youtube.com\r\nCookie: YSC=IGlhTGGwZjk; VISITOR_INFO1_LIVE=n_63LNnPOjQ\r\n\r\n'
send: u'GET /browse_ajax?action_continuation=1&continuation=4qmFsgIiEhpWTFVVNFBvb2lYMzdQbGQxVDhKNVNZVC1TURoEQ1BVRA%253D%253D HTTP/1.1\r\nAccept-Language: en-us,en;q=0.5\r\nAccept-Encoding: gzip, deflate\r\nConnection: close\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/20.0 (Chrome)\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nHost: www.youtube.com\r\nCookie: YSC=IGlhTGGwZjk; VISITOR_INFO1_LIVE=n_63LNnPOjQ\r\n\r\n'
send: u'GET /browse_ajax?action_continuation=1&continuation=4qmFsgIiEhpWTFVVNFBvb2lYMzdQbGQxVDhKNVNZVC1TURoEQ05rRQ%253D%253D HTTP/1.1\r\nAccept-Language: en-us,en;q=0.5\r\nAccept-Encoding: gzip, deflate\r\nConnection: close\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/20.0 (Chrome)\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nHost: youtube.com\r\nCookie: YSC=IGlhTGGwZjk; VISITOR_INFO1_LIVE=n_63LNnPOjQ\r\n\r\n'
send: u'GET /browse_ajax?action_continuation=1&continuation=4qmFsgIiEhpWTFVVNFBvb2lYMzdQbGQxVDhKNVNZVC1TURoEQ05rRQ%253D%253D HTTP/1.1\r\nAccept-Language: en-us,en;q=0.5\r\nAccept-Encoding: gzip, deflate\r\nConnection: close\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/20.0 (Chrome)\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nHost: www.youtube.com\r\nCookie: YSC=IGlhTGGwZjk; VISITOR_INFO1_LIVE=n_63LNnPOjQ\r\n\r\n'
send: u'GET /browse_ajax?action_continuation=1&continuation=4qmFsgIiEhpWTFVVNFBvb2lYMzdQbGQxVDhKNVNZVC1TURoEQ0wwRg%253D%253D HTTP/1.1\r\nAccept-Language: en-us,en;q=0.5\r\nAccept-Encoding: gzip, deflate\r\nConnection: close\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/20.0 (Chrome)\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nHost: youtube.com\r\nCookie: YSC=IGlhTGGwZjk; VISITOR_INFO1_LIVE=n_63LNnPOjQ\r\n\r\n'
send: u'GET /browse_ajax?action_continuation=1&continuation=4qmFsgIiEhpWTFVVNFBvb2lYMzdQbGQxVDhKNVNZVC1TURoEQ0wwRg%253D%253D HTTP/1.1\r\nAccept-Language: en-us,en;q=0.5\r\nAccept-Encoding: gzip, deflate\r\nConnection: close\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/20.0 (Chrome)\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nHost: www.youtube.com\r\nCookie: YSC=IGlhTGGwZjk; VISITOR_INFO1_LIVE=n_63LNnPOjQ\r\n\r\n'
send: u'GET /browse_ajax?action_continuation=1&continuation=4qmFsgIiEhpWTFVVNFBvb2lYMzdQbGQxVDhKNVNZVC1TURoEQ0tFRw%253D%253D HTTP/1.1\r\nAccept-Language: en-us,en;q=0.5\r\nAccept-Encoding: gzip, deflate\r\nConnection: close\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/20.0 (Chrome)\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nHost: youtube.com\r\nCookie: YSC=IGlhTGGwZjk; VISITOR_INFO1_LIVE=n_63LNnPOjQ\r\n\r\n'
send: u'GET /browse_ajax?action_continuation=1&continuation=4qmFsgIiEhpWTFVVNFBvb2lYMzdQbGQxVDhKNVNZVC1TURoEQ0tFRw%253D%253D HTTP/1.1\r\nAccept-Language: en-us,en;q=0.5\r\nAccept-Encoding: gzip, deflate\r\nConnection: close\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/20.0 (Chrome)\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nHost: www.youtube.com\r\nCookie: YSC=IGlhTGGwZjk; VISITOR_INFO1_LIVE=n_63LNnPOjQ\r\n\r\n'
send: u'GET /browse_ajax?action_continuation=1&continuation=4qmFsgIiEhpWTFVVNFBvb2lYMzdQbGQxVDhKNVNZVC1TURoEQ0lVSA%253D%253D HTTP/1.1\r\nAccept-Language: en-us,en;q=0.5\r\nAccept-Encoding: gzip, deflate\r\nConnection: close\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/20.0 (Chrome)\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nHost: youtube.com\r\nCookie: YSC=IGlhTGGwZjk; VISITOR_INFO1_LIVE=n_63LNnPOjQ\r\n\r\n'
send: u'GET /browse_ajax?action_continuation=1&continuation=4qmFsgIiEhpWTFVVNFBvb2lYMzdQbGQxVDhKNVNZVC1TURoEQ0lVSA%253D%253D HTTP/1.1\r\nAccept-Language: en-us,en;q=0.5\r\nAccept-Encoding: gzip, deflate\r\nConnection: close\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/20.0 (Chrome)\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nHost: www.youtube.com\r\nCookie: YSC=IGlhTGGwZjk; VISITOR_INFO1_LIVE=n_63LNnPOjQ\r\n\r\n'
send: u'GET /watch?v=VTPz9z7M6B8&gl=US&hl=en&has_verified=1&bpctr=9999999999 HTTP/1.1\r\nAccept-Language: en-us,en;q=0.5\r\nAccept-Encoding: gzip, deflate\r\nConnection: close\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/20.0 (Chrome)\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nHost: www.youtube.com\r\nCookie: YSC=IGlhTGGwZjk; VISITOR_INFO1_LIVE=n_63LNnPOjQ; PREF=f1=50000000&hl=en\r\n\r\n'
send: u'GET /get_video_info?&video_id=VTPz9z7M6B8&el=info&ps=default&eurl=&gl=US&hl=en HTTP/1.1\r\nAccept-Language: en-us,en;q=0.5\r\nAccept-Encoding: gzip, deflate\r\nConnection: close\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/20.0 (Chrome)\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nHost: www.youtube.com\r\nCookie: YSC=IGlhTGGwZjk; VISITOR_INFO1_LIVE=n_63LNnPOjQ; PREF=f1=50000000&hl=en; s_gl=1d69aac621b2f9c0a25dade722d6e24bcwIAAABVUw==\r\n\r\n'
send: u'GET /api/manifest/dash/expire/1441720680/itag/0/fexp/9407051%2C9408255%2C9408710%2C9409069%2C9415365%2C9415436%2C9415485%2C9416023%2C9416074%2C9416126%2C9416729%2C9417380%2C9417707%2C9417838%2C9417877%2C9418153%2C9418201%2C9418214%2C9418448%2C9419444%2C9420020%2C9420054%2C9420348%2C9420433/sparams/as%2Cid%2Cip%2Cipbits%2Citag%2Cmm%2Cmn%2Cms%2Cmv%2Cnh%2Cpl%2Cplayback_host%2Crequiressl%2Csource%2Cexpire/source/youtube/sver/3/requiressl/yes/signature/09F7998DC47D817522E2EAFDFC6EEFB8F61CDB6E.E5F5428B3241A054B4E11D292777E01CCC59E838/pl/23/nh/IgpwcjA0LmRmdzA2KgkxMjcuMC4wLjE/as/fmp4_audio_clear%2Cwebm_audio_clear%2Cfmp4_sd_hd_clear%2Cwebm_sd_hd_clear%2Cwebm2_sd_hd_clear/upn/MXmRugsYLaw/id/o-ADr-jsGFh8OvOHbxtzx3dlBlfhnFMpV0nMRdaoGVOV25/playback_host/r3---sn-q4fl6n7d.googlevideo.com/ipbits/0/key/yt5/ip/172.0.42.27/ms/au/mv/m/mt/1441698971/mn/sn-q4fl6n7d/mm/31 HTTP/1.1\r\nAccept-Language: en-us,en;q=0.5\r\nAccept-Encoding: gzip, deflate\r\nConnection: close\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/20.0 (Chrome)\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nHost: manifest.googlevideo.com\r\n\r\n'
send: u'GET /watch?v=h6O7POOq9CQ&gl=US&hl=en&has_verified=1&bpctr=9999999999 HTTP/1.1\r\nAccept-Language: en-us,en;q=0.5\r\nAccept-Encoding: gzip, deflate\r\nConnection: close\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/20.0 (Chrome)\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nHost: www.youtube.com\r\nCookie: YSC=IGlhTGGwZjk; VISITOR_INFO1_LIVE=n_63LNnPOjQ; PREF=f1=50000000&hl=en; s_gl=1d69aac621b2f9c0a25dade722d6e24bcwIAAABVUw==\r\n\r\n'
send: u'GET /get_video_info?&video_id=h6O7POOq9CQ&el=info&ps=default&eurl=&gl=US&hl=en HTTP/1.1\r\nAccept-Language: en-us,en;q=0.5\r\nAccept-Encoding: gzip, deflate\r\nConnection: close\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/20.0 (Chrome)\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nHost: www.youtube.com\r\nCookie: YSC=IGlhTGGwZjk; VISITOR_INFO1_LIVE=n_63LNnPOjQ; PREF=f1=50000000&hl=en; s_gl=1d69aac621b2f9c0a25dade722d6e24bcwIAAABVUw==\r\n\r\n'
send: u'GET /api/manifest/dash/sparams/as%2Cid%2Cip%2Cipbits%2Citag%2Cmm%2Cmn%2Cms%2Cmv%2Cnh%2Cpl%2Cplayback_host%2Crequiressl%2Csource%2Cexpire/as/fmp4_audio_clear%2Cwebm_audio_clear%2Cfmp4_sd_hd_clear%2Cwebm_sd_hd_clear%2Cwebm2_sd_hd_clear/ipbits/0/playback_host/r3---sn-q4f7sne7.googlevideo.com/requiressl/yes/source/youtube/key/yt5/signature/1826064D8C4DA4DEF5E17D6376CC81DA0AF4CEC1.6B4DA003B8D0F041EEB586EEE6F75F9AFFE6AA96/fexp/9407051%2C9408255%2C9408710%2C9409069%2C9415365%2C9415436%2C9415485%2C9416023%2C9416074%2C9416126%2C9416729%2C9417380%2C9417707%2C9417838%2C9417877%2C9418153%2C9418201%2C9418214%2C9418448%2C9419444%2C9420020%2C9420054%2C9420348%2C9420433/nh/IgpwcjAxLmRmdzA2KgkxMjcuMC4wLjE/sver/3/expire/1441720681/upn/U4OCAKm4BW8/itag/0/pl/23/ip/172.0.42.27/ms/au/mv/m/mt/1441698971/id/o-AEtkhTxMqaPFJ24flyoZyq8XT9FEQUZph7CIwRxhbpQ7/mn/sn-q4f7sne7/mm/31 HTTP/1.1\r\nAccept-Language: en-us,en;q=0.5\r\nAccept-Encoding: gzip, deflate\r\nConnection: close\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/20.0 (Chrome)\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nHost: manifest.googlevideo.com\r\n\r\n'
...

I know that youtube-dl already has all of the video IDs, but it won't just print them out. This used to work earlier this year, so this appears to be a regression (caused by YouTube, of course).

Thanks.

@alphapapa
Copy link
Contributor Author

@alphapapa alphapapa commented Sep 27, 2015

By the way, this is a regression, so I'm not sure the "request" tag is accurate...

@dstftw dstftw closed this in 648e6a1 Oct 17, 2015
@alphapapa
Copy link
Contributor Author

@alphapapa alphapapa commented Oct 17, 2015

@dstftw You might find the test I wrote useful, because it tests that the titles that are found match the known titles for certain videos.

BTW, I appreciate your work on this project, and I know it's a very busy project with lots of people asking for fixes and submitting patches. Having said that, your "bedside manner" could use some work.

Here was a regression that I reported twice, and after no response for a month on the second report, I fixed it myself, spending several hours figuring out how to work with the existing code. Then you abruptly instructed me to make certain changes. Being your project, I made them promptly, as requested. And I even wrote a comprehensive test to ensure that the regression wouldn't happen again.

Then, three weeks passed with no response from any developers, the pull request waiting. Hey, people have lives, this is all volunteer work, that's fine. I'm busy too, that's why I didn't get around to working on it for a month.

But then, with no comment, you merge your own code, completely disregarding all of my work.

I understand that you wanted to refactor the code a bit, and that's fine. But it's rude to disregard someone's work in this way. The test I wrote was more comprehensive than the one you merged, yet you ignored it. And I even added missing docstrings and comments to make the code cleaner and easier to work with, but all of that was for naught.

Well, I can handle being mildly snubbed. No big deal. The time wasn't entirely wasted, because I fixed the bug, which allowed my personal project to work again, and I learned some new things. And we're all human.

But being treated this way doesn't encourage me to spend more time contributing in the future. And I'm just one random internet person. If this is a pattern, if this is how potential contributors are typically treated here, then it's a shame, because it's surely discouraging people who could help the project.

Anyway, just some friendly advice from someone who's enjoyed using this project for several years. Thanks.

@dstftw
Copy link
Collaborator

@dstftw dstftw commented Oct 17, 2015

I appreciate your efforts on fixing this issue and sorry for not commenting here - I'm working on several issues in row/parallel and don't always remember to comment everywhere. Here is rationale why I wasn't satisfied with solution provided.
The code you've written was already implemented (I've figured it out just today) in youtube extractor, so to avoid code duplication I've just adapted and generalized it. Of course, I could point it out here and wait for you to fix it, then to fix something else and so on, but after I've already touched it myself it's much quicker and easier for me to just fix it myself right now and not to delay it for another couple of weeks.
The test you've written does not test the actual problem - missing titles, but tests for particular videos' titles to be in the playlist. If some videos eventually go off the playlist the test will fail but this does not mean that titles are not extracted anymore. In the first place test should test the issue that was fixed, i.e. missing titles.
In order not to feel disregarded under what name would you like to be credited?

@alphapapa
Copy link
Contributor Author

@alphapapa alphapapa commented Oct 18, 2015

Thanks for your response. I hope I wasn't rude in what I said.

I do understand that it was easier for you to do it than for me. Hey, if you have the time, I'm happy for you to do it yourself. I figured, as busy as you are, I would try to help reduce your workload. :)

I do realize that if the playlists that are in the test change, the test will no longer pass. I chose a couple of videos that shouldn't be disappearing anytime soon, as they're part of a major channel that gets millions of hits every day. I thought it would be important to test that the titles were extracted correctly, matching known video titles, rather than simply testing that something was extracted. But you're the expert. :)

I appreciate your offer, but it's not necessary to add me to the credits. I just want to see the project continue to do well. I know you get lots of bug reports and complaints, which tends to be a thankless job, but I'm glad you were able to get the time to fix it properly.

Keep up the good work! :D

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.