Skip to content

/ youtube-dl

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Problems with AdultSwim #13306

Closed
keybounce opened this issue Jun 6, 2017 · 2 comments
Closed

Problems with AdultSwim #13306

keybounce opened this issue Jun 6, 2017 · 2 comments
Labels
geo-restricted tv-provider-account-needed

Comments

@keybounce
Copy link

@keybounce keybounce commented Jun 6, 2017

Please follow the guide below

  • You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
  • Put an x into all the boxes [ ] relevant to your issue (like that [x])
  • Use Preview tab to see how your issue will actually look like

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2017.06.05. If it's not read this FAQ entry and update. Issues with outdated version will be rejected.

  • [x ] I've verified and I assure that I'm running youtube-dl 2017.06.05

Before submitting an issue make sure you have:

  • [x ] At least skimmed through README and most notably FAQ and BUGS sections
  • [ x] Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

  • [ x] Bug report (encountered problems with youtube-dl)
  • [x ] Site support request (request for adding support for a new site)
  • [x ] Feature request (request for a new functionality)
  • Question
  • [x ] Other

So attempting to fetch from Adult Swim does not work. (Big surprise). Here's what I found when I tried, and some thoughts/ideas -- and, a feature request that comes out of this.

keybounceMBP:AdultSwim michael$ ./yt-dl -v http://www.adultswim.com/videos/dragon-ball-z-kai/stunning-youngsters-no-18s-uphill-battle/
[debug] System config: []
[debug] User config: ['-k', '-o', '%(title)s-%(timestamp)6i.%(ext)s', '-f', '\nbest[ext=mp4][height>431][height<=576]/\nbestvideo[ext=mp4][height=480]+bestaudio[ext=m4a]/\nbest[ext=mp4][height>340][height<=431]/\nbestvideo[ext=mp4][height>360][height<=576]+bestaudio/\nbest[height>340][height<=576]/\nbestvideo[height>360][height<=576]+bestaudio/\nbestvideo[height=360]+bestaudio/\nbest[ext=mp4][height>=280][height<=360]/\nbest[height<=576]/\nworst', '--ap-mso', 'Dish', '--ap-username', 'PRIVATE', '--ap-password', 'PRIVATE', '--write-sub', '--write-auto-sub', '--sub-lang', 'en,enUS,en-us', '--sub-format', 'ass/srt/best', '--convert-subs', 'ass', '--embed-subs', '--recode-video', 'mp4', '--mark-watched', '--download-archive', 'downloaded-videos.txt']
[debug] Custom config: []
[debug] Command-line args: ['-o', '%(series)s/s%(season_number)02d-e%(episode_number)02d-%(title)s.%(ext)s', '-v', 'http://www.adultswim.com/videos/dragon-ball-z-kai/stunning-youngsters-no-18s-uphill-battle/']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2017.06.05
[debug] Python version 3.6.1 - Darwin-13.4.0-x86_64-i386-64bit
[debug] exe versions: ffmpeg 3.2.4, ffprobe 3.2.4, rtmpdump 2.4
[debug] Proxy map: {}
[AdultSwim] stunning-youngsters-no-18s-uphill-battle: Downloading webpage
[AdultSwim] yg6tkv1KS3yUltXY4HS5zA: Downloading XML
[AdultSwim] 1400176: Retrieving Media Token
[AdultSwim] 1400176: Downloading XML
[AdultSwim] 1400176: Downloading m3u8 information
[AdultSwim] 1400176: Downloading m3u8 information
[debug] Invoking downloader on 'http://androidhls-secure.cdn.turner.com/adultswim/big/secure/tve/2017-05/ASOE1004271700075224_HLS/ASOE1004271700075224_Layer6_vod.m3u8'
[download] Destination: Dragon Ball Z Kai/s01-e118-Dragon Ball Z Kai - DBZ Kai 118.mp4
[debug] ffmpeg command line: ffmpeg -y -loglevel verbose -seekable 0 -headers 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding: gzip, deflate
Accept-Language: en-us,en;q=0.5
Cookie: hdnea2=expires=1496864785~access=/adultswim/big/secure/tve/2017-05/ASOE1004271700075224_HLS/*~md5=6166f27139b0a703f8fd454c80cf48de
' -i http://androidhls-secure.cdn.turner.com/adultswim/big/secure/tve/2017-05/ASOE1004271700075224_HLS/ASOE1004271700075224_Layer6_vod.m3u8 -c copy -f mp4 'file:Dragon Ball Z Kai/s01-e118-Dragon Ball Z Kai - DBZ Kai 118.mp4.part'
ffmpeg version 3.2.4 Copyright (c) 2000-2017 the FFmpeg developers
  built with Apple LLVM version 6.0 (clang-600.0.57) (based on LLVM 3.5svn)
  configuration: --prefix=/opt/local --enable-swscale --enable-avfilter --enable-avresample --enable-libmp3lame --enable-libvorbis --enable-libopus --enable-libtheora --enable-libschroedinger --enable-libopenjpeg --enable-libmodplug --enable-libvpx --enable-libsoxr --enable-libspeex --enable-libass --enable-libbluray --enable-lzma --enable-gnutls --enable-fontconfig --enable-libfreetype --enable-libfribidi --disable-libxcb --disable-libxcb-shm --disable-libxcb-xfixes --disable-indev=jack --disable-outdev=xv --enable-audiotoolbox --enable-sdl2 --mandir=/opt/local/share/man --enable-shared --enable-pthreads --cc=/usr/bin/clang --enable-vda --enable-videotoolbox --arch=x86_64 --enable-yasm --enable-libx265 --enable-gpl --enable-postproc --enable-libx264 --enable-libxvid
  libavutil      55. 34.101 / 55. 34.101
  libavcodec     57. 64.101 / 57. 64.101
  libavformat    57. 56.101 / 57. 56.101
  libavdevice    57.  1.100 / 57.  1.100
  libavfilter     6. 65.100 /  6. 65.100
  libavresample   3.  1.  0 /  3.  1.  0
  libswscale      4.  2.100 /  4.  2.100
  libswresample   2.  3.100 /  2.  3.100
  libpostproc    54.  1.100 / 54.  1.100
[hls,applehttp @ 0x7fcc42000000] HLS request for url 'http://androidhls-secure.cdn.turner.com/adultswim/big/secure/tve/2017-05/ASOE1004271700075224_HLS/ASOE1004271700075224_Layer6/8996_Period1/segment0.ts', offset 0, playlist 0
[h264 @ 0x7fcc4200d000] Reinit context to 848x480, pix_fmt: yuv420p
Input #0, hls,applehttp, from 'http://androidhls-secure.cdn.turner.com/adultswim/big/secure/tve/2017-05/ASOE1004271700075224_HLS/ASOE1004271700075224_Layer6_vod.m3u8':
  Duration: 00:21:45.98, start: 0.750000, bitrate: 0 kb/s
  Program 0 
    Metadata:
      variant_bitrate : 0
    Stream #0:0: Video: h264 (Constrained Baseline), 1 reference frame ([27][0][0][0] / 0x001B), yuv420p(tv, left), 848x480, Closed Captions, 29.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc
    Metadata:
      variant_bitrate : 0
    Stream #0:1: Audio: aac (HE-AAC) ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp
    Metadata:
      variant_bitrate : 0
Output #0, mp4, to 'file:Dragon Ball Z Kai/s01-e118-Dragon Ball Z Kai - DBZ Kai 118.mp4.part':
  Metadata:
    encoder         : Lavf57.56.101
    Stream #0:0: Video: h264 (Constrained Baseline), 1 reference frame ([33][0][0][0] / 0x0021), yuv420p(tv, left), 848x480 (0x0), q=2-31, 29.97 fps, 29.97 tbr, 90k tbn, 90k tbc
    Metadata:
      variant_bitrate : 0
    Stream #0:1: Audio: aac (HE-AAC) ([64][0][0][0] / 0x0040), 44100 Hz, stereo
    Metadata:
      variant_bitrate : 0
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
Automatically inserted bitstream filter 'aac_adtstoasc'; args=''
[hls,applehttp @ 0x7fcc42000000] HLS request for url 'http://androidhls-secure.cdn.turner.com/adultswim/big/secure/tve/2017-05/ASOE1004271700075224_HLS/ASOE1004271700075224_Layer6/8996_Period1/segment1.ts', offset 0, playlist 0
[hls,applehttp @ 0x7fcc42000000] HLS request for url 'http://androidhls-secure.cdn.turner.com/adultswim/big/secure/tve/2017-05/ASOE1004271700075224_HLS/ASOE1004271700075224_Layer6/8996_Period1/segment2.ts', offset 0, playlist 0
[hls,applehttp @ 0x7fcc42000000] HLS request for url 'http://androidhls-secure.cdn.turner.com/adultswim/big/secure/tve/2017-05/ASOE1004271700075224_HLS/ASOE1004271700075224_Layer6/8996_Period1/segment3.ts', offset 0, playlist 0
[hls,applehttp @ 0x7fcc42000000] HLS request for url 'http://androidhls-secure.cdn.turner.com/adultswim/big/secure/tve/2017-05/ASOE1004271700075224_HLS/ASOE1004271700075224_Layer6/8996_Period1/segment4.ts', offset 0, playlist 0
[hls,applehttp @ 0x7fcc42000000] HLS request for url 'http://androidhls-secure.cdn.turner.com/adultswim/big/secure/tve/2017-05/ASOE1004271700075224_HLS/ASOE1004271700075224_Layer6/8996_Period1/segment5.ts', offset 0, playlist 0
[hls,applehttp @ 0x7fcc42000000] HLS request for url 'http://androidhls-secure.cdn.turner.com/adultswim/big/secure/tve/2017-05/ASOE1004271700075224_HLS/ASOE1004271700075224_Layer6/8996_Period1/segment6.ts', offset 0, playlist 0
[hls,applehttp @ 0x7fcc42000000] HLS request for url 'http://androidhls-secure.cdn.turner.com/adultswim/big/secure/tve/2017-05/ASOE1004271700075224_HLS/ASOE1004271700075224_Layer6/8996_Period1/segment7.ts', offset 0, playlist 0
[hls,applehttp @ 0x7fcc42000000] HLS request for url 'http://androidhls-secure.cdn.turner.com/adultswim/big/secure/tve/2017-05/ASOE1004271700075224_HLS/ASOE1004271700075224_Layer6/8996_Period1/segment8.ts', offset 0, playlist 0
[hls,applehttp @ 0x7fcc42000000] HLS request for url 'http://androidhls-secure.cdn.turner.com/adultswim/big/secure/tve/2017-05/ASOE1004271700075224_HLS/ASOE1004271700075224_Layer6/8996_Period1/segment9.ts', offset 0, playlist 0
[hls,applehttp @ 0x7fcc42000000] HLS request for url 'http://androidhls-secure.cdn.turner.com/adultswim/big/secure/tve/2017-05/ASOE1004271700075224_HLS/ASOE1004271700075224_Layer6/8996_Period1/segment10.ts', offset 0, playlist 0
[hls,applehttp @ 0x7fcc42000000] HLS request for url 'http://androidhls-secure.cdn.turner.com/adultswim/big/secure/tve/2017-05/ASOE1004271700075224_HLS/ASOE1004271700075224_Layer6/8996_Period1/segment11.ts', offset 0, playlist 0
^C^C[crypto @ 0x7fcc43012fa0] Unable to open resource: http://androidhls-secure.cdn.turner.com/adultswim/big/secure/tve/2017-05/ASOE1004271700075224_HLS/ASOE1004271700075224_Layer6/8996_Period1/segment11.ts
av_interleaved_write_frame(): Immediate exit requested
Error writing trailer of file:Dragon Ball Z Kai/s01-e118-Dragon Ball Z Kai - DBZ Kai 118.mp4.part: Immediate exit requestedframe= 3296 fps=163 q=-1.0 Lsize=   25239kB time=00:01:50.01 bitrate=1879.4kbits/s speed=5.43x    
video:24448kB audio:877kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Input file #0 (http://androidhls-secure.cdn.turner.com/adultswim/big/secure/tve/2017-05/ASOE1004271700075224_HLS/ASOE1004271700075224_Layer6_vod.m3u8):
  Input stream #0:0 (video): 3296 packets read (25034783 bytes); 
  Input stream #0:1 (audio): 2370 packets read (897579 bytes); 
  Total: 5666 packets (25932362 bytes) demuxed
Output file #0 (file:Dragon Ball Z Kai/s01-e118-Dragon Ball Z Kai - DBZ Kai 118.mp4.part):
  Output stream #0:0 (video): 3296 packets muxed (25034783 bytes); 
  Output stream #0:1 (audio): 2370 packets muxed (897579 bytes); 
  Total: 5666 packets (25932362 bytes) muxed

ERROR: Interrupted by user
Exiting normally, received signal 2.
keybounceMBP:AdultSwim michael$

Now, the things to note:
First, it works for a while, and then fails.
Second, a big difference between this and watching in a browser: This is downloading much faster than watching in a browser.

Third, I decided to try to play around. I read up on the docs for ffmpeg, and found, among other things, both "seekable" and "-re".

Seekable is interesting -- you specify seekable of 0, so every time ffmpeg starts up, it has to start from the beginning. It never has a chance to say "lets resume from where we left off", which would allow the file to eventually complete.

-re is an ffmpeg option for, essentially, "real time" downloading -- instead of trying to fetch the file as fast as possible, about 8 times faster than watching speed, fetch at watching speed. Now, this will make things slow, but it will make things look like watching in a browser.

Fourth, and the feature request: I decided to try to plop that ffmpeg command line into a file, and try executing it. This was step 1 to trying to play around with the command line options.

What happened?

keybounceMBP:AdultSwim michael$ ./test
+ ffmpeg -y -loglevel verbose -seekable -1 -headers 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding: gzip, deflate
Accept-Language: en-us,en;q=0.5
Cookie: hdnea2=expires=1496864785~access=/adultswim/big/secure/tve/2017-05/ASOE1004271700075224_HLS/*~md5=6166f27139b0a703f8fd454c80cf48de
' -i http://androidhls-secure.cdn.turner.com/adultswim/big/secure/tve/2017-05/ASOE1004271700075224_HLS/ASOE1004271700075224_Layer6_vod.m3u8 -c copy -f mp4 'file:Dragon Ball Z Kai/s01-e118-Dragon Ball Z Kai - DBZ Kai 118.mp4.part'
ffmpeg version 3.2.4 Copyright (c) 2000-2017 the FFmpeg developers
  built with Apple LLVM version 6.0 (clang-600.0.57) (based on LLVM 3.5svn)
  configuration: --prefix=/opt/local --enable-swscale --enable-avfilter --enable-avresample --enable-libmp3lame --enable-libvorbis --enable-libopus --enable-libtheora --enable-libschroedinger --enable-libopenjpeg --enable-libmodplug --enable-libvpx --enable-libsoxr --enable-libspeex --enable-libass --enable-libbluray --enable-lzma --enable-gnutls --enable-fontconfig --enable-libfreetype --enable-libfribidi --disable-libxcb --disable-libxcb-shm --disable-libxcb-xfixes --disable-indev=jack --disable-outdev=xv --enable-audiotoolbox --enable-sdl2 --mandir=/opt/local/share/man --enable-shared --enable-pthreads --cc=/usr/bin/clang --enable-vda --enable-videotoolbox --arch=x86_64 --enable-yasm --enable-libx265 --enable-gpl --enable-postproc --enable-libx264 --enable-libxvid
  libavutil      55. 34.101 / 55. 34.101
  libavcodec     57. 64.101 / 57. 64.101
  libavformat    57. 56.101 / 57. 56.101
  libavdevice    57.  1.100 / 57.  1.100
  libavfilter     6. 65.100 /  6. 65.100
  libavresample   3.  1.  0 /  3.  1.  0
  libswscale      4.  2.100 /  4.  2.100
  libswresample   2.  3.100 /  2.  3.100
  libpostproc    54.  1.100 / 54.  1.100
[http @ 0x7fa112d2a4e0] HTTP error 403 Forbidden
http://androidhls-secure.cdn.turner.com/adultswim/big/secure/tve/2017-05/ASOE1004271700075224_HLS/ASOE1004271700075224_Layer6_vod.m3u8: Server returned 403 Forbidden (access denied)
keybounceMBP:AdultSwim michael$

So what happened? It looks like this --

Cookie: hdnea2=expires=1496864785~access=/adultswim/big/secure/tve/2017-05/ASOE1004271700075224_HLS/*~md5=6166f27139b0a703f8fd454c80cf48de

is a one-use cookie. Since you pass it to ffmpeg, and ffmpeg fails, I cannot reuse it.

So a feature request: Prepare an ffmpeg command, but instead of executing it, either output it, or put it into a file, so that the user can play around / alter it to test things out.
@dstftw
Copy link
Collaborator

@dstftw dstftw commented Jun 6, 2017

Dish MSO is not supported.
As per request: everything needed for ffmpeg command construction is in --dump-json.
@dstftw dstftw closed this Jun 6, 2017
@keybounce
Copy link
Author

@keybounce keybounce commented Jun 8, 2017

dump-json? Thank you, now to figure out how to use that.

Re: Dish MSO: Pull request #12000 works wonders and is ready for merging.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
geo-restricted tv-provider-account-needed
Projects
None yet
Milestone
No milestone
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
@keybounce @dstftw
You can’t perform that action at this time.