Skip to content

/ youtube-dl

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[teamcoco] ERROR: An extractor error has occurred. #17099

Closed
jmraker opened this issue Jul 29, 2018 · 3 comments · May be fixed by #17101
Closed

[teamcoco] ERROR: An extractor error has occurred. #17099

jmraker opened this issue Jul 29, 2018 · 3 comments · May be fixed by #17101

Comments

@jmraker
Copy link

@jmraker jmraker commented Jul 29, 2018

Please follow the guide below

  • You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
  • Put an x into all the boxes [ ] relevant to your issue (like this: [x])
  • Use the Preview tab to see what your issue will actually look like

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2018.07.29. If it's not, read this FAQ entry and update. Issues with outdated version will be rejected.

  • I've verified and I assure that I'm running youtube-dl 2018.07.29

Before submitting an issue make sure you have:

  • At least skimmed through the README, most notably the FAQ and BUGS sections
  • Searched the bugtracker for similar issues including closed ones
  • Checked that provided video/audio/playlist URLs (if any) are alive and playable in a browser

What is the purpose of your issue?

  • Bug report (encountered problems with youtube-dl)
  • Site support request (request for adding support for a new site)
  • Feature request (request for a new functionality)
  • Question
  • Other

The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your issue

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:

Add the -v flag to your command line you run youtube-dl with (youtube-dl -v <your command line>), copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):

pi@www:~/www/teamcoco $ youtube-dl -v "http://teamcoco.com/comiccon/andy-finds-the-infinity-gauntlet?playlist=x;eyJ0eXBlIjoidGFnIiwiaWQiOjF9"
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'http://teamcoco.com/comiccon/andy-finds-the-infinity-gauntlet?playlist=x;eyJ0eXBlIjoidGFnIiwiaWQiOjF9']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2018.07.29
[debug] Python version 2.7.13 (CPython) - Linux-4.14.52-v7+-armv7l-with-debian-9.4
[debug] exe versions: none
[debug] Proxy map: {}
[Teamcoco] comiccon/andy-finds-the-infinity-gauntlet: Downloading JSON metadata
[Teamcoco] 104328: Downloading JSON metadata
ERROR: An extractor error has occurred. (caused by KeyError(u'findRecordVideoSource',)); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 502, in extract
    ie_result = self._real_extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/teamcoco.py", line 152, in _real_extract
    }''', 'RecordVideoSource', video_id) or {}
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/teamcoco.py", line 90, in _graphql_call
    }))['data'][find_object]
KeyError: u'findRecordVideoSource'
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 792, in extract_info
    ie_result = ie.extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 515, in extract
    raise ExtractorError('An extractor error has occurred.', cause=e)
ExtractorError: An extractor error has occurred. (caused by KeyError(u'findRecordVideoSource',)); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
@gfabiano gfabiano mentioned this issue Jul 29, 2018
Open
6 of 9 tasks complete
@jmraker
Copy link
Author

@jmraker jmraker commented Aug 7, 2018 •
edited

Since it's been 9 days and this missed the next release I tested the fix above on the same Raspberry Pi youtube-dl gave the verbose output that opened this. (it's downloaded many teamcoco videos beforehand)

It ultimately works, but with the fix it downloads a m3u8 file and requires either ffmpeg or avconv. After installing ffmpeg, ffmpeg downloads a video.ts file and an audio.ts file (where before it just downloaded a mp4 file)

It'll be ok as-is, but there's one thing that'll make the fix better. Maybe it's normal behavior, but when it executes ffmpeg, ffmpeg is unnecessarily verbose when youtube-dl isn't asked to be verbose. In the output below ffmpeg outputted the bulk of the output, that while good for verbose mode, could have been hidden.

pi@www:~/tmp/youtube-dl-master $ ./youtube-dl http://teamcoco.com/comiccon/andy-finds-the-infinity-gauntlet
[Teamcoco] comiccon/andy-finds-the-infinity-gauntlet: Downloading JSON metadata
[Teamcoco] 104328: Downloading JSON metadata
[Teamcoco] 104328: Downloading m3u8 information
[download] Destination: Andy Finds The Infinity Gauntlet-104328.f2336.mp4
ffmpeg version 3.2.10-1~deb9u1+rpt1 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 6.3.0 (Raspbian 6.3.0-18+rpi1) 20170516
  configuration: --prefix=/usr --extra-version='1~deb9u1+rpt1' --toolchain=hardened --libdir=/usr/lib/arm-linux-gnueabihf --incdir=/usr/include/arm-linux-gnueabihf --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libebur128 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx-rpi --enable-mmal --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared
  libavutil      55. 34.101 / 55. 34.101
  libavcodec     57. 64.101 / 57. 64.101
  libavformat    57. 56.101 / 57. 56.101
  libavdevice    57.  1.100 / 57.  1.100
  libavfilter     6. 65.100 /  6. 65.100
  libavresample   3.  1.  0 /  3.  1.  0
  libswscale      4.  2.100 /  4.  2.100
  libswresample   2.  3.100 /  2.  3.100
  libpostproc    54.  1.100 / 54.  1.100
[hls,applehttp @ 0x1c99740] Opening 'http://ak.storage-w.teamcococdn.com/cdn/2018-07/104328/bca7/hls/CONAN_072218_Comedy_5-2m.ts' for reading
Input #0, hls,applehttp, from 'http://ak.storage-w.teamcococdn.com/cdn/2018-07/104328/bca7/hls/CONAN_072218_Comedy_5-2m_v4.m3u8':
  Duration: 00:02:25.18, start: 1.466733, bitrate: 0 kb/s
  Program 0
    Metadata:
      variant_bitrate : 0
    Stream #0:0: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p(tv, bt709), 1024x576 [SAR 1:1 DAR 16:9], 29.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc
    Metadata:
      variant_bitrate : 0
Output #0, mp4, to 'file:Andy Finds The Infinity Gauntlet-104328.f2336.mp4.part':
  Metadata:
    encoder         : Lavf57.56.101
    Stream #0:0: Video: h264 (Main) ([33][0][0][0] / 0x0021), yuv420p(tv, bt709), 1024x576 [SAR 1:1 DAR 16:9], q=2-31, 29.97 fps, 29.97 tbr, 90k tbn, 90k tbc
    Metadata:
      variant_bitrate : 0
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
Press [q] to stop, [?] for help
[hls,applehttp @ 0x1c99740] Opening 'http://ak.storage-w.teamcococdn.com/cdn/2018-07/104328/bca7/hls/CONAN_072218_Comedy_5-2m.ts' for reading
[hls,applehttp @ 0x1c99740] Opening 'http://ak.storage-w.teamcococdn.com/cdn/2018-07/104328/bca7/hls/CONAN_072218_Comedy_5-2m.ts' for reading
frame= 4351 fps=191 q=-1.0 Lsize=   32967kB time=00:02:25.07 bitrate=1861.5kbits/s speed=6.37x
video:32974kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
[ffmpeg] Downloaded 33757781 bytes
[download] 100% of 32.19MiB in 00:25
[download] Destination: Andy Finds The Infinity Gauntlet-104328.f71.mp4
ffmpeg version 3.2.10-1~deb9u1+rpt1 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 6.3.0 (Raspbian 6.3.0-18+rpi1) 20170516
  configuration: --prefix=/usr --extra-version='1~deb9u1+rpt1' --toolchain=hardened --libdir=/usr/lib/arm-linux-gnueabihf --incdir=/usr/include/arm-linux-gnueabihf --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libebur128 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx-rpi --enable-mmal --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared
  libavutil      55. 34.101 / 55. 34.101
  libavcodec     57. 64.101 / 57. 64.101
  libavformat    57. 56.101 / 57. 56.101
  libavdevice    57.  1.100 / 57.  1.100
  libavfilter     6. 65.100 /  6. 65.100
  libavresample   3.  1.  0 /  3.  1.  0
  libswscale      4.  2.100 /  4.  2.100
  libswresample   2.  3.100 /  2.  3.100
  libpostproc    54.  1.100 / 54.  1.100
[hls,applehttp @ 0x12a6740] Opening 'http://ak.storage-w.teamcococdn.com/cdn/2018-07/104328/bca7/hls/CONAN_072218_Comedy_5-audio-64k.ts' for reading
Input #0, hls,applehttp, from 'http://ak.storage-w.teamcococdn.com/cdn/2018-07/104328/bca7/hls/CONAN_072218_Comedy_5-audio-64k_v4.m3u8':
  Duration: 00:02:25.28, start: 1.400000, bitrate: 0 kb/s
  Program 0
    Metadata:
      variant_bitrate : 0
    Stream #0:0: Audio: aac (HE-AAC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp
    Metadata:
      variant_bitrate : 0
Output #0, mp4, to 'file:Andy Finds The Infinity Gauntlet-104328.f71.mp4.part':
  Metadata:
    encoder         : Lavf57.56.101
    Stream #0:0: Audio: aac (HE-AAC) ([64][0][0][0] / 0x0040), 48000 Hz, stereo
    Metadata:
      variant_bitrate : 0
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
Press [q] to stop, [?] for help
[hls,applehttp @ 0x12a6740] Opening 'http://ak.storage-w.teamcococdn.com/cdn/2018-07/104328/bca7/hls/CONAN_072218_Comedy_5-audio-64k.ts' for reading
[hls,applehttp @ 0x12a6740] Opening 'http://ak.storage-w.teamcococdn.com/cdn/2018-07/104328/bca7/hls/CONAN_072218_Comedy_5-audio-64k.ts' for reading
size=    1126kB time=00:02:25.28 bitrate=  63.5kbits/s speed= 114x
video:0kB audio:1141kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
[ffmpeg] Downloaded 1153122 bytes
[download] 100% of 1.10MiB in 00:04
[ffmpeg] Merging formats into "Andy Finds The Infinity Gauntlet-104328.mp4"
Deleting original file Andy Finds The Infinity Gauntlet-104328.f2336.mp4 (pass -k to keep)
Deleting original file Andy Finds The Infinity Gauntlet-104328.f71.mp4 (pass -k to keep)
pi@www:~/tmp/youtube-dl-master $ ls -al
total 36100
drwxr-xr-x 8 pi pi     4096 Aug  7 20:39 .
drwxr-xr-x 3 pi pi     4096 Aug  7 20:19 ..
-rw-r--r-- 1 pi pi 34960097 Aug  7 20:39 Andy Finds The Infinity Gauntlet-104328.mp4
-rw-r--r-- 1 pi pi     3511 Aug  7 16:27 AUTHORS
drwxr-xr-x 2 pi pi     4096 Aug  7 16:27 bin
-rw-r--r-- 1 pi pi   121003 Aug  7 16:27 ChangeLog
-rw-r--r-- 1 pi pi    18685 Aug  7 20:31 CONTRIBUTING.md
drwxr-xr-x 3 pi pi     4096 Aug  7 16:27 devscripts
drwxr-xr-x 2 pi pi     4096 Aug  7 16:27 docs
drwxr-xr-x 2 pi pi     4096 Aug  7 16:27 .github
-rw-r--r-- 1 pi pi      521 Aug  7 16:27 .gitignore
-rw-r--r-- 1 pi pi     1211 Aug  7 16:27 LICENSE
-rw-r--r-- 1 pi pi     5464 Aug  7 16:27 Makefile
-rw-r--r-- 1 pi pi      219 Aug  7 16:27 MANIFEST.in
-rw-r--r-- 1 pi pi    88636 Aug  7 20:31 README.md
-rw-r--r-- 1 pi pi      225 Aug  7 16:27 setup.cfg
-rw-r--r-- 1 pi pi     4086 Aug  7 16:27 setup.py
drwxr-xr-x 4 pi pi     4096 Aug  7 16:27 test
-rw-r--r-- 1 pi pi      614 Aug  7 16:27 tox.ini
-rw-r--r-- 1 pi pi      616 Aug  7 16:27 .travis.yml
drwxr-xr-x 5 pi pi     4096 Aug  7 20:26 youtube_dl
-rwxr-xr-x 1 pi pi  1688143 Aug  7 20:31 youtube-dl
-rw-r--r-- 1 pi pi     1009 Aug  7 16:27 youtube-dl.plugin.zsh
@jmraker
Copy link
Author

@jmraker jmraker commented Sep 24, 2018

The site changed again and this unreleased fix is broken. Now the video urls are in a json response from http://teamcoco.com/_truman/d/104328?v=v1536871782669 where the 104328 number is a video id that is/was probably parsed. For the ?v=v1536871782669, it'll return the same json without the parameter. It's probably from http://teamcoco.com/_truman/f/clips-v1536871782669.html where the http://teamcoco.com/_truman/d/104328 is assembled from concatenated constants there. Not sure where v1536871782669 comes from.

{"meta":{"useFreewheel":false,"freewheelId":"tbs-node-104328","turnerMediaId":null,"turnerMediaAuthToken":"","duration":"0:02:25","poster":{"url":"http://cdn.teamcococdn.com/image/640x360,frame:1/1189-comedy-5-5b53f952bc599.png"},"src":{"hd":{"label":"hd","src":"https://ak-storage-west.teamcococdn.com/cdn/2018-07/104328/bca7/CONAN_072218_Comedy_5-high.mp4","type":"video/mp4"},"sd":{"label":"sd","src":"https://ak-storage-west.teamcococdn.com/cdn/2018-07/104328/bca7/CONAN_072218_Comedy_5-standard.mp4","type":"video/mp4"},"hls":{"label":"hls","src":"https://ak-storage-west.teamcococdn.com/cdn/2018-07/104328/bca7/main.m3u8","type":"application/x-mpegurl"},"uhd":{"label":"uhd","src":"https://ak-storage-west.teamcococdn.com/cdn/2018-07/104328/bca7/CONAN_072218_Comedy_5-1080-6mb.mp4","type":"video/mp4"},"low":{"label":"low","src":"https://ak-storage-west.teamcococdn.com/cdn/2018-07/104328/bca7/CONAN_072218_Comedy_5-500k.mp4","type":"video/mp4"}}}}
@scfc
Copy link
Contributor

@scfc scfc commented Nov 30, 2018

If I look at the source code of https://teamcoco.com/video/full-episode-japan, this would suggest that 104863 and v1543517376530 had some meaning. However, if I build the URL https://teamcoco.com/_truman/d/104863?v=v1543517376530, this just returns a URL for a thumbnail, but no links to videos, however turnerMediaId and turnerMediaAuthToken are set whereas in @jmraker's example they are not.

Just to point out the obvious: The initial:

ERROR: Unable to download JSON metadata: HTTP Error 307: Temporary Redirect (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

is caused by TeamcocoIE._graphql_call()'s 'http://teamcoco.com/graphql/' which must be 'https://teamcoco.com/graphql/' instead, but after changing this, it still fails with:

ERROR: An extractor error has occurred. (caused by KeyError(u'data',)); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
@remitamine remitamine closed this in 69e6efa Apr 5, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Linked pull requests

Successfully merging a pull request may close this issue.

[teamcoco] fix and improve extraction
2 participants
@jmraker @scfc
You can’t perform that action at this time.