Skip to content

/ youtube-dl

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ADN] Unable to extract player config #20549

Closed
ghost opened this issue Apr 3, 2019 · 15 comments
Closed

[ADN] Unable to extract player config #20549

ghost opened this issue Apr 3, 2019 · 15 comments

Comments

@ghost
Copy link

@ghost ghost commented Apr 3, 2019

Please follow the guide below

  • You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
  • Put an x into all the boxes [ ] relevant to your issue (like this: [x])
  • Use the Preview tab to see what your issue will actually look like

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2019.04.01. If it's not, read this FAQ entry and update. Issues with outdated version will be rejected.

  • I've verified and I assure that I'm running youtube-dl 2019.04.01

Before submitting an issue make sure you have:

  • At least skimmed through the README, most notably the FAQ and BUGS sections
  • Searched the bugtracker for similar issues including closed ones
  • Checked that provided video/audio/playlist URLs (if any) are alive and playable in a browser

What is the purpose of your issue?

  • Bug report (encountered problems with youtube-dl)
  • Site support request (request for adding support for a new site)
  • Feature request (request for a new functionality)
  • Question
  • Other

The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your issue

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:

Add the -v flag to your command line you run youtube-dl with (youtube-dl -v <your command line>), copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):

youtube-dl -F https://animedigitalnetwork.fr/video/one-piece/9148-episode-846-nami-et-zeus-le-cumulonimbus -v

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-F', 'https://animedigitalnetwork.fr/video/one-piece/9148-episode-846-nami-et-zeus-le-cumulonimbus', '-v']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2019.04.01
[debug] Python version 3.7.3 (CPython) - Linux-5.0.4-arch1-1-ARCH-x86_64-with-arch
[debug] exe versions: ffmpeg 4.1.2, ffprobe 4.1.2, rtmpdump 2.4
[debug] Proxy map: {}
[ADN] 9814: Downloading webpage
ERROR: Unable to extract player config; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/lib/python3.7/site-packages/youtube_dl/YoutubeDL.py", line 794, in extract_info
    ie_result = ie.extract(url)
  File "/usr/lib/python3.7/site-packages/youtube_dl/extractor/common.py", line 529, in extract
    ie_result = self._real_extract(url)
  File "/usr/lib/python3.7/site-packages/youtube_dl/extractor/adn.py", line 103, in _real_extract
    r'playerConfig\s*=\s*({.+});', webpage, 'player config'), video_id)
  File "/usr/lib/python3.7/site-packages/youtube_dl/extractor/common.py", line 1004, in _search_regex
    raise RegexNotFoundError('Unable to extract %s' % _name)
youtube_dl.utils.RegexNotFoundError: Unable to extract player config; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

If the purpose of this issue is a site support request please provide all kinds of example URLs support for which should be included (replace following example URLs by yours):

Note that youtube-dl does not support sites dedicated to copyright infringement. In order for site support request to be accepted all provided example URLs should not violate any copyrights.

Description of your issue, suggested solution and other information

Everything is there, you can't do anything even if you are logged with a subscribed account, it will crash when youtube-dl is downloading the webpage.

EDIT: By the way, if you could add an .ass extractor with custom styles instead of .srt would be nice.
@remitamine remitamine closed this Apr 3, 2019
@remitamine remitamine added the duplicate label Apr 3, 2019
@ghost
Copy link
Author

@ghost ghost commented Apr 3, 2019

@remitamine Could you gimme the link of the duplicate topic ? Didn't found anything about that before. Thanks.
@remitamine
Copy link
Collaborator

@remitamine remitamine commented Apr 3, 2019 •
edited

Duplicate of #17084.
@ghost
Copy link
Author

@ghost ghost commented Apr 3, 2019

It's the same error for you ? He can download something instead of me, can't even get the page download.
@remitamine
Copy link
Collaborator

@remitamine remitamine commented Apr 3, 2019

ok, same error happen for free videos so this is caused by a new change in the website.

youtube-dl -v https://animedigitalnetwork.fr/video/bloom-into-you/9407-bande-annonce-1-bloom-into-you-a-la-decouverte-de-l-amour
[debug] System config: []
[debug] User config: ['-f', 'best[height<=768]/bestvideo[height<=768]+bestaudio/best', '--hls-prefer-ffmpeg', '--external-downloader', 'curl', '--sub-format', 'ass/tt/ttml/dfxp/scc/vtt/srt/sami/best']
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://animedigitalnetwork.fr/video/bloom-into-you/9407-bande-annonce-1-bloom-into-you-a-la-decouverte-de-l-amour']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2019.04.01
[debug] Python version 3.7.3 (CPython) - Linux-5.0.5-arch1-1-ARCH-x86_64-with-arch
[debug] exe versions: ffmpeg 4.1.2, ffprobe 4.1.2, phantomjs 2.1.1, rtmpdump 2.4
[debug] Proxy map: {}
[ADN] 9407: Downloading webpage
ERROR: Unable to extract player config; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/lib/python3.7/site-packages/youtube_dl/YoutubeDL.py", line 796, in extract_info
    ie_result = ie.extract(url)
  File "/usr/lib/python3.7/site-packages/youtube_dl/extractor/common.py", line 529, in extract
    ie_result = self._real_extract(url)
  File "/usr/lib/python3.7/site-packages/youtube_dl/extractor/adn.py", line 103, in _real_extract
    r'playerConfig\s*=\s*({.+});', webpage, 'player config'), video_id)
  File "/usr/lib/python3.7/site-packages/youtube_dl/extractor/common.py", line 1004, in _search_regex
    raise RegexNotFoundError('Unable to extract %s' % _name)
youtube_dl.utils.RegexNotFoundError: Unable to extract player config; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
@remitamine remitamine reopened this Apr 3, 2019
@remitamine remitamine removed the duplicate label Apr 3, 2019
@remitamine remitamine closed this in b966740 Apr 4, 2019
@LASThope1337
Copy link

@LASThope1337 LASThope1337 commented Apr 4, 2019 •
edited

My script to beautify the JS code doesn't work anymore

import base64
import os
import re
import urllib.request
from collections import deque

import jsbeautifier
from jsbeautifier.unpackers import UNPACKERS

for unpacker in UNPACKERS:
    if 'javascriptobfuscator' in unpacker.__name__:
        def unpack(code):
            matches = re.search(r"(?s)((?:var\s+)?(_0x[0-9a-f]+)\s*=\s*\[\s*(.+?)\s*\].+?}\(\2\s*,\s*(0x[0-9a-f]+)\)\);\s*)(?:var\s+)?(_0x[0-9a-f]+)", code)
            if matches:
                repl_array, shift, repl_func = matches.group(3, 4, 5)
                repl_array = deque(base64.b64decode(x[1:-1].encode().decode('unicode_escape')).decode().replace(r"'", r"\'") for x in repl_array.split(','))
                repl_array.rotate(-int(shift, 16))
                code = code[len(matches.group(1)):]
                code = re.sub(r"%s\('(0x[0-9a-f]+)'\)" % repl_func, lambda mobj: "'%s'" % repl_array[int(mobj.group(1), 16)], code)
            return code
        unpacker.unpack = unpack
        break

with urllib.request.urlopen('https://animedigitalnetwork.fr/components/com_vodvideo/videojs/adn-vjs.min.js') as adn_min:
    opts = jsbeautifier.default_options()
    opts.eol = os.linesep
    opts.unescape_strings = True
    code = jsbeautifier.beautify(adn_min.read().decode(), opts)
    with open('adn-beautifier.js', 'wb') as adn_unmin:
        adn_unmin.write(code.encode())

urllib.error.HTTPError: HTTP Error 403: Forbidden

Any idea?
@remitamine
Copy link
Collaborator

@remitamine remitamine commented Apr 4, 2019

replace:

with urllib.request.urlopen('https://animedigitalnetwork.fr/components/com_vodvideo/videojs/adn-vjs.min.js') as adn_min:

with:

with urllib.request.urlopen(urllib.request.Request('https://animedigitalnetwork.fr/components/com_vodvideo/videojs/adn-vjs.min.js', headers={'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:67.0) Gecko/20100101 Firefox/67.0'})) as adn_min:
@LASThope1337
Copy link

@LASThope1337 LASThope1337 commented Apr 4, 2019

b966740 didn't fix anything for me.

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['https://animedigitalnetwork.fr/video/one-punch-man-saison-2/9814-episode-0-episode-special', '--username', 'PRIVATE', '--password', 'PRIVATE', '--cookies', 'adn.txt', '--format', 'best', '--write-sub', '--verbose']
[debug] Encodings: locale cp1252, fs utf-8, out utf-8, pref cp1252
[debug] youtube-dl version 2019.04.01
[debug] Python version 3.7.2 (CPython) - Windows-10-10.0.17763-SP0
[debug] exe versions: ffmpeg 4.1.1, ffprobe 4.1.1
[debug] Proxy map: {}
[ADN] 9814: Downloading webpage
ERROR: Unable to extract player config; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "youtube-dl\youtube_dl\YoutubeDL.py", line 794, in extract_info
    ie_result = ie.extract(url)
  File "youtube-dl\youtube_dl\extractor\common.py", line 529, in extract
    ie_result = self._real_extract(url)
  File "youtube-dl\youtube_dl\extractor\adn.py", line 103, in _real_extract
    r'playerConfig\s*=\s*({.+});', webpage, 'player config'), video_id)
  File "youtube-dl\youtube_dl\extractor\common.py", line 1004, in _search_regex
    raise RegexNotFoundError('Unable to extract %s' % _name)
youtube_dl.utils.RegexNotFoundError: Unable to extract player config; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
@remitamine
Copy link
Collaborator

@remitamine remitamine commented Apr 4, 2019

you're not using the latest commit:

python __main__.py --all-subs https://animedigitalnetwork.fr/video/one-punch-man-saison-2/9814-episode-0-episode-special
[ADN] 9814: Downloading webpage
[ADN] 9814: Downloading player config JSON metadata
[ADN] 9814: Downloading links JSON metadata
[ADN] 9814: Downloading vostf mobile JSON metadata
[ADN] 9814: Downloading m3u8 information
[ADN] 9814: Downloading vostf sd JSON metadata
[ADN] 9814: Downloading m3u8 information
[ADN] 9814: Downloading subtitles data
[info] Writing video subtitles to: ONE-PUNCH MAN Saison 2 - Épisode 0-9814.fr.ssa
[download] Destination: ONE-PUNCH MAN Saison 2 - Épisode 0-9814.mp4
ffmpeg version n4.1.2 Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 8.2.1 (GCC) 20181127
  configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libdrm --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enable-libjack --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-nvdec --enable-nvenc --enable-omx --enable-shared --enable-version3
  libavutil      56. 22.100 / 56. 22.100
  libavcodec     58. 35.100 / 58. 35.100
  libavformat    58. 20.100 / 58. 20.100
  libavdevice    58.  5.100 / 58.  5.100
  libavfilter     7. 40.101 /  7. 40.101
  libswscale      5.  3.100 /  5.  3.100
  libswresample   3.  3.100 /  3.  3.100
  libpostproc    55.  3.100 / 55.  3.100
[hls,applehttp @ 0x5615a1a2c100] Opening 'https://free5.animedigitalnetwork.fr/1554426509-0-7a1cc197c8a9bedab6c9c81c140965b4/video0_free_480p/sequence_0.ts' for reading
[hls,applehttp @ 0x5615a1a2c100] Opening 'https://free5.animedigitalnetwork.fr/1554426509-0-7a1cc197c8a9bedab6c9c81c140965b4/video0_free_480p/sequence_1.ts' for reading
Input #0, hls,applehttp, from 'https://free5.animedigitalnetwork.fr/1554426509-0-7a1cc197c8a9bedab6c9c81c140965b4/video0_free_480p/playlist.m3u8':
  Duration: 00:24:04.40, start: 0.050422, bitrate: 0 kb/s
  Program 0 
    Metadata:
      variant_bitrate : 0
    Stream #0:0: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p, 848x480 [SAR 160:159 DAR 16:9], 23.98 fps, 23.98 tbr, 90k tbn, 47.95 tbc
    Metadata:
      variant_bitrate : 0
    Stream #0:1: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp
    Metadata:
      variant_bitrate : 0
Output #0, mp4, to 'file:ONE-PUNCH MAN Saison 2 - Épisode 0-9814.mp4.part':
  Metadata:
    encoder         : Lavf58.20.100
    Stream #0:0: Video: h264 (High) (avc1 / 0x31637661), yuv420p, 848x480 [SAR 160:159 DAR 16:9], q=2-31, 23.98 fps, 23.98 tbr, 90k tbn, 90k tbc
    Metadata:
      variant_bitrate : 0
    Stream #0:1: Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp
    Metadata:
      variant_bitrate : 0
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
frame=  142 fps=0.0 q=-1.0 size=     512kB time=00:00:05.95 bitrate= 704.8kbits
@LASThope1337
Copy link

@LASThope1337 LASThope1337 commented Apr 4, 2019

My way of do was bad, sorry for the disruption.
@ghost
Copy link
Author

@ghost ghost commented Apr 8, 2019

@remitamine If the key is not automatically changed every day in the extractor, it will not work. So the new youtube-dl version (2019-04-07) doesn't work.
@remitamine
Copy link
Collaborator

@remitamine remitamine commented Apr 8, 2019

this is expected, obviously i'm not going to update the key daily, i don't even use the website, that's why I have explained how to get a new key at #12724.
@ghost
Copy link
Author

@ghost ghost commented Apr 8, 2019 •
edited by ghost

@remitamine I've send you by mail something to update automatically the key inside of the adn.py file a few days ago, this is not possible to commit it addition of the adn.py file ?

EDIT: I've send you the updated version from the new extractor.
@remitamine
Copy link
Collaborator

@remitamine remitamine commented Apr 8, 2019

there are security concerns about executing arbitrary js code with Node.js, that's why this solution can't be included.
@rautamiekka
Copy link

@rautamiekka rautamiekka commented Apr 8, 2019

there are security concerns about executing arbitrary js code with Node.js, that's why this solution can't be included.

Surely there's a way to restrict the JS' abilities so it doesn't do something bad ?
@remitamine
Copy link
Collaborator

@remitamine remitamine commented Apr 8, 2019

I didn't check on this, but in my opinion, the prefered way would be to use the API to extract the raw WebVTT subtitles.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
@rautamiekka @remitamine @LASThope1337
You can’t perform that action at this time.