Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error 403 with JWPlayer. Possible Solution Included. #26074

Closed
DevParapalli opened this issue Jul 21, 2020 · 0 comments
Closed

Error 403 with JWPlayer. Possible Solution Included. #26074

DevParapalli opened this issue Jul 21, 2020 · 0 comments

Comments

@DevParapalli
Copy link

@DevParapalli DevParapalli commented Jul 21, 2020

Checklist

  • I'm reporting a broken site support
  • I've verified that I'm running youtube-dl version 2020.06.16.1
  • I've checked that all provided URLs are alive and playable in a browser
  • I've checked that all URLs and arguments with special characters are properly quoted or escaped
  • I've searched the bugtracker for similar issues including closed ones

Verbose log

Caused when using the webpage as input.

youtube-dl -F https://cdn.jwplayer.com/players/2YD5fq8v-dVg6Al1Y.html -v --> use this link to get the current tokenized js
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-F', 'https://cdn.jwplayer.com/players/2YD5fq8v-dVg6Al1Y.html', '-v']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2020.06.16.1
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.19041
[debug] exe versions: ffmpeg git-2020-04-17-889ad93, ffprobe git-2020-04-17-889ad93
[debug] Proxy map: {}
[JWPlatform] 2YD5fq8v: Downloading JSON metadata
ERROR: Unable to download JSON metadata: HTTP Error 403: Forbidden (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpspbsbtmq\build\youtube_dl\extractor\common.py", line 627, in _request_webpage
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpspbsbtmq\build\youtube_dl\YoutubeDL.py", line 2238, in urlopen
  File "C:\Python\Python34\lib\urllib\request.py", line 470, in open
  File "C:\Python\Python34\lib\urllib\request.py", line 580, in http_response
  File "C:\Python\Python34\lib\urllib\request.py", line 508, in error
  File "C:\Python\Python34\lib\urllib\request.py", line 442, in _call_chain
  File "C:\Python\Python34\lib\urllib\request.py", line 588, in http_error_default

Caused by using the tokenized link to JSON.

youtube-dl -F https://content.jwplatform.com/v2/media/tcg4Rnxm?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJyZWNvbW1lbmRhdGlvbnNfcGxheWxpc3RfaWQiOiJvM3haWTgxQiIsInJlc291cmNlIjoiL3YyL21lZGlhL3RjZzRSbnhtIiwiZXhwIjoxNTk0OTc5NTIwfQ.zQzAXq9KLDueU01YlX5uZ891PO-L1yDn5Teuy90Y7ko -v
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-F', 'https://content.jwplatform.com/v2/media/tcg4Rnxm?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJyZWNvbW1lbmRhdGlvbnNfcGxheWxpc3RfaWQiOiJvM3haWTgxQiIsInJlc291cmNlIjoiL3YyL21lZGlhL3RjZzRSbnhtIiwiZXhwIjoxNTk0OTc5NTIwfQ.zQzAXq9KLDueU01YlX5uZ891PO-L1yDn5Teuy90Y7ko', '-v']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2020.06.16.1
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.19041
[debug] exe versions: ffmpeg git-2020-04-17-889ad93, ffprobe git-2020-04-17-889ad93
[debug] Proxy map: {}
[JWPlatform] tcg4Rnxm: Downloading JSON metadata
ERROR: Unable to download JSON metadata: HTTP Error 403: Forbidden (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpspbsbtmq\build\youtube_dl\extractor\common.py", line 627, in _request_webpage
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpspbsbtmq\build\youtube_dl\YoutubeDL.py", line 2238, in urlopen
  File "C:\Python\Python34\lib\urllib\request.py", line 470, in open
  File "C:\Python\Python34\lib\urllib\request.py", line 580, in http_response
  File "C:\Python\Python34\lib\urllib\request.py", line 508, in error
  File "C:\Python\Python34\lib\urllib\request.py", line 442, in _call_chain
  File "C:\Python\Python34\lib\urllib\request.py", line 588, in http_error_default

Description

I've been trying to download some files of off my institute's website. They use jwplayer with the {media_id}-{player_id}.js format. The .js file contains the exact URL (token included) to access the required JSON metadata but youtube-dl doesn't work even if I have provided the actual tokenized version of the JSON link.

A solution I've come up with works on all video preview page that is obtained by using any link of the format https://cdn.jwplayer.com/players/{media_id}-{player_id}.html

A js parser(noob's implimentation below) that extracts the var jwConfig = {} from the script_tag and subsequently extracts the correct media_uri/playlist_uri would do the trick.

The .js data(request.content) should be treated as a str/bytes object and split using the (";") delimiter. The element [-5] (please verify) should be the appropriate variable (jwConfig). Removing the var jwConfig = (incl. spaces) then using json.loads() to get a dictionary. This dictionary has a playlist key which contains the JSON metadata. (might also have to replace the // with https://)

Probable solution for the preview page of most media files (jwplayer-hosted/cloud-hosted)

import json
import requests
#
main_url = "https://cdn.jwplayer.com/players/<media_id>-<player_id>.html"

#
def get_playlist(js_text): # besure to change split wrt the r.content from requests.
    _list = js_content.split(";")
    jwConfig = _list[-5]
    _dict = jwConfig.split("= ", 1)[-1]
    _config = json.loads(_dict)
    _playlist = _config['playlist']
    playlist_url = _playlist.replace("//", "https://")
    return(playlist_url)


# this is the input preview webpage. (and only the preview webpage) (only use if 403?)
resp = requests.get(main_url)

def get_js_from_webpage(response): 
    resp_str = response.decode('ascii', "ignore")
    index = resp_str.find("script.src = ")
    return(resp_str[index + 14 : index + 73]) # i have no idea how to do this part
    #(are hardcoded values ok ?) media id and player id have constant length.


url = get_js_from_webpage(resp.content)
js_content = requests.get(url).text
playlist = get_playlist(js_content)
print(playlist) # all of this can be done under one function i guess.
# works fine but im a noob so
# edited to add
playlist_json = requests.get(playlist).json()
mpeg_url = playlist_json['playlist'][0]['sources'][0]['file']
print(mpeg_url) 
# this gives me proper download. in this case links are valid for 3 hours 

When visiting the playlist url returned after this, we get a JSON document similar to the ones we get when using a untokenized media.

{
   "title":"Hb Xi C Ch-8 Chem Bond Cl 18",
   "description":"",
   "kind":"Single Item",
   "playlist":[
      {
         "title":"Hb Xi C Ch-8 Chem Bond Cl 18",
         "mediaid":"tcg4Rnxm",
         "link":"https://content.jwplatform.com/previews/tcg4Rnxm?exp=1594979760&sig=ce7935627dda7be1085357c080e22c3b",
         "image":"https://content.jwplatform.com/v2/media/tcg4Rnxm/poster.jpg?width=720",
         "images":[
            {
               "src":"https://content.jwplatform.com/v2/media/tcg4Rnxm/poster.jpg?width=320",
               "width":320,
               "type":"image/jpeg"
            },
            {
               "src":"https://content.jwplatform.com/v2/media/tcg4Rnxm/poster.jpg?width=480",
               "width":480,
               "type":"image/jpeg"
            },
            {
               "src":"https://content.jwplatform.com/v2/media/tcg4Rnxm/poster.jpg?width=640",
               "width":640,
               "type":"image/jpeg"
            },
            {
               "src":"https://content.jwplatform.com/v2/media/tcg4Rnxm/poster.jpg?width=720",
               "width":720,
               "type":"image/jpeg"
            }
         ],
         "duration":3562,
         "pubdate":1588320989,
         "description":"",
         "recommendations":"https://content.jwplatform.com/v2/playlists/o3xZY81B?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJyZWxhdGVkX21lZGlhX2lkIjoidGNnNFJueG0iLCJleHAiOjE1OTQ5Nzk3NjAsInJlc291cmNlIjoiL3YyL3BsYXlsaXN0cy9vM3haWTgxQiJ9.phjoZ4M9PYpe8d3nWeO7v3ZpDFLlYbGA7TmEJx-CaxM",
         "sources":[
            {
               "file":"https://content.jwplatform.com/manifests/tcg4Rnxm.m3u8?exp=1594979760&sig=b3fb6ae11e470fd582017ff9382a438c",
               "type":"application/vnd.apple.mpegurl"
            },
            {
               "file":"https://content.jwplatform.com/videos/tcg4Rnxm-LKUbW2zZ.mp4?exp=1594979760&sig=f4ebbde57bf9f3fe02a5a7e42c9ab88c",
               "type":"video/mp4",
               "height":180,
               "width":320,
               "label":"180p"
            },
            {
               "file":"https://content.jwplatform.com/videos/tcg4Rnxm-6SUujk2h.mp4?exp=1594979760&sig=c88111d374dd9a0fafa5739d09e76ff0",
               "type":"video/mp4",
               "height":270,
               "width":480,
               "label":"270p"
            },
            {
               "file":"https://content.jwplatform.com/videos/tcg4Rnxm-RfJ1YLUn.mp4?exp=1594979760&sig=2a940230ec914d6524dfeb7297f28f37",
               "type":"video/mp4",
               "height":720,
               "width":1280,
               "label":"720p"
            },
            {
               "file":"https://content.jwplatform.com/videos/tcg4Rnxm-eFTQgXDM.m4a?exp=1594979760&sig=457512da11ab968945b9512a82ca8b32",
               "type":"audio/mp4",
               "label":"AAC Audio"
            },
            {
               "file":"https://content.jwplatform.com/videos/tcg4Rnxm-fD4bxXyo.mp4?exp=1594979760&sig=1b1335fb87eafaacc2e752e04cdd2ebb",
               "type":"video/mp4",
               "height":1080,
               "width":1920,
               "label":"1080p"
            },
            {
               "file":"https://content.jwplatform.com/videos/tcg4Rnxm-VBGx6SI9.mp4?exp=1594979760&sig=55df053c9fe7234755db817553fbe20f",
               "type":"video/mp4",
               "height":360,
               "width":640,
               "label":"360p"
            },
            {
               "file":"https://content.jwplatform.com/videos/tcg4Rnxm-PFZiwg3E.mp4?exp=1594979760&sig=dbe74ccf154696875f2c6d64db731ac6",
               "type":"video/mp4",
               "height":540,
               "width":960,
               "label":"540p"
            }
         ],
         "tracks":[
            {
               "file":"https://content.jwplatform.com/strips/tcg4Rnxm-120.vtt",
               "kind":"thumbnails"
            }
         ],
         "variations":{

         }
      }
   ],
   "feed_instance_id":"11aeef4f-cf4b-4606-8494-01c8959441be"
}

A snippet of the js (the important bit) is included below.

var jwConfig = {
  "aboutlink": "<redacted>",
  "abouttext": "<redacted>",
  "aspectratio": "16:9",
  "autoPause": {},
  "autostart": false,
  "cast": {
    "appid": "00000000"
  },
  "controls": true,
  "displaydescription": true,
  "displaytitle": false,
  "flashplayer": "//ssl.p.jwpcdn.com/player/v/8.16.3/jwplayer.flash.swf",
  "height": 360,
  "key": "<redacted>",
  "logo": {
    "file": "https://assets-jpcust.jwpsrv.com/watermarks/Qnqjvg74.png",
    "hide": true,
    "margin": "10",
    "position": "top-right"
  },
  "mute": false,
  "ph": 3,
  "pid": "dVg6Al1Y",
  "playbackRateControls": true,
  "playlist": "//content.jwplatform.com/v2/media/2YD5fq8v?token=<redacted>",
  "preload": "none",
  "repeat": false,
  "stagevideo": false,
  "stretching": "uniform",
  "width": "100%"
};
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
1 participant
You can’t perform that action at this time.