[extractor/safari] Partially Fix Extractor⁠

[D357R0Y3R]

No description provided.

[dirkf]

Is there an issue that should be linked? Otherwise you could post a failing verbose log (using the master or nightly code).

[dirkf]

Probably the free version is what was analyzed. For an entire course, your API URL can be fetched with _download_json() and the playlist extracted. Tyhe playlist items like https://learning.oreilly.com/api/v1/book/9781835465141/chapter/video1_2.html also give JSON (despite appearances) and then you have a block like this for each video clip:

    {
      'thumbnail': null,
      'reference_id': '9781835465141-video1_2',
      'brightcove_id': null,
      'video_still': null,
      'created_time': null,
      'last_modified_time': null,
      'duration_seconds': 346,
      'streaming_source': 'Kaltura'
    }

This was served to me in a non-JS browser with no authentication.

IIRC Kaltura needs the video ID (or ref_id, as here, I guess) and partner/account IDs, which are probably effectively constant but should be found in the page or linked JS code. From the clip parameters one might infer that some clips are hosted at Brightcove, so it would be good to find a case where that applies.

[dirkf]

Also, see yt-dlp/yt-dlp#3526 (comment).

[dirkf]

I made a version of your PR from yt-dl with yt-dlp (other than conventional) changes back-ported.

With no login, I'm able to get the formats for https://www.oreilly.com/library/view/python-complete/9781835465141, with durations != 1 minute (470 items):

$ python3 -m youtube_dl --get-filename -o  '%(duration)s' 'https://www.oreilly.com/library/view/python-complete/9781835465141'
155
346
311
162
184
323
467
165
288
435
665
729
587
350
487
488
...

However, when I get the video it comes as 60s rather than the expected duration.

Presumably with a valid login it takes the if self.LOGGED_IN: branch and gets the actual course media.

Please show the actual command that gives the unexpected format. I very much hope that there is some CLI problem that is causing the specified format to be ignored.

[dirkf]

I've logged the changes in my version of your code. Then it runs under Py2 and Py3 per yt-dl requirements:

$ python -m youtube_dl -Fv 'https://www.oreilly.com/library/view/python-complete/9781835465141' --playlist-items 1
[debug] System config: [u'--prefer-ffmpeg']
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-Fv', u'https://www.oreilly.com/library/view/python-complete/9781835465141', u'--playlist-items', u'1']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2021.12.17
[debug] Git HEAD: 0eafa09a3
[debug] Python 2.7.18 (CPython i686 32bit) - Linux-4.4.0-210-generic-i686-with-Ubuntu-16.04-xenial - OpenSSL 1.1.1w  11 Sep 2023 - glibc 2.15
[debug] exe versions: avconv 4.3, avprobe 4.3, ffmpeg 4.3, ffprobe 4.3
[debug] Proxy map: {}
[safari:course] 9781835465141: Downloading course JSON
[download] Downloading playlist: Python – Complete Python, Django, Data Science and ML Guide
[safari:course] playlist Python – Complete Python, Django, Data Science and ML Guide: Collected 470 video ids (downloading 1 of them)
[download] Downloading video 1 of 1
[safari:api] 9781835465141/video1_1: Downloading part JSON
[Kaltura] 9781835465141-video1_1: Downloading webpage
[Kaltura] 1_4kzn6p7b: Downloading video info JSON
[Kaltura] 1_4kzn6p7b: Checking mp4-1829 URL
[Kaltura] 1_4kzn6p7b: Downloading m3u8 information
[info] Available formats for 1_4kzn6p7b:
format code  extension  resolution note
mp4-56       mp4        audio only   56k , isom container, 0fps, ~1.04MiB
hls-58       mp4        audio only   58k , mp4a.40.2
mp4-464      mp4        640x360     464k , isom container, avc1, 30fps, ~8.60MiB
hls-508      mp4        640x360     508k 
mp4-660      mp4        854x480     660k , isom container, avc1, 30fps, ~12.22MiB
hls-713      mp4        854x480     713k 
mp4-827      mp4        960x540     827k , isom container, avc1, 30fps, ~15.32MiB
hls-887      mp4        960x540     887k 
mp4-1272     mp4        1280x720   1272k , isom container, avc1, 30fps, ~23.55MiB
hls-1353     mp4        1280x720   1353k 
mp4-1356     mp4        1280x720   1356k , isom container, avc1, 30fps, ~25.11MiB
mp4-1829     mp4        1920x1080  1829k , m4v container, avc1, 60fps, ~33.85MiB
mp4-2363     mp4        1920x1080  2363k , isom container, avc1, 30fps, ~43.74MiB (best)
[download] Finished downloading playlist: Python – Complete Python, Django, Data Science and ML Guide
$ 

[dirkf]

I see that the https://github.com/sulyi/yt-dlp/blob/oreilly-login-fix/yt_dlp/extractor/safari.py code is using a different set of URLs for auth. Presumably there's some issue about setting cookies if you're getting different behaviour with yt-dl vs yt-dlp