Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RTV SLO 4D] Add extractor #28134

Open
wants to merge 10 commits into
base: master
Choose a base branch
from
Open

Conversation

franga2000
Copy link

@franga2000 franga2000 commented Feb 10, 2021

Before submitting a pull request make sure you have:

In order to be accepted and merged into youtube-dl each piece of code must be in public domain or released under Unlicense. Check one of the following options:

  • I am the original author of this code and I am willing to release it under Unlicense
  • I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)

What is the purpose of your pull request?

  • Bug fix
  • Improvement
  • New extractor
  • New feature

Description of your pull request and other information

This PR implements an extractor for the RTV SLO 4D video portal (archive of the Slovenian national broadcaster). The corresponding site request issue is #15590, with #19292 being a duplicate of it (the latter also mentions capodistria.rtvslo.si, but videos are no longer hosted there as they have been migrated to the main portal).

As noted in the code, it does not currently support radio shows, but I will come back and fix that once I can spare the time. In the mean time, I've tested the video support on some 10 videos without issues.

Progress

Copy link
Collaborator

@remitamine remitamine left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read coding conventions.

youtube_dl/extractor/rtvslo.py Outdated Show resolved Hide resolved
youtube_dl/extractor/rtvslo.py Outdated Show resolved Hide resolved
youtube_dl/extractor/rtvslo.py Outdated Show resolved Hide resolved
youtube_dl/extractor/rtvslo.py Outdated Show resolved Hide resolved
youtube_dl/extractor/rtvslo.py Outdated Show resolved Hide resolved
@franga2000
Copy link
Author

Is this some kind of well-known API structure that I'm unaware of? You seem to know things about it that I don't...

@remitamine
Copy link
Collaborator

Is this some kind of well-known API structure that I'm unaware of?

an old one.

youtube_dl/extractor/rtvslo.py Outdated Show resolved Hide resolved
youtube_dl/extractor/rtvslo.py Outdated Show resolved Hide resolved
youtube_dl/extractor/rtvslo.py Outdated Show resolved Hide resolved
youtube_dl/extractor/rtvslo.py Outdated Show resolved Hide resolved
youtube_dl/extractor/rtvslo.py Outdated Show resolved Hide resolved
youtube_dl/extractor/rtvslo.py Outdated Show resolved Hide resolved
youtube_dl/extractor/rtvslo.py Outdated Show resolved Hide resolved
@franga2000
Copy link
Author

Turns out that not all videos expose the addaptiveMedia dict on /getRecording and my original way of getting a JWT from /getRecordingDrm and using it to get /getMedia that was originally rejected seems to be the only get a valid URL for them (mediaFiles seems to be invalid for those recordings).

Got any more of that magic knowledge of obscure endpoints to share? And if not, should I switch everything to use getMedia or do I add yet another special case for videos without the "adaptive" HLS links?

@remitamine
Copy link
Collaborator

Turns out that not all videos expose the addaptiveMedia dict on /getRecording and my original way of getting a JWT from /getRecordingDrm and using it to get /getMedia that was originally rejected seems to be the only get a valid URL for them (mediaFiles seems to be invalid for those recordings).

post an example.

@franga2000
Copy link
Author

franga2000 commented Mar 25, 2021

post an example.

Video

getRecording:

"mediaFiles": [
      {
        "mediaType": "MP4",
        "id": 126065508,
        "filename": "2012/01/15/Slovenskiavtoleta2012x201201152139x700000x351x413x.mp4",
        "width": 351,
        "height": 413,
        "filesize": 0,
        "streamer": "rtmp://vodstr.rtvslo.si/ava_archive00",
        "bitrate": 700000,
        "streamers": {
          "http": "https://progressive.rtvslo.si/ava_archive00",
          "rtmp": "rtmp://vodstr.rtvslo.si/ava_archive00",
          "rtsp": "rtsp://vodstr.rtvslo.si/ava_archive00",
          "hls_sec": "https://vodstr.rtvslo.si/ava_archive00",
          "hls": "http://vodstr.rtvslo.si/ava_archive00"
        }
      }
    ]

Putting the parts together, like with audio files: https://progressive.rtvslo.si/ava_archive00/2012/01/15/Slovenskiavtoleta2012x201201152139x700000x351x413x.mp4404

getMedia (with JWT from getRecordingDrm):

"mediaFiles": [
      {
        "id": 126065508,
        "mediaType": "MP4",
        "streams": {
          "hls_sec": "https://vodstr.rtvslo.si/encrypted00/_definst_/2012/01/15/Slovenskiavtoleta2012x201201152139x700000x351x413x.mp4/playlist.m3u8?keylockhash=bxmYaFb7yrXAREoou44LwhN3zPvvbnggR-jLCJMtKvE",
          "hls": "http://vodstr.rtvslo.si/encrypted00/_definst_/2012/01/15/Slovenskiavtoleta2012x201201152139x700000x351x413x.mp4/playlist.m3u8?keylockhash=bxmYaFb7yrXAREoou44LwhN3zPvvbnggR-jLCJMtKvE",
          "mpeg-dash": "http://vodstr.rtvslo.si/encrypted00/_definst_/2012/01/15/Slovenskiavtoleta2012x201201152139x700000x351x413x.mp4/manifest.mpd?keylockhash=bxmYaFb7yrXAREoou44LwhN3zPvvbnggR-jLCJMtKvE",
          "http": "http://progressive.rtvslo.si/encrypted00/2012/01/15/Slovenskiavtoleta2012x201201152139x700000x351x413x.mp4?keylockhash=L9aYk1YxWK4YQe9j7yWEPom_aEgoQj7-VfKQQbL-aBU"
        },
        "bitrate": 700000,
        "height": null,
        "filesize": null,
        "width": null
      }
    ]

Both the HTTP and HLS links in streams work. "Guessing" the URLs won't work, since without a valid keylockhash, nothing is returned.

@remitamine
Copy link
Collaborator

in this case, the getMedia should be used for all videos instead of getRecording.

@ghost
Copy link

ghost commented Mar 28, 2021

@franga2000 Thanks a lot. I've been thinking about this for a long time. I used mostly FVD extension for browsers. One thing I noticed recently is that the *.mp4 files are actually MPEG-TS renamed to .mp4 extension, at least for the options that FVD gave, along with a .ts option too, so I've been downloading just the *.ts instead. I've analyzed and compared both with ffprobe a while ago but I forgot if they're identical or if the .ts ones are in better quality, I think they're the same.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants