Support RTP

[peterpt]

I am using the 2020-01-01 version of youtube-dl in windows environment .
Last year youtube-dl worked perfectly on this website , however probably some changes were probably made in website streaming service .

Example of streaming link :
https://www.rtp.pt/play/p6383/italia-selvagem-os-maiores-predadores

Error from youtube-dl :

youtube-dl --verbose -F https://www.rtp.pt/play/p6383/italia-selvagem-
os-maiores-predadores
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--verbose', '-F', 'https://www.rtp.pt/play/p6383/it
alia-selvagem-os-maiores-predadores']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2020.01.01
[debug] Python version 3.4.4 (CPython) - Windows-7-6.1.7601-SP1
[debug] exe versions: ffmpeg N-92458-gfc94e9704e, ffprobe N-92458-gfc94e9704e
[debug] Proxy map: {}
[RTP] italia-selvagem-os-maiores-predadores: Downloading webpage
ERROR: italia-selvagem-os-maiores-predadores: Failed to parse JSON (caused by V
alueError("Expecting ',' delimiter: line 11 column 11 (char 715)",)); please rep
ort this issue on https://yt-dl.org/bug . Make sure you are using the latest ver
sion; type youtube-dl -U to update. Be sure to call youtube-dl with the --verb
ose flag and include its complete output.
Traceback (most recent call last):
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpj4acy7
ln\build\youtube_dl\extractor\common.py", line 899, in parse_json
File "C:\Python\Python34\lib\json_init.py", line 318, in loads
File "C:\Python\Python34\lib\json\decoder.py", line 343, in decode
File "C:\Python\Python34\lib\json\decoder.py", line 359, in raw_decode
ValueError: Expecting ',' delimiter: line 11 column 11 (char 715)
Traceback (most recent call last):
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpj4acy7
ln\build\youtube_dl\extractor\common.py", line 899, in parse_json
File "C:\Python\Python34\lib\json_init.py", line 318, in loads
File "C:\Python\Python34\lib\json\decoder.py", line 343, in decode
File "C:\Python\Python34\lib\json\decoder.py", line 359, in raw_decode
ValueError: Expecting ',' delimiter: line 11 column 11 (char 715)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpj4acy7
ln\build\youtube_dl\YoutubeDL.py", line 796, in extract_info
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpj4acy7
ln\build\youtube_dl\extractor\common.py", line 530, in extract
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpj4acy7
ln\build\youtube_dl\extractor\rtp.py", line 37, in _real_extract
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpj4acy7
ln\build\youtube_dl\extractor\common.py", line 903, in _parse_json
youtube_dl.utils.ExtractorError: italia-selvagem-os-maiores-predadores: Failed t
o parse JSON (caused by ValueError("Expecting ',' delimiter: line 11 column 11
(char 715)",)); please report this issue on https://yt-dl.org/bug . Make sure yo
u are using the latest version; type youtube-dl -U to update. Be sure to call
youtube-dl with the --verbose flag and include its complete output.

[israel-lugo]

I've found the problem.

The RTP InfoExtractor is parsing the HTML here to get its config.

This captures the following snippet:

{
          id: "player_prog",
          fileKey: "/nas2.share/pt/h264/512x384/p6265/p6265_1_201912152233180805.mp4",
          file : "https://ondemand-streaming.rtp.pt/nas2.share/pt/h264/512x384/p6265/index.m3u8?tlm=hls&streams=p6265_1_201912152233180805.mp4.m3u8:1024",
                                  seekBarThumbnailsLoc: '//cdn-images.rtp.pt/multimedia/screenshots/p6265/preview/p6265_1_201912152233180805_preview.vtt',
                      adsType: 'dfp',
          subadUnit: 'Play',
          streamType:"ondemand",
          googleCast: true,
          //extraSettings: { googleCastReceiverAppId:'1A6F2224', skin:'s3',  skinAccentColor: '0073FF'},
          extraSettings: { googleCastReceiverAppId:'1A6F2224'},
          mediaType: "video",
          poster: "https://cdn-images.rtp.pt/EPG/imagens/37684_55429_70561.jpg?v=3&w=1170"
        }

Note the commented //extraSettings... line. This should be dealt with by js_to_json. But the transformed output (caught after transformation) is:

{
          "id": "player_prog",
          "fileKey": "/nas2.share/pt/h264/512x384/p6265/p6265_1_201912152233180805.mp4",
          "file" : "https://ondemand-streaming.rtp.pt/nas2.share/pt/h264/512x384/p6265/index.m3u8?tlm=hls&streams=p6265_1_201912152233180805.mp4.m3u8:1024",
                                  "seekBarThumbnailsLoc": "//cdn-images.rtp.pt/multimedia/screenshots/p6265/preview/p6265_1_201912152233180805_preview.vtt",
                      "adsType": "dfp",
          "subadUnit": "Play",
          "streamType":"ondemand",
          "googleCast": true
          
          "extraSettings": { "googleCastReceiverAppId":"1A6F2224"},
          "mediaType": "video",
          "poster": "https://cdn-images.rtp.pt/EPG/imagens/37684_55429_70561.jpg?v=3&w=1170"
        }

Note the missing trailing , after "googleCast": true".

This is a bug in js_to_json. It's eating up the trailing comma on the previous line, when it skips a line which is entirely commented.

[israel-lugo]

Incidentally, this issue should be tagged with broken-IE (broken information extractor), probably more than site-support-request (which is IIUC requesting support for a new site). RTP is already supported, it's just broken.