RTE.ie Player support #3953

httpless · 2014-10-14T16:44:03Z

Is it possible to get support for the rte.ie player (http://www.rte.ie/player)? Output from --verbose on a freshly updated install to an example video

$ youtube-dl --verbose http://www.rte.ie/player/se/show/10333418/
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['--verbose', 'http://www.rte.ie/player/se/show/10333418/']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2014.10.13
[debug] Python version 2.7.6 - Linux-3.13.0-36-generic-x86_64-with-Ubuntu-14.04-trusty
[debug] Proxy map: {}
[generic] 10333418: Requesting header
WARNING: Falling back on generic information extractor.
[generic] 10333418: Downloading webpage
[generic] 10333418: Extracting information
ERROR: Unsupported URL: http://www.rte.ie/player/se/show/10333418/; please report this issue on https://yt-dl.org/bug . Be sure to call     youtube-dl with the --verbose flag and include its complete output. Make sure you are using the latest version; type  youtube-dl -U  to update.
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/youtube_dl/extractor/generic.py", line 540, in _real_extract
    doc = parse_xml(webpage)
  File "/usr/local/lib/python2.7/dist-packages/youtube_dl/utils.py", line 1550, in parse_xml
    tree = xml.etree.ElementTree.XML(s.encode('utf-8'), **kwargs)
  File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1300, in XML
    parser.feed(text)
  File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1642, in feed
    self._raiseerror(v)
  File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1506, in _raiseerror
    raise err
ParseError: undefined entity: line 12, column 25
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/youtube_dl/YoutubeDL.py", line 524, in extract_info
    ie_result = ie.extract(url)
  File "/usr/local/lib/python2.7/dist-packages/youtube_dl/extractor/common.py", line 193, in extract
    return self._real_extract(url)
  File "/usr/local/lib/python2.7/dist-packages/youtube_dl/extractor/generic.py", line 911, in _real_extract
    raise ExtractorError('Unsupported URL: %s' % url)
ExtractorError: Unsupported URL: http://www.rte.ie/player/se/show/10333418/; please report this issue on https://yt-dl.org/bug . Be sure to call youtube-dl with the --verbose flag and include its complete output. Make sure you are using the latest version; type  youtube-dl -U  to update.

Dineshs91 · 2014-10-20T19:29:29Z

I was trying to implement extractor for this. There is a catch here. They have bitrate in one manifest and other information in a different manifest.

Example: http://www.rte.ie/player/in/show/10333418/

Feeds url: http://feeds.rasset.ie/rteavgen/player/playlist/?type=iptv&format=xml&showId=10333418

Format can be JSON also.

1. Bitrate and link for 2nd manifest

http://vod.hds.rasset.ie/hds-vod/2014/1013/20141013_rteone-ninenews-ninenews_cl10333418_10333422_260_/manifest.f4m

<manifest xmlns="http://ns.adobe.com/f4m/2.0">
    <media href="20141013_rteone-ninenews-ninenews_cl10333418_10333422_260__512k.mp4.f4m" bitrate="512" width="512" height="288"/>
    <media href="20141013_rteone-ninenews-ninenews_cl10333418_10333422_260__1024k.mp4.f4m" bitrate="1024" width="640" height="360"/>
</manifest>

2. BootstrapInfo, metadata

http://vod.hds.rasset.ie/hds-vod/2014/1013/20141013_rteone-ninenews-ninenews_cl10333418_10333422_260_/20141013_rteone-ninenews-ninenews_cl10333418_10333422_260__512k.mp4.f4m

<manifest xmlns="http://ns.adobe.com/f4m/1.0">
    <id>
    20141013_rteone-ninenews-ninenews_cl10333418_10333422_260__512k.mp4
    </id>
    <mimeType></mimeType>
    <streamType>recorded</streamType>
    <duration>1562.48</duration>
    <fragment-duration>4000</fragment-duration>
    <bootstrapInfo profile="named" id="bootstrap4233">
AAAAi2Fic3QAAAAAAAAAAgAAAAPoAAAAAAAX12YAAAAAAAAAAAAAAAAAAQAAABlhc3J0AAAAAAAAAAABAAAAAQAAAYcBAAAARmFmcnQAAAAAAAAD6AAAAAADAAAAAQAAAAAAAAAAAAAPoAAAAYcAAAAAABfNwAAACcQAAAAAAAAAAAAAAAAAAAAAAA==
    </bootstrapInfo>
    <media streamId="20141013_rteone-ninenews-ninenews_cl10333418_10333422_260__512k.mp4" url="20141013_rteone-ninenews-ninenews_cl10333418_10333422_260__512k.mp4" bootstrapInfoId="bootstrap4233">
        <metadata>
AgAKb25NZXRhRGF0YQgAAAAAAAhkdXJhdGlvbgBAmGnrhR64UgAFd2lkdGgAQIAAAAAAAAAABmhlaWdodABAcgAAAAAAAAAMdmlkZW9jb2RlY2lkAgAEYXZjMQAMYXVkaW9jb2RlY2lkAgAEbXA0YQAKYXZjcHJvZmlsZQBAU0AAAAAAAAAIYXZjbGV2ZWwAQD4AAAAAAAAABmFhY2FvdAAAAAAAAAAAAAAOdmlkZW9mcmFtZXJhdGUAQDkAAAAAAAAAD2F1ZGlvc2FtcGxlcmF0ZQBA1YiAAAAAAAANYXVkaW9jaGFubmVscwBAAAAAAAAAAAAJdHJhY2tpbmZvCgAAAAIDAAZsZW5ndGgAQOMSwAAAAAAACXRpbWVzY2FsZQBAOQAAAAAAAAAIbGFuZ3VhZ2UCAANgYGAAAAkDAAZsZW5ndGgAQYBtpmAAAAAACXRpbWVzY2FsZQBA1YiAAAAAAAAIbGFuZ3VhZ2UCAANgYGAAAAkAAAk=
        </metadata>
    </media>
</manifest>

But F4mFD expects all information like bitrate, bootstrapinfo and metadata to be in one single manifest.
Here this information is in 2 different manifests.

Any suggestions on how to proceed ?

My suggestion would be to set bitrate in info_dict in RteIE and modify F4mFD to use it if bitrate is provided, else proceed with the existing functionality.

Also note that version 2.0 is used in the first manifest. I believe this should also be implemented.

Thanks !

jaimeMF · 2014-10-20T20:33:57Z

Does it fail to download the video if you pass as the video url http://vod.hds.rasset.ie/hds-vod/2014/1013/20141013_rteone-ninenews-ninenews_cl10333418_10333422_260_/20141013_rteone-ninenews-ninenews_cl10333418_10333422_260__512k.mp4.f4m? If it fails, could you upload your current code to test it?

You, are right, we should support the version 2.0. If someone wants to have a look, it's documented at https://www.adobe.com/devnet/hds.html .

Dineshs91 · 2014-10-21T17:18:26Z

That video (http://www.rte.ie/player/in/show/10333418/) doesn't exist anymore.

So using this link now - http://www.rte.ie/player/in/show/10336191/

[rte] extractor that I wrote - rte.py - https://gist.github.com/Dineshs91/0a41274138a5d640a384

Here's the command I used to test and the result

$ python test/test_download.py TestDownload.test_Rte
[Rte] 10336191: Downloading webpage
[Rte] 10336191: Downloading JSON metadata
[Rte] 10336191: Downloading f4m manifest
[info] Writing video description metadata as JSON to: 10336191.info.json
[debug] Invoking downloader on u'http://vod.hds.rasset.ie/hds-vod/2014/1021/20141021_rteone-onenews-onenews_cl10336191_10336207_260_/20141021_rteone-onenews-onenews_cl10336191_10336207_260__1024k.mp4.f4m'
[download] Downloading f4m manifest
[download] Destination: 10336191.mp4

E

ERROR: test_Rte (main.TestDownload)

Traceback (most recent call last):
File "test/test_download.py", line 124, in test_template
res_dict = ydl.extract_info(test_case['url'])
File "/Users/dinesh/Documents/developer/youtube-dl/youtube-dl/youtube_dl/YoutubeDL.py", line 535, in extract_info
return self.process_ie_result(ie_result, download, extra_info)
File "/Users/dinesh/Documents/developer/youtube-dl/youtube-dl/youtube_dl/YoutubeDL.py", line 577, in process_ie_result
return self.process_video_result(ie_result, download=download)
File "/Users/dinesh/Documents/developer/youtube-dl/youtube-dl/youtube_dl/YoutubeDL.py", line 845, in process_video_result
self.process_info(new_info)
File "test/test_download.py", line 48, in process_info
return super(YoutubeDL, self).process_info(info_dict)
File "/Users/dinesh/Documents/developer/youtube-dl/youtube-dl/youtube_dl/YoutubeDL.py", line 1037, in process_info
success = dl(filename, info_dict)
File "/Users/dinesh/Documents/developer/youtube-dl/youtube-dl/youtube_dl/YoutubeDL.py", line 1013, in dl
return fd.download(name, info)
File "/Users/dinesh/Documents/developer/youtube-dl/youtube-dl/youtube_dl/downloader/common.py", line 292, in download
return self.real_download(filename, info_dict)
File "/Users/dinesh/Documents/developer/youtube-dl/youtube-dl/youtube_dl/downloader/f4m.py", line 244, in real_download
lambda f: int(f[0]) == requested_bitrate, formats))[0]
IndexError: list index out of range

Ran 1 test in 6.015s

I think this is because in F4mFD in real_download method
Line 237 in f4m.py
formats = [(int(f.attrib.get('bitrate', -1)), f) for f in doc.findall(_add_ns('media'))]

In case of requested_bitrate is not None then,
the doc has the below xml, which doesn't have bitrate. So formats list is empty.

http://vod.hds.rasset.ie/hds-vod/2014/1021/20141021_rteone-onenews-onenews_cl10336191_10336207_260_/20141021_rteone-onenews-onenews_cl10336191_10336207_260__512k.mp4.f4m

if I make requested_bitrate as None, I am able to download the video as you said.

jaimeMF · 2014-10-23T20:30:30Z

Integrate the _extract_f4m_formats method in the InfoExtractor class and open a PR, I'll review it and fix it if needed.

Dineshs91 · 2014-10-23T21:14:03Z

Created PR #4015 - Integrated _extract_f4m_formats method.

httpless · 2014-12-02T20:37:28Z

@jaimeMF @Dineshs91 thanks for this is there any update?

httpless · 2015-01-28T17:42:37Z

I've tested this and it appears to be working well. Thank you

jaimeMF · 2015-01-28T19:56:29Z

We forgot to close this issue, @Dineshs91 added support for it.

httpless · 2015-02-04T17:37:09Z

hi @jaimeMF / @Dineshs91 I've noticed some audio problems on downloads, should I create new ticket or is it better to reopen this?

jaimeMF · 2015-02-04T17:45:43Z

Please open a new issue with the output of youtube-dl --verbose URL, the player you use and a description of the problem.

dstftw added the site-support-request label Oct 14, 2014

jaimeMF closed this Jan 28, 2015

ytdl-org / youtube-dl

RTE.ie Player support #3953

RTE.ie Player support #3953

httpless commented Oct 14, 2014

Dineshs91 commented Oct 20, 2014

jaimeMF commented Oct 20, 2014

Dineshs91 commented Oct 21, 2014

jaimeMF commented Oct 23, 2014

Dineshs91 commented Oct 23, 2014

httpless commented Dec 2, 2014

httpless commented Jan 28, 2015

jaimeMF commented Jan 28, 2015

httpless commented Feb 4, 2015

jaimeMF commented Feb 4, 2015

ytdl-org / youtube-dl

Join GitHub today

GitHub is where the world builds software

RTE.ie Player support #3953

RTE.ie Player support #3953

Comments

httpless commented Oct 14, 2014

Dineshs91 commented Oct 20, 2014

jaimeMF commented Oct 20, 2014

Dineshs91 commented Oct 21, 2014

E

ERROR: test_Rte (main.TestDownload)

jaimeMF commented Oct 23, 2014

Dineshs91 commented Oct 23, 2014

httpless commented Dec 2, 2014

httpless commented Jan 28, 2015

jaimeMF commented Jan 28, 2015

httpless commented Feb 4, 2015

jaimeMF commented Feb 4, 2015

Essential cookies

Always active

Analytics cookies