site www.prosieben.de broken again

[realdmitchell]

Hi
Apologies, but seems like this site has again changed its code.

Many thanks,

youtube-dl -v http://www.prosieben.de/tv/galileo/videos/2015187-sonntag-bling-bling-fuer-die-augen-kontaktlinsen-aus-gold-ganze-folge
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'--restrict-filenames', u'-v', u'http://www.prosieben.de/tv/galileo/videos/2015187-sonntag-bling-bling-fuer-die-augen-kontaktlinsen-aus-gold-ganze-folge']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2015.07.07
[debug] Python version 2.7.6 - Linux-3.13.0-32-generic-x86_64-with-Ubuntu-14.04-trusty
[debug] exe versions: none
[debug] Proxy map: {}
[prosiebensat1] tv/galileo/videos/2015187-sonntag-bling-bling-fuer-die-augen-kontaktlinsen-aus-gold-ganze-folge: Downloading webpage
[prosiebensat1] 3900422: Downloading videos JSON
[prosiebensat1] 3900422: Downloading sources JSON
[prosiebensat1] 3900422: Downloading urls JSON
[debug] Invoking downloader on u'http://vas.sim-technik.de/f4mgenerator.f4m?cid=3900422&ttl=604800&access_token=prosieben&cdn=akamai&token=9cef1a3b2c762757df93f6b999e2b031402b860d62d6769896fbd6686201d2d8'
[download] Downloading f4m manifest
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/local/bin/youtube-dl/__main__.py", line 19, in <module>
  File "/usr/local/bin/youtube-dl/youtube_dl/__init__.py", line 406, in main
  File "/usr/local/bin/youtube-dl/youtube_dl/__init__.py", line 396, in _real_main
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1501, in download
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 665, in extract_info
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 711, in process_ie_result
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1171, in process_video_result
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1433, in process_info
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1375, in dl
  File "/usr/local/bin/youtube-dl/youtube_dl/downloader/common.py", line 342, in download
  File "/usr/local/bin/youtube-dl/youtube_dl/downloader/f4m.py", line 294, in real_download
  File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1300, in XML
    parser.feed(text)
  File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1642, in feed
    self._raiseerror(v)
  File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1506, in _raiseerror
    raise err
xml.etree.ElementTree.ParseError: XML or text declaration not at start of entity: line 2, column 0

[dstftw]

This issue will be fixed in the next version. Thanks for the report.

[dstftw]

@jaimeMF don't you think we should include this fix in generic downloader and common extractor. And also properly handle full path URLs here (media_el.attrib.get('href') is http://vod-akamai-psd-hds.p7s1digital.de/z/clips/0c/2c/2104602-1nx39t7-tp06.mp4/manifest.f4m?hdnea=st%3D1436979277%7Eexp%3D1437065677%7Eacl%3D%2Fz%2Fclips%2F%2A%2F2104602-%2A%7Ehmac%3D037fcf329d7da0d3059f78ff5ad121ea16f73a12fe1e02db3d55378f108bcd92&hdcore=3). I don't know if this violates the specification but I stubmle second time already on full path URLs here.

[jaimeMF]

@dstftw Sounds OK.

[dstftw]

@jaimeMF well, there is another problem: bitrates in generated manifest (http://vas.sim-technik.de/f4mgenerator.f4m?cid=2104602&ttl=604800&access_token=prosieben&cdn=akamai&token=9fa0ea1fa895d21de668d34bfc87266a2dbf759add0a7e5a54156521fd456cd3&g=EWLYAQCJSEUQ&hdcore=3.7.0&plugin=aasp-3.7.0.39.44) and concrete manifests (e.g. http://vod-akamai-psd-hds.p7s1digital.de/z/clips/0c/2c/2104602-1nx39t7-tp06.mp4/manifest.f4m?hdnea=st%3D1436979277%7Eexp%3D1437065677%7Eacl%3D%2Fz%2Fclips%2F%2A%2F2104602-%2A%7Ehmac%3D037fcf329d7da0d3059f78ff5ad121ea16f73a12fe1e02db3d55378f108bcd92&hdcore=3) doesn't match leading to exception here:

  File "youtube_dl\downloader\f4m.py", line 308, in real_download
    lambda f: int(f[0]) == requested_bitrate, formats))[0]
IndexError: list index out of range

Is it safe to do the recursive manifest extraction (at least one more level deep) here when media_el.attrib.get('href') or media_el.attrib.get('url') is itself f4m manifest?