Initialization URL, MPD format #11605

Open
kkeiichi opened this Issue Jan 4, 2017 · 2 comments

Projects

None yet

3 participants

@kkeiichi
kkeiichi commented Jan 4, 2017

Please follow the guide below

  • You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
  • Put an x into all the boxes [ ] relevant to your issue (like that [x])
  • Use Preview tab to see how your issue will actually look like

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2017.01.05. If it's not read this FAQ entry and update. Issues with outdated version will be rejected.

  • I've verified and I assure that I'm running youtube-dl 2017.01.05

Before submitting an issue make sure you have:

  • At least skimmed through README and most notably FAQ and BUGS sections
  • Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

  • Bug report (encountered problems with youtube-dl)
  • Site support request (request for adding support for a new site)
  • Feature request (request for a new functionality)
  • Question
  • Other

The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your issue


If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:

Add -v flag to your command line you run youtube-dl with, copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):


The example video http://n-1-18.dcs.redcdn.pl/dash/o2/n/planete/15005218_Swiat_wedlug_niemowlecia/ss/Manifest.ism?t=1483527972874 and many other on this server contains reference to bitrate in the intialization parameter in the Manifest.ism file: initialization="?type=video&bitrate=$Bandwidth$&id=$RepresentationID$". In the common.py, InfoExtractor class, _parse_mpd_formats method only replace RepresentationID parameter. This makes invalid request to the server with error response 500.

$ python3 bin/youtube-dl --print-traffic -v -f Manifest-1 "http://n-1-18.dcs.redcdn.pl/dash/o2/n/planete/15005218_Swiat_wedlug_niemowlecia/ss/Manifest.ism?t=1483527972874"
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--print-traffic', '-v', '-f', 'Manifest-1', 'http://n-1-18.dcs.redcdn.pl/dash/o2/n/planete/15005218_Swiat_wedlug_niemowlecia/ss/Manifest.ism?t=1483527972874']
[debug] Encodings: locale ISO-8859-2, fs iso8859-2, out ISO-8859-2, pref ISO-8859-2
[debug] youtube-dl version 2017.01.05
[debug] Git HEAD: 7232bb299
[debug] Python version 3.5.2+ - Linux-4.8.0-1-686-pae-i686-with-debian-stretch-sid
[debug] exe versions: ffmpeg 3.2.2-1, ffprobe 3.2.2-1, rtmpdump 2.4
[debug] Proxy map: {}
[generic] Manifest: Requesting header
send: b'HEAD /dash/o2/n/planete/15005218_Swiat_wedlug_niemowlecia/ss/Manifest.ism?t=1483527972874 HTTP/1.1\r\nHost: n-1-18.dcs.redcdn.pl\r\nConnection: close\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)\r\nAccept-Encoding: gzip, deflate\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nAccept-Language: en-us,en;q=0.5\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
WARNING: Falling back on generic information extractor.
[generic] Manifest: Downloading webpage
header: Date header: Server header: Connection header: Content-Type send: b'GET /dash/o2/n/planete/15005218_Swiat_wedlug_niemowlecia/ss/Manifest.ism?t=1483527972874 HTTP/1.1\r\nHost: n-1-18.dcs.redcdn.pl\r\nConnection: close\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)\r\nAccept-Encoding: *\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nAccept-Language: en-us,en;q=0.5\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
[generic] Manifest: Extracting information
header: Date header: Server header: Connection header: Transfer-Encoding header: Content-Type [debug] Invoking downloader on 'http://n-1-18.dcs.redcdn.pl/dash/o2/n/planete/15005218_Swiat_wedlug_niemowlecia/ss/Manifest.ism'
[dashsegments] Total fragments: 2480
[download] Destination: Manifest-Manifest.mp4
send: b'GET /dash/o2/n/planete/15005218_Swiat_wedlug_niemowlecia/ss/Manifest.ism?type=video&bitrate=$Bandwidth$&id=1 HTTP/1.1\r\nHost: n-1-18.dcs.redcdn.pl\r\nAccept-Encoding: identity\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nConnection: close\r\nAccept-Language: en-us,en;q=0.5\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\n\r\n'
reply: 'HTTP/1.1 500 Internal Server Error\r\n'
header: Date header: Server header: Content-Length header: Connection header: Content-Type send: b'GET /dash/o2/n/planete/15005218_Swiat_wedlug_niemowlecia/ss/Manifest.ism?type=video&bitrate=$Bandwidth$&id=1 HTTP/1.1\r\nHost: n-1-18.dcs.redcdn.pl\r\nConnection: close\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)\r\nAccept-Encoding: gzip, deflate\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nAccept-Language: en-us,en;q=0.5\r\n\r\n'
reply: 'HTTP/1.1 500 Internal Server Error\r\n'
header: Date header: Server header: Content-Length header: Connection header: Content-Type send: b'GET /dash/o2/n/planete/15005218_Swiat_wedlug_niemowlecia/ss/Manifest.ism?type=video&bitrate=$Bandwidth$&id=1 HTTP/1.1\r\nHost: n-1-18.dcs.redcdn.pl\r\nConnection: close\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)\r\nAccept-Encoding: gzip, deflate\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nAccept-Language: en-us,en;q=0.5\r\n\r\n'
reply: 'HTTP/1.1 500 Internal Server Error\r\n'
header: Date header: Server header: Content-Length header: Connection header: Content-Type send: b'GET /dash/o2/n/planete/15005218_Swiat_wedlug_niemowlecia/ss/Manifest.ism?type=video&bitrate=$Bandwidth$&id=1 HTTP/1.1\r\nHost: n-1-18.dcs.redcdn.pl\r\nConnection: close\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)\r\nAccept-Encoding: gzip, deflate\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nAccept-Language: en-us,en;q=0.5\r\n\r\n'
reply: 'HTTP/1.1 500 Internal Server Error\r\n'
header: Date header: Server header: Content-Length header: Connection header: Content-Type send: b'GET /dash/o2/n/planete/15005218_Swiat_wedlug_niemowlecia/ss/Manifest.ism?type=video&bitrate=$Bandwidth$&id=1 HTTP/1.1\r\nHost: n-1-18.dcs.redcdn.pl\r\nConnection: close\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)\r\nAccept-Encoding: gzip, deflate\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nAccept-Language: en-us,en;q=0.5\r\n\r\n'
reply: 'HTTP/1.1 500 Internal Server Error\r\n'
header: Date header: Server header: Content-Length header: Connection header: Content-Type send: b'GET /dash/o2/n/planete/15005218_Swiat_wedlug_niemowlecia/ss/Manifest.ism?type=video&bitrate=$Bandwidth$&id=1 HTTP/1.1\r\nHost: n-1-18.dcs.redcdn.pl\r\nConnection: close\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)\r\nAccept-Encoding: gzip, deflate\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nAccept-Language: en-us,en;q=0.5\r\n\r\n'
reply: 'HTTP/1.1 500 Internal Server Error\r\n'
header: Date header: Server header: Content-Length header: Connection header: Content-Type send: b'GET /dash/o2/n/planete/15005218_Swiat_wedlug_niemowlecia/ss/Manifest.ism?type=video&bitrate=$Bandwidth$&id=1 HTTP/1.1\r\nHost: n-1-18.dcs.redcdn.pl\r\nConnection: close\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)\r\nAccept-Encoding: gzip, deflate\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nAccept-Language: en-us,en;q=0.5\r\n\r\n'
reply: 'HTTP/1.1 500 Internal Server Error\r\n'
header: Date header: Server header: Content-Length header: Connection header: Content-Type send: b'GET /dash/o2/n/planete/15005218_Swiat_wedlug_niemowlecia/ss/Manifest.ism?type=video&bitrate=$Bandwidth$&id=1 HTTP/1.1\r\nHost: n-1-18.dcs.redcdn.pl\r\nConnection: close\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)\r\nAccept-Encoding: gzip, deflate\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nAccept-Language: en-us,en;q=0.5\r\n\r\n'
reply: 'HTTP/1.1 500 Internal Server Error\r\n'
header: Date header: Server header: Content-Length header: Connection header: Content-Type send: b'GET /dash/o2/n/planete/15005218_Swiat_wedlug_niemowlecia/ss/Manifest.ism?type=video&bitrate=$Bandwidth$&id=1 HTTP/1.1\r\nHost: n-1-18.dcs.redcdn.pl\r\nConnection: close\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)\r\nAccept-Encoding: gzip, deflate\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nAccept-Language: en-us,en;q=0.5\r\n\r\n'
reply: 'HTTP/1.1 500 Internal Server Error\r\n'
header: Date header: Server header: Content-Length header: Connection header: Content-Type send: b'GET /dash/o2/n/planete/15005218_Swiat_wedlug_niemowlecia/ss/Manifest.ism?type=video&bitrate=$Bandwidth$&id=1 HTTP/1.1\r\nHost: n-1-18.dcs.redcdn.pl\r\nConnection: close\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)\r\nAccept-Encoding: gzip, deflate\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nAccept-Language: en-us,en;q=0.5\r\n\r\n'
reply: 'HTTP/1.1 500 Internal Server Error\r\n'
header: Date header: Server header: Content-Length header: Connection header: Content-Type send: b'GET /dash/o2/n/planete/15005218_Swiat_wedlug_niemowlecia/ss/Manifest.ism?type=video&bitrate=$Bandwidth$&id=1 HTTP/1.1\r\nHost: n-1-18.dcs.redcdn.pl\r\nConnection: close\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)\r\nAccept-Encoding: gzip, deflate\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nAccept-Language: en-us,en;q=0.5\r\n\r\n'
reply: 'HTTP/1.1 500 Internal Server Error\r\n'
ERROR: giving up after 10 retries
  File "bin/youtube-dl", line 6, in <module>
    youtube_dl.main()
  File "/data/samba/youtube-dl/youtube_dl/__init__.py", line 444, in main
    _real_main(argv)
  File "/data/samba/youtube-dl/youtube_dl/__init__.py", line 434, in _real_main
    retcode = ydl.download(all_urls)
  File "/data/samba/youtube-dl/youtube_dl/YoutubeDL.py", line 1791, in download
    url, force_generic_extractor=self.params.get('force_generic_extractor', False))
  File "/data/samba/youtube-dl/youtube_dl/YoutubeDL.py", line 705, in extract_info
    return self.process_ie_result(ie_result, download, extra_info)
  File "/data/samba/youtube-dl/youtube_dl/YoutubeDL.py", line 751, in process_ie_result
    return self.process_video_result(ie_result, download=download)
  File "/data/samba/youtube-dl/youtube_dl/YoutubeDL.py", line 1435, in process_video_result
    self.process_info(new_info)
  File "/data/samba/youtube-dl/youtube_dl/YoutubeDL.py", line 1699, in process_info
    success = dl(filename, info_dict)
  File "/data/samba/youtube-dl/youtube_dl/YoutubeDL.py", line 1641, in dl
    return fd.download(name, info)
  File "/data/samba/youtube-dl/youtube_dl/downloader/common.py", line 353, in download
    return self.real_download(filename, info_dict)
  File "/data/samba/youtube-dl/youtube_dl/downloader/dash.py", line 73, in real_download
    if not process_segment(segment, ctx['tmpfilename'], i):
  File "/data/samba/youtube-dl/youtube_dl/downloader/dash.py", line 46, in process_segment
    success = ctx['dl'].download(target_filename, {'url': segment_url})
  File "/data/samba/youtube-dl/youtube_dl/downloader/common.py", line 353, in download
    return self.real_download(filename, info_dict)
  File "/data/samba/youtube-dl/youtube_dl/downloader/http.py", line 131, in real_download
    self.report_error('giving up after %s retries' % retries)
  File "/data/samba/youtube-dl/youtube_dl/downloader/common.py", line 162, in report_error
    self.ydl.report_error(*args, **kargs)
  File "/data/samba/youtube-dl/youtube_dl/YoutubeDL.py", line 568, in report_error
    self.trouble(error_message, tb)
  File "/data/samba/youtube-dl/youtube_dl/YoutubeDL.py", line 530, in trouble
    tb_data = traceback.format_list(traceback.extract_stack())

header: Date header: Server header: Content-Length header: Connection header: Content-Type 
@dstftw
Collaborator
dstftw commented Jan 4, 2017

Forbidden

You don't have permission to access /dash/o2/n/planete/15005218_Swiat_wedlug_niemowlecia/ss/Manifest.ism on this server.
Apache Server at n-1-18.dcs.redcdn.pl Port 80

@kkeiichi
kkeiichi commented Jan 5, 2017

Maybe this video is avaible only from my Country. I attach downloaded manifest.
As i said the problem is with "initialization" attribute of SegmentTemplate element. The "media" attribute is processed correctly and all parameters are replaced to values. In "initialization" attribute only $RepresentationID$ parameter was changed to value 1 and $Bandwidth$ parameter remains in created request.

In example request is:

GET /dash/o2/n/planete/15005218_Swiat_wedlug_niemowlecia/ss/Manifest.ism?type=video&bitrate=$Bandwidth$&id=1 HTTP/1.1

but should be:

GET /dash/o2/n/planete/15005218_Swiat_wedlug_niemowlecia/ss/Manifest.ism?type=video&bitrate=1600000&id=1 HTTP/1.1

Manifest.zip

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment