Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cosmos with Neil DeGraqsse-Tyson from Fox: Unsupported URL #2541

Closed
jayseye opened this issue Mar 10, 2014 · 7 comments
Closed

Cosmos with Neil DeGraqsse-Tyson from Fox: Unsupported URL #2541

jayseye opened this issue Mar 10, 2014 · 7 comments

Comments

@jayseye
Copy link

@jayseye jayseye commented Mar 10, 2014

Hi, we may need someone in the U.S. to trace this, as I've been told that the video is only available in this country:

$ youtube-dl --verbose http://www.fox.com/watch/183733315515
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['--verbose', 'http://www.fox.com/watch/183733315515']
[debug] Encodings: locale 'UTF-8', fs 'UTF-8', out 'UTF-8', pref: 'UTF-8'
[debug] youtube-dl version 2014.03.10
[debug] Python version 2.6.6 - Linux-2.6.37.6-smp-i686-Intel-R-Core-TM-2_Duo_CPU_____T5450_@_1.66GHz-with-slackware-13.37.0
[debug] Proxy map: {}
[generic] 183733315515: Requesting header
WARNING: Falling back on generic information extractor.
[generic] 183733315515: Downloading webpage
[generic] 183733315515: Extracting information
ERROR: Unsupported URL: http://www.fox.com/watch/183733315515; please report this issue on https://yt-dl.org/bug . Be sure to call youtube-dl with the --verbose flag and include its complete output. Make sure you are using the latest version; type youtube-dl -U to update.
Traceback (most recent call last):
File "/usr/bin/ytd/youtube_dl/extractor/generic.py", line 277, in _real_extract
doc = xml.etree.ElementTree.fromstring(webpage.encode('utf-8'))
File "/usr/lib/python2.6/xml/etree/ElementTree.py", line 963, in XML
parser.feed(text)
File "/usr/lib/python2.6/xml/etree/ElementTree.py", line 1245, in feed
self._parser.Parse(data, 0)
ExpatError: not well-formed (invalid token): line 50, column 86
Traceback (most recent call last):
File "/usr/bin/ytd/youtube_dl/YoutubeDL.py", line 503, in extract_info
ie_result = ie.extract(url)
File "/usr/bin/ytd/youtube_dl/extractor/common.py", line 162, in extract
return self._real_extract(url)
File "/usr/bin/ytd/youtube_dl/extractor/generic.py", line 472, in _real_extract
raise ExtractorError('Unsupported URL: %s' % url)
ExtractorError: Unsupported URL: http://www.fox.com/watch/183733315515; please report this issue on https://yt-dl.org/bug . Be sure to call youtube-dl with the --verbose flag and include its complete output. Make sure you are using the latest version; type youtube-dl -U to update.

@jayseye
Copy link
Author

@jayseye jayseye commented Mar 10, 2014

Here's an alternate source for the video, at Hulu, also reported as Unsupported URL: http://www.hulu.com/cosmos-a-spacetime-odyssey . Would detailed output for that URL also be helpful?

@phihag
Copy link
Contributor

@phihag phihag commented Mar 10, 2014

Generally, URLs are enough for unsupported URL reports (we just mandate the full output to check that users don't use a really, really old version, despite the message telling them to update. You wouldn't believe how often that still happens.). However, Hulu and Fox use completely different services, so there should be two separate issues. There already is one for Hulu, #1629.

@jayseye
Copy link
Author

@jayseye jayseye commented Mar 10, 2014

Thanks for the quick response. I definitely understand how hard it can be to get folks to pay attention: have been a software developer for a long time, and also provided computer support and consulting. Anyway, please let me know if you need help tracing errors at either of these sites, as I live in the U.S. and use Python. And I'd really like to see this revival of Cosmos!

@pishposhmcgee
Copy link
Contributor

@pishposhmcgee pishposhmcgee commented Mar 12, 2014

The official Fox site (http://www.cosmosontv.com/watch/183733315515, which appears to be identical to http://www.fox.com/watch/183733315515) appears to use technology from "The Platform". I'm not familiar with that service, but it appears there's already an InfoExtractor in the youtube-dl repository.

If you dig through the Cosmos page source (at least while using the iPad useragent) you can find a link such as:
http://link.theplatform.com/s/fox.com/ex9dJjGhVsUu?mbr=true&feed=FAPI%20-%20Cosmos

Using that link in youtube-dl results in the following error:
[ThePlatform] ex9dJjGhVsUu: Downloading XML
[ThePlatform] ex9dJjGhVsUu: Downloading webpage
<Element '{http://www.w3.org/2005/SMIL21/Language}head' at 0x2de9710>
Traceback (most recent call last):
File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
"main", fname, loader, pkg_name)
File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/usr/local/bin/youtube-dl/main.py", line 18, in
File "/usr/local/bin/youtube-dl/youtube_dl/init.py", line 810, in main
File "/usr/local/bin/youtube-dl/youtube_dl/init.py", line 800, in _real_main
File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 992, in download
File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 500, in extract_info
File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 162, in extract
File "/usr/local/bin/youtube-dl/youtube_dl/extractor/theplatform.py", line 111, in _real_extract
File "/usr/local/bin/youtube-dl/youtube_dl/extractor/theplatform.py", line 69, in _get_info
KeyError: 'base'

Similarly, running 'curl' on the video URLs gives an "HTTP/1.0 405 Method Not Allowed" error. Not sure where to go from here.

As a possible (outside chance) incentive, the Jimmy Fallon player (Page: http://www.nbc.com/the-tonight-show/episodes/17, Platform link: http://link.theplatform.com/s/NnzsPC/H9nP0Gyh44jp?mbr=true&player=Onsite%20Player%20--%20No%20End%20Card&policy=43674) appears to both use the same service and give the same 405 errors.

@jayseye
Copy link
Author

@jayseye jayseye commented Apr 1, 2014

Here's a different site for Cosmos, also reporting Unsupported URL with the latest youtube-dl. Would you please let me know if you'd prefer a new Issue for this? thanks:
./youtube-dl --verbose http://www.cosmosontv.com/watch/211631683935
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['--verbose', 'http://www.cosmosontv.com/watch/211631683935']
[debug] Encodings: locale 646, fs 646, out 646, pref 646
[debug] youtube-dl version 2014.04.01.2
[debug] Python version 2.7.5 - OpenBSD-5.4-i386-32bit-ELF
[debug] Proxy map: {}
[generic] 211631683935: Requesting header
WARNING: Falling back on generic information extractor.
[generic] 211631683935: Downloading webpage
[generic] 211631683935: Extracting information
ERROR: Unsupported URL: http://www.cosmosontv.com/watch/211631683935; please report this issue onhttps://yt-dl.org/bug . Be sure to call youtube-dl with the --verbose flag and include its complee output. Make sure you are using the latest version; type youtube-dl -U to update.
Traceback (most recent call last):
File "./youtube-dl/youtube_dl/extractor/generic.py", line 379, in _real_extract
doc = parse_xml(webpage)
File "./youtube-dl/youtube_dl/utils.py", line 1315, in parse_xml
return xml.etree.ElementTree.XML(s.encode('utf-8'), **kwargs)
File "/fuguita/usr/local/lib/python2.7/xml/etree/ElementTree.py", line 1300, in XML
parser.feed(text)
File "/fuguita/usr/local/lib/python2.7/xml/etree/ElementTree.py", line 1642, in feed
self._raiseerror(v)
File "/fuguita/usr/local/lib/python2.7/xml/etree/ElementTree.py", line 1506, in _raiseerror
raise err
ParseError: not well-formed (invalid token): line 54, column 104
Traceback (most recent call last):
File "./youtube-dl/youtube_dl/YoutubeDL.py", line 511, in extract_info
ie_result = ie.extract(url)
File "./youtube-dl/youtube_dl/extractor/common.py", line 161, in extract
return self._real_extract(url)
File "./youtube-dl/youtube_dl/extractor/generic.py", line 612, in _real_extract
raise ExtractorError('Unsupported URL: %s' % url)
ExtractorError: Unsupported URL: http://www.cosmosontv.com/watch/211631683935; please report thisissue on https://yt-dl.org/bug . Be sure to call youtube-dl with the --verbose flag and include is complete output. Make sure you are using the latest version; type youtube-dl -U to update.

@pdball
Copy link

@pdball pdball commented Apr 2, 2014

My attempt at creating an IE for these videos:

$ youtube-dl -v http://www.cosmosontv.com/watch/211631683935
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['-v', 'http://www.cosmosontv.com/watch/211631683935']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2014.04.02
[debug] Git HEAD: fd6c3bf
[debug] Python version 2.6.6 - Linux-2.6.32-431.11.2.el6.x86_64-x86_64-with-redhat-6.5-Carbon
[debug] Proxy map: {}
[Cosmos] 211631683935: Downloading webpage
[Cosmos] 8OIyY3L411CO: Downloading XML
[Cosmos] 8OIyY3L411CO: Downloading webpage
ERROR: unable to download video data: HTTP Error 403: Forbidden
Traceback (most recent call last):
File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 996, in process_info
success = dl(filename, info_dict)
File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 972, in dl
return fd.download(name, info)
File "/usr/local/bin/youtube-dl/youtube_dl/downloader/common.py", line 290, in download
return self.real_download(filename, info_dict)
File "/usr/local/bin/youtube-dl/youtube_dl/downloader/http.py", line 54, in real_download
data = self.ydl.urlopen(request)
File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1200, in urlopen
return self._opener.open(req, timeout=self._socket_timeout)
File "/usr/lib64/python2.6/urllib2.py", line 397, in open
response = meth(req, response)
File "/usr/lib64/python2.6/urllib2.py", line 510, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib64/python2.6/urllib2.py", line 435, in error
return self._call_chain(_args)
File "/usr/lib64/python2.6/urllib2.py", line 369, in _call_chain
result = func(_args)
File "/usr/lib64/python2.6/urllib2.py", line 518, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 403: Forbidden

$ youtube-dl http://www.cosmosontv.com/watch/211631683935 --get-url
http://fbchdvod-f.akamaihd.net/z/Fox.com/100/719/Cosmos_YNF105c_2500.mp4?hdnea=st=1396455668~exp=1396455788~acl=/z/Fox.com/100/719/Cosmos_YNF105c_*~id=bbdd7131-e1d0-429f-9c8d-d4a9a8e71683~hmac=a8b4ca7224d42bc10f4decf521ac95ede183ffc6548ac31a0f4e7da15351442f

I don't know where to go from here. All suggestions are appreciated.

@remitamine
Copy link
Collaborator

@remitamine remitamine commented Feb 9, 2016

cosmosontv.com redirects to fox.com(youtube-dl can extract videos from fox.com).

@remitamine remitamine closed this Feb 9, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
6 participants
You can’t perform that action at this time.