rtve.es:alacarta not working. Unable to download webpage.

[helq]

[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['--verbose', 'http://www.rtve.es/alacarta/videos/ciudad-k/ciudad-20100927-2131/888631/']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2015.02.19.1
[debug] Python version 3.4.2 - Linux-3.18.6-1-ARCH-x86_64-with-arch
[debug] exe versions: ffmpeg 2.5.4, ffprobe 2.5.4, rtmpdump 2.4
[debug] Proxy map: {}
[rtve.es:alacarta] 888631: Downloading JSON metadata
[rtve.es:alacarta] 888631: Downloading url information
[rtve.es:alacarta] 888631: Getting video url
ERROR: Unable to download webpage: HTTP Error 404: Not Found (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "/usr/lib/python3.4/site-packages/youtube_dl/extractor/common.py", line 308, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/usr/lib/python3.4/site-packages/youtube_dl/YoutubeDL.py", line 1589, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python3.4/urllib/request.py", line 461, in open
    response = meth(req, response)
  File "/usr/lib/python3.4/urllib/request.py", line 571, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.4/urllib/request.py", line 499, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.4/urllib/request.py", line 433, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.4/urllib/request.py", line 579, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)``` 

[helq]

Hi, thank you for your incredible software! And sorry for only coming here to post about the support of one uninteresting page. ^^

[yan12125]

I've partially fixed the extraction in yan12125/youtube-dl@24523c5. Seems rtve.es has totally changed how stuff works, so I rewrote rtve.py. I didn't issue a pull request primarily because in my codes Pycrypto is used for Blowfish encryption and decryption. If someone can provide a Blowfish implementation that:

• Implemented in pure python
• Compatible with python 2.6~python3.4
• Compatible with the license of youtube-dl

I'll issue a pull request to merge my changes.

It's also possible to use the openssl command for crytographic jobs like what this page says, but I don't think it's a good idea.

[jaimeMF]

For a few videos I have tested (the current tests and some from http://www.rtve.es/documentales/), the current method works.
I implemented it using the mobile player and I think that video was not availabe from mobile. I guess you have used the flash player, that's why it looks completely different.

[yan12125]

Yes - I implement the algorithm based on the flash player. If someone can provide a Blowfish implementation, I think it will be great to merge the two approaches.

[yan12125]

I've checked out how Android Apps works and seems they're using AES now. The good news is that I can integrate the algorithm into youtube-dl. See yan12125/youtube-dl@0eede00. The bad news is that videos got on Android is smaller. For exmaple, for http://www.rtve.es/alacarta/videos/el-ministerio-del-tiempo/emdt-6-280315/3066559/, I got a 1024x576, 772.31MiB file using the BlowFish version, and got a 640x368, 379.41MiB file with the AES version.

[jaimeMF]

@yan12125 with e9f65f8 we extract the best quality version of the video. Your branch doesn't work for me, I get HTTP Error 403: Forbidden in the downloader (maybe it's because I'm downloading from Spain and the process is different).

[yan12125]

@jaimeMF Which video are you testing?

By the way, upon commit e9f65f8, I've tested several UA strings for the video proposed by @helq (with --user-agent). Some UA strings works (outside Spain):

Mozilla/5.0 (Linux; Android 4.4.4; GT-N7000 Build/KTU84Q) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/33.0.0.0 Mobile Safari/537.36
Mozilla/5.0 (Linux; U; Android 4.0.3; ko-kr; LG-L160L Build/IML74K) AppleWebkit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30
Mozilla/5.0 (iPhone; U; CPU iPhone OS4_0 like Mac OS X; en-us) AppleWebKit/532.9 (KHTML, like Gecko) Version/4.0.5 Mobile/8A293 Safari/6531.22.7
Apache-HttpClient/UNAVAILABLE (java 1.4)
Mobile
java
iPhone
iOS

While some gives me a 404 error:

Opera/12.02 (Android 4.1; Linux; Opera Mobi/ADR-1111101157; U; en-US) Presto/2.9.201 Version/12.02
Dalvik/1.6.0 (Linux; U; Android 4.4.4; GT-N7000 Build/KTU84Q)
Mozilla/5.0 (Linux; U; Android 4.4.4; en-US; GT-N7000 Build/KTU84Q) RTVE.es | Móvil/1.6.6
Mozilla/5.0 (Windows NT 6.3; rv:36.0) Gecko/20100101 Firefox/36.0
Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36
Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; AS; rv:11.0) like Gecko
Android

Note that in successful cases, the downloaded file is smaller and have lower resolutions in comparison with the file downloaded with my fork.

I guess the reason is that the current algorithm is based on the mobile app, so some videos cannot be fetched unless you claim you're using a mobile device. Nevertheless, changing user agent strings is a reasonable workaround.