Crunchyroll: UnicodeDecodeError: 'ascii' codec can't decode byte

[yldrx]

When downloading a video from Crunchyroll, I encoutered this problem.

Cmdline: youtube-dl 'http://www.crunchyroll.com/case-closed/episode-754-the-tragedy-of-the-red-woman-steam-663019?p360=1' -F

version: 2014.11.13

[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['http://www.crunchyroll.com/case-closed/episode-754-the-tragedy-of-the-red-woman-steam-663019?p360=1', '-F', '--verbose']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2014.11.13
[debug] Python version 2.7.3
[debug] exe versions: ffmpeg 0.8.12, ffprobe 0.8.12
[debug] Proxy map: {'http': 'http://127.0.0.1:4324', u'https': 'http://127.0.0.1:4324'}
[Crunchyroll] 663019: Downloading webpage
[Crunchyroll] 663019: Downloading media info
[Crunchyroll] 663019: Downloading media info for 360p
[Crunchyroll] 663019: Downloading media info for 480p
[Crunchyroll] 663019: Downloading media info for 720p
[Crunchyroll] 663019: Downloading media info for 1080p
[Crunchyroll] 663019: Downloading subtitles for English (US)
Traceback (most recent call last):
  File "/usr/lib64/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/local/bin/youtube-dl/__main__.py", line 18, in <module>
  File "/usr/local/bin/youtube-dl/youtube_dl/__init__.py", line 351, in main

  File "/usr/local/bin/youtube-dl/youtube_dl/__init__.py", line 341, in _real_main

  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1094, in download
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 536, in extract_info
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 199, in extract
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/crunchyroll.py", line 262, in _real_extract
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/crunchyroll.py", line 108, in _decrypt_subtitles
  File "/usr/local/bin/youtube-dl/youtube_dl/utils.py", line 847, in intlist_to_bytes
UnicodeDecodeError: 'ascii' codec can't decode byte 0x9c in position 0: ordinal not in range(128)

Analysis

I've taken a look at the intlist_to_bytes function with my limited python knowledge, but was not able to fix the issue:
Things such as trying to set an encoding (utf-8) did not work out, and only changed the error, persumably because the bytes are consumed one by one by the chr function.

Also changing the function to essentially a return bytes(xs) did not work out, as the zlib.decode in the _decrypt_subtitles failed to work.

I'm not at all knowledgeable in python, but I think something is deep wrong with the intlist_to_bytes function. it should be an easy built-in python function, and should work with byte-strings or something. But who am I to say that. :)

Can you take a look at this?

[phihag]

Thank you for the report. You are correct that in Python one can usually use bytes objects (modulo a couple of different behaviors in Python 2 and Python 3, both of which we support).

However, crunchyroll's URL generation scheme is quite crazy and involves lots of numeric (not bitwise) operations, so it was more convenient to write code that deals with integers.

I have fixed the problem in youtube-dl 2014.11.13.1 and newer (type sudo youtube-dl -U to update). However, I suspect that crunchyroll downloads still won't work, because of #2759.

[yldrx]

Thanks for your attention.
I tried 2014.11.13.3, and there is a need to a code change.
(I'm using python 2.7 and seeing this problem)

error:

File "youtube_dl/utils.py", line 846, in intlist_to_bytes
TypeError: Struct() argument 1 must be string, not unicode

so on utils.py line 846, change it to:

struct.pack(b'%dB' % len(xs), *xs)

I only added a 'b' before the string.

[phihag]

Thanks, I've fixed this as well. Note that current 2.7 versions should work fine here, you seem to be running a somewhat older version.