UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 46: ordinal not in range(128)

[dcgc]

Traceback (most recent call last):
      File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 162, in _run_module_as_main
      "__main__", fname, loader, pkg_name)
      File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 72, in _run_code
        exec code in run_globals
      File "/Users/ostv/eclipse_workspace/VerificarVideos/youtube-dl.py/__main__.py", line 18, in <module>
      File "/Users/ostv/eclipse_workspace/VerificarVideos/youtube-dl.py/youtube_dl/__init__.py", line 837, in main
      File "/Users/ostv/eclipse_workspace/VerificarVideos/youtube-dl.py/youtube_dl/__init__.py", line 696, in _real_main
    UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 46: ordinal not in range(128)

That input happens when I introduce the following command:

youtube-dl.py -o video.mov https://www.youtube.com/watch?v=hROC2Am_WBo --no-check-certificates --verbose

If the name of the file (video.mov) and the youtube address (https://www.youtube.com/watch?v=hROC2Am_WBo) are retrieved from the command that simulates the download of the address, it works fine, if I retrieve the name of the file and youtube address from a csv file (all of this is done in the Java Programming language). I really don't understand how I can have a UnicodeEncodeError, when the text is exactly the same. I'm confused.

[jaimeMF]

We need the whole output you get using the --verbose option, In this case the first line are important, something like:

[debug] System config: []
[debug] User config: ['--keep-video', '--write-info-json', '--write-thumbnail', '--verbose']
[debug] Command-line args: ['http://www.youtube.com/watch?v=BaW_jenozKc', '--verbose']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2014.04.30.1
[debug] Git HEAD: 10c0e2d
[debug] Python version 3.4.0 - Darwin-13.1.0-x86_64-i386-64bit
[debug] Proxy map: {}
[youtube] Setting language
[youtube] BaW_jenozKc: Downloading webpage
[youtube] BaW_jenozKc: Downloading video info webpage
[youtube] BaW_jenozKc: Extracting video information
[info] Writing video description metadata as JSON to: youtube-dl test video ''_ä↭𝕐-BaW_jenozKc.info.json
[youtube] BaW_jenozKc: Downloading thumbnail ...
[youtube] BaW_jenozKc: Writing thumbnail to: youtube-dl test video ''_ä↭𝕐-BaW_jenozKc.jpg
[download] Resuming download at byte 523264
[download] Destination: youtube-dl test video ''_ä↭𝕐-BaW_jenozKc.mp4
[download] 100% of 1.53MiB in 00:01

[jaimeMF]

And make sure you are using the last version.

[dcgc]

I'm using the latest version of youtube-dl and I was using the verbose option. Here's the output as requested:

[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['-o', '/Volumes/emissao/VV180-pastas/TRANSFERIDOS/The\xc2\xa0Dodos|Good.mov', 'https://www.youtube.com/watch?v=hROC2Am_WBo', '--no-check-certificate', '--verbose']
Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/Users/ostv/eclipse_workspace/VerificarVideos/youtube-dl.py/__main__.py", line 18, in <module>
  File "/Users/ostv/eclipse_workspace/VerificarVideos/youtube-dl.py/youtube_dl/__init__.py", line 837, in main
  File "/Users/ostv/eclipse_workspace/VerificarVideos/youtube-dl.py/youtube_dl/__init__.py", line 696, in _real_main
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 46: ordinal not in range(128)```

[dcgc]

How do you define encoding to UTF-8 in the command? I tried but the option -encoding doesn't seem to work.

Because it's using the ascii codec to check on ascii extended characters.

[montanaflynn]

I just hit this issue as well:

$ youtube-dl --verbose -g "https://www.youtube.com/watch?v=5Yj-eHhyano"
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'--verbose', u'-g', u'https://www.youtube.com/watch?v=5Yj-eHhyano']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2017.06.05
[debug] Python version 2.7.12 - Darwin-16.6.0-x86_64-i386-64bit
[debug] exe versions: ffmpeg 3.2, ffprobe 3.2, rtmpdump 2.4
[debug] Proxy map: {}
ERROR: Error in output template: 'ascii' codec can't encode characters in position 4-5: ordinal not in range(128) (encoding: 'UTF-8')
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 676, in prepare_filename
    filename = expand_path(outtmpl % template_dict)
  File "/usr/local/bin/youtube-dl/youtube_dl/utils.py", line 546, in expand_path
    return os.path.expandvars(compat_expanduser(s))
  File "/usr/local/bin/youtube-dl/youtube_dl/compat.py", line 2689, in compat_expanduser
    pwent = pwd.getpwnam(path[1:i])
UnicodeEncodeError

[Hubbitus]

I have similar issue:

$ youtube-dl 'https://www.youtube.com/watch?v=LpP4Q4-mQV0' --verbose
[debug] System config: ['--prefer-free-formats']
[debug] User config: ['--abort-on-error', '--xattr-set-filesize', '--no-overwrites', '--continue', '--write-description', '--abort-on-unavailable-fragment', '--prefer-free-formats', '--merge-output-format', 'mkv', '--add-metadata', '--xattrs', '-o', '%(playlist)s.%(playlist_index)s %(title)s.%(ext)s', '-f', 'best[width<=720]/best[filesize<=500M]', '--exec', 'rename "NA.NA " "" NA.NA* || [ $? -eq 4 ] #{}']
[debug] Custom config: []
[debug] Command-line args: ['https://www.youtube.com/watch?v=LpP4Q4-mQV0', '--verbose']
WARNING: Assuming --restrict-filenames since file system encoding cannot encode all characters. Set the LC_ALL environment variable to fix this.
[debug] Encodings: locale ANSI_X3.4-1968, fs ascii, out ANSI_X3.4-1968, pref ANSI_X3.4-1968
[debug] youtube-dl version 2017.05.29
[debug] Python version 3.5.3 - Linux-4.11.6-201.fc25.x86_64-x86_64-with-fedora-25-Twenty_Five
[debug] exe versions: ffmpeg 3.1.9, ffprobe 3.1.9
[debug] Proxy map: {}
[youtube] LpP4Q4-mQV0: Downloading webpage
[youtube] LpP4Q4-mQV0: Downloading video info webpage
[youtube] LpP4Q4-mQV0: Extracting video information
[youtube] LpP4Q4-mQV0: Downloading MPD manifest
[youtube] LpP4Q4-mQV0: Downloading MPD manifest
[info] Video description is already present
[debug] Invoking downloader on 'https://r3---sn-vcpu-axqe.googlevideo.com/videoplayback?pcm2=no&gir=yes&beids=%5B9466593%5D&source=youtube&pl=20&dur=0.000&mt=1498806164&mv=m&ei=C_hVWZcJhaRgqMaQ-As&id=o-AF1_7UYHgncuxCo1Ueb4pW0zP7ZknQWkPJRCut1BvABR&ms=au&mm=31&signature=898ABA2BFC8CC7972C5A4E6786E6BBCEB4CBC98F.61F2BB4617888EF4B2E889F2FD1C4EBFAC7AF5A4&mn=sn-vcpu-axqe&key=yt6&ip=85.114.14.170&itag=43&initcwndbps=2372500&lmt=1427626586757229&sparams=clen%2Cdur%2Cei%2Cgir%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpcm2%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cexpire&expire=1498827883&ratebypass=yes&ipbits=0&mime=video%2Fwebm&clen=111211098&requiressl=yes'
[download] NA.NA _.webm has already been downloaded
[download] 100% of 106.06MiB
[ffmpeg] Adding metadata to 'NA.NA _.webm'
[debug] ffmpeg command line: ffmpeg -y -i 'file:NA.NA _.webm' -c copy -metadata 'description=   ,             .  , ,  ,        .       . http://landwatersun.ru/viewtopic.php?id=293' -metadata 'title=      ' -metadata date=20150329 -metadata 'comment=   ,             .  , ,  ,        .       . http://landwatersun.ru/viewtopic.php?id=293' -metadata 'purl=https://www.youtube.com/watch?v=LpP4Q4-mQV0' -metadata artist=landwatersun 'file:NA.NA _.temp.webm'
Traceback (most recent call last):
  File "/usr/bin/youtube-dl", line 9, in <module>
    load_entry_point('youtube-dl==2017.5.29', 'console_scripts', 'youtube-dl')()
  File "/usr/lib/python3.5/site-packages/youtube_dl/__init__.py", line 464, in main
    _real_main(argv)
  File "/usr/lib/python3.5/site-packages/youtube_dl/__init__.py", line 454, in _real_main
    retcode = ydl.download(all_urls)
  File "/usr/lib/python3.5/site-packages/youtube_dl/YoutubeDL.py", line 1895, in download
    url, force_generic_extractor=self.params.get('force_generic_extractor', False))
  File "/usr/lib/python3.5/site-packages/youtube_dl/YoutubeDL.py", line 770, in extract_info
    return self.process_ie_result(ie_result, download, extra_info)
  File "/usr/lib/python3.5/site-packages/youtube_dl/YoutubeDL.py", line 824, in process_ie_result
    return self.process_video_result(ie_result, download=download)
  File "/usr/lib/python3.5/site-packages/youtube_dl/YoutubeDL.py", line 1538, in process_video_result
    self.process_info(new_info)
  File "/usr/lib/python3.5/site-packages/youtube_dl/YoutubeDL.py", line 1876, in process_info
    self.post_process(filename, info_dict)
  File "/usr/lib/python3.5/site-packages/youtube_dl/YoutubeDL.py", line 1941, in post_process
    files_to_delete, info = pp.run(info)
  File "/usr/lib/python3.5/site-packages/youtube_dl/postprocessor/ffmpeg.py", line 465, in run
    self.run_ffmpeg_multiple_files(in_filenames, temp_filename, options)
  File "/usr/lib/python3.5/site-packages/youtube_dl/postprocessor/ffmpeg.py", line 199, in run_ffmpeg_multiple_files
    p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, stdin=subprocess.PIPE)
  File "/usr/lib64/python3.5/subprocess.py", line 676, in __init__
    restore_signals, start_new_session)
  File "/usr/lib64/python3.5/subprocess.py", line 1221, in _execute_child
    restore_signals, start_new_session, preexec_fn)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 12-23: ordinal not in range(128)

[pythonoma]

I overridden the prepare_filename method to fix similar issue regarding filenames (Too long filenames > 255 unicode chars) with this

class MyYoutubeDLM(youtube_dl.YoutubeDL):

    def prepare_filename(self, info_dict):
        sanitized_path = super(MyYoutubeDLM, self).prepare_filename(info_dict)
        short_sanitized_path = self.restrict_file_name_length(sanitized_path)
        return short_sanitized_path


    def restrict_file_name_length(self, sanitized_path):
        filename_with_ext = os.path.basename(sanitized_path)
        filename_without_ext = os.path.splitext(filename_with_ext)[0]
        ext = os.path.splitext(filename_with_ext)[1]

        filename_encoded = filename_without_ext.encode('utf-8')
        
        filename_encoded_short = filename_encoded[:240]
        filename_decoded_short = filename_encoded_short.decode('utf-8', errors='replace') + ext

        short_sanitized_path = os.path.join(os.path.dirname(sanitized_path), filename_decoded_short)
        return short_sanitized_path

then you use your class instead of YoutubeDL

 with MyYoutubeDLM(ydl_opts) as ydl:
        print('============================')
        print('Downloading Youtube videos...')
        ydl.download([url])

[kenorb]

Try running:

export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8

and re-run the command, I think it should solve the issue.

See also: UnicodeEncodeError: 'ascii' codec can't encode character.

[snowman]

nice dude, you do a great job @kenorb