UnicodeEncodeError

[pearl-yu]

Hi,

Anyone got this error before? UnicodeEncodeError: 'charmap' codec can't encode character '\U0001f60e' in position 12: character maps to .

Attaching the full error message below:

Traceback (most recent call last):
  File "C:\Users\youmi\anaconda3\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\youmi\anaconda3\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\Users\youmi\anaconda3\Scripts\twitch-dl.exe\__main__.py", line 7, in <module>
  File "C:\Users\youmi\anaconda3\lib\site-packages\twitchdl\console.py", line 321, in main
    args.func(args)
  File "C:\Users\youmi\anaconda3\lib\site-packages\twitchdl\commands\download.py", line 170, in download
    download_one(video_id, args)
  File "C:\Users\youmi\anaconda3\lib\site-packages\twitchdl\commands\download.py", line 176, in download_one
    return _download_video(video_id, args)
  File "C:\Users\youmi\anaconda3\lib\site-packages\twitchdl\commands\download.py", line 273, in _download_video
    print_out("Found: <blue>{}</blue> by <yellow>{}</yellow>".format(
  File "C:\Users\youmi\anaconda3\lib\site-packages\twitchdl\output.py", line 61, in print_out
    print(*args, **kwargs)
  File "C:\Users\youmi\anaconda3\lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\U0001f60e' in position 12: character maps to <undefined>

[garoto]

VoD id?

[pearl-yu]

Hi,

Thanks for the reply.
For example:

1783928313
1783171981
1817180599

[ihabunek]

Well it's something to do with emoji, here's the titles:

1783171981 - Short Stream For Solo CC Finals 🏆
1783928313 - $100 JackRiper Duo Tourney w/ Synthetic 🏆
1817180599 - NITE O'Clock 🕰️

It tries to encode them to cp1252 which fails. I would expect it to default to UTF-8. It's a windows-specific bug.

Could you please run twitch-dl env and post the results here?

[pearl-yu]

I see. Thx. Do you think there's a workaorund?

twitch-dl 2.1.3
Platform: Windows-10-10.0.22621-SP0
Python 3.10.9 | packaged by Anaconda, Inc. | (main, Mar 1 2023, 18:18:15) [MSC v.1916 64 bit (AMD64)]

I'm downloading batches of videos, in a jupyter notebook using subprocess.

[ihabunek]

You can try running chcp 65001 and then run twitch-dl.

Found some other workarounds here: https://stackoverflow.com/a/57134096/84245

I don't have a windows machine so can't test.

[garoto]

I encountered this same issue some months ago (Windows 8.1+Python 3.8.2), when piping twitch-dl output to a grep-like program. Setting the environmental variable PYTHONIOENCODING to UTF-8 solved it for me (set "PYTHONIOENCODING=UTF-8").

[ihabunek]

@garoto Thanks. Another option is setting PYTHONUTF8=1 to enable the UTF-8 mode.

Docs:

• https://docs.python.org/3/using/cmdline.html#envvar-PYTHONIOENCODING
• https://docs.python.org/3/using/cmdline.html#envvar-PYTHONUTF8

[garoto]

@garoto Thanks. Another option is setting PYTHONUTF8=1 to enable the UTF-8 mode.

PYTHONUTF8=1 also works. Nice find, thanks.

[pearl-yu]

Thanks a lot! Setting PYTHONUTF8=1 or UTF-8 works for me too.