UnicodeEncodeError: 'charmap' codec can't encode character '\x80' in position 23385: character maps to <undefined>

[JoshuaFern]

Running itch-dl on Python 3.11.1 installed via pip on WIndows 11 powershell.

...
INFO:root:Downloading page 49 (found 2400 keys total)
INFO:root:Downloading page 50 (found 2450 keys total)
INFO:root:Downloading page 51 (found 2500 keys total)
INFO:root:Downloading page 52 (found 2550 keys total)
INFO:root:Downloading page 53 (found 2600 keys total)
INFO:root:Fetched 2648 download keys.
Games:   0%|                                                                               | 0/49557 [00:00<?, ?game/s]INFO:root:Downloading https://dkelroqur.itch.io/black-jumper
https://img.itch.zone/aW1hZ2UvMzMwNzcvMTQxODU4LnBuZw==/original/oKU282.png: 100%|██| 19.8k/19.8k [00:00<00:00, 142kB/s]
Traceback (most recent call last):NzcvMTQxODU4LnBuZw==/original/oKU282.png: 100%|██| 19.8k/19.8k [00:00<00:00, 142kB/s]
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\Proxy_ydkavi9\AppData\Local\Programs\Python\Python311\Scripts\itch-dl.exe\__main__.py", line 7, in <module>
  File "C:\Users\Proxy_ydkavi9\AppData\Local\Programs\Python\Python311\Lib\site-packages\itch_dl\cli.py", line 81, in run
    return drive_downloads(jobs, download_to, args.mirror_web, settings, keys, parallel=args.parallel)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Proxy_ydkavi9\AppData\Local\Programs\Python\Python311\Lib\site-packages\itch_dl\downloader.py", line 353, in drive_downloads
    results = [downloader.download(job) for job in tqdm(jobs, **tqdm_args)]
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Proxy_ydkavi9\AppData\Local\Programs\Python\Python311\Lib\site-packages\itch_dl\downloader.py", line 353, in <listcomp>
    results = [downloader.download(job) for job in tqdm(jobs, **tqdm_args)]
               ^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Proxy_ydkavi9\AppData\Local\Programs\Python\Python311\Lib\site-packages\itch_dl\downloader.py", line 324, in download
    f.write(site.prettify())
  File "C:\Users\Proxy_ydkavi9\AppData\Local\Programs\Python\Python311\Lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeEncodeError: 'charmap' codec can't encode character '\x80' in position 23385: character maps to <undefined>
Games:   0%|                                                                               | 0/49557 [00:01<?, ?game/s]

As a workaround I've been running the script under Ubuntu WSL2 where it seems to be working fine.

[DragoonAethis]

Fix for this tagged along with 0.3.3 - I've tested a similar scenario (Win11, Python 3.11.1 from Scoop, PowerShell, installed in a venv with pip) and game downloads look good now. Can you confirm if it works correctly for you now?

[JoshuaFern]

It works! No more error and the Cyrillic text is no longer garbled on the archived site as mentioned in #8

Closing as completed.