Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UnicodeDecodeError on Windows(Locales where UTF-8 is not the default encoding) #23

Closed
fukusuket opened this issue Jul 6, 2023 · 0 comments · Fixed by #24
Closed

UnicodeDecodeError on Windows(Locales where UTF-8 is not the default encoding) #23

fukusuket opened this issue Jul 6, 2023 · 0 comments · Fixed by #24

Comments

@fukusuket
Copy link
Contributor

fukusuket commented Jul 6, 2023

Hello, Thank you for maintaining the tool :) There was an error about encoding, so I report it.

Describe the issue
An error occurs on Windows in locales where UTF-8 is not the default encoding.

Step to Reproduce

pip3 install git+https://github.com/MISP/PyTaxonomies
python
>>> from pytaxonomies import Taxonomies
>>> taxonomies = Taxonomies()

Actual behavior
UnicodeDecodeError occur as follows.

C:\Users\fukusuke>python
Python 3.11.4 (tags/v3.11.4:d2340ef, Jun  7 2023, 05:45:37) [MSC v.1934 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from pytaxonomies import Taxonomies
>>> taxonomies = Taxonomies()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\fukusuke\AppData\Local\Programs\Python\Python311\Lib\site-packages\pytaxonomies\api.py", line 257, in __init__
    self.manifest = self.loader(manifest_path)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\fukusuke\AppData\Local\Programs\Python\Python311\Lib\site-packages\pytaxonomies\api.py", line 282, in __load_path
    return json.load(f)
           ^^^^^^^^^^^^
  File "C:\Users\fukusuke\AppData\Local\Programs\Python\Python311\Lib\json\__init__.py", line 293, in load
    return loads(fp.read(),
                 ^^^^^^^^^
UnicodeDecodeError: 'cp932' codec can't decode byte 0x93 in position 15191: illegal multibyte sequence

Expected behavior
UnicodeDecodeError does not occur.

Environment

  • OS: Windows 11
  • Python: 3.11.4
PS C:\Users\fukusuke> [System.Text.Encoding]::Default
BodyName          : iso-2022-jp
EncodingName      : 日本語 (シフト JIS)
HeaderName        : iso-2022-jp
WebName           : shift_jis
WindowsCodePage   : 932
IsBrowserDisplay  : True
IsBrowserSave     : True
IsMailNewsDisplay : True
IsMailNewsSave    : True
IsSingleByte      : False
EncoderFallback   : System.Text.InternalEncoderBestFitFallback
DecoderFallback   : System.Text.InternalDecoderBestFitFallback
IsReadOnly        : True
CodePage          : 932
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant