Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[bug] UnicodeEncodeError upon dynaconf init #664

Closed
ebenh opened this issue Oct 5, 2021 · 8 comments
Closed

[bug] UnicodeEncodeError upon dynaconf init #664

ebenh opened this issue Oct 5, 2021 · 8 comments
Assignees

Comments

@ebenh
Copy link

ebenh commented Oct 5, 2021

Describe the bug

dynaconf init -f yaml results in a UnicodeEncodeError

To Reproduce
Steps to reproduce the behavior:

  1. git clone -b dynaconf https://github.com/ebenh/django-flex-user.git
  2. py -m pipenv install --dev
  3. py -m pipenv shell
  4. export DJANGO_SETTINGS_MODULE=test_project.settings
  5. dynaconf init -f yaml

Error Message

Traceback (most recent call last):
  File "C:\Users\eben\AppData\Local\Programs\Python\Python37\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Users\eben\AppData\Local\Programs\Python\Python37\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\eben\.virtualenvs\django-flex-user-ab_cVlY8\Scripts\dynaconf.exe\__main__.py", line 7, in <module>
  File "c:\users\eben\.virtualenvs\django-flex-user-ab_cvly8\lib\site-packages\dynaconf\vendor\click\core.py", line 221, in __call__
    def __call__(A,*B,**C):return A.main(*B,**C)
  File "c:\users\eben\.virtualenvs\django-flex-user-ab_cvly8\lib\site-packages\dynaconf\vendor\click\core.py", line 205, in main
    H=E.invoke(F)
  File "c:\users\eben\.virtualenvs\django-flex-user-ab_cvly8\lib\site-packages\dynaconf\vendor\click\core.py", line 345, in invoke
    with C:return F(C.command.invoke(C))
  File "c:\users\eben\.virtualenvs\django-flex-user-ab_cvly8\lib\site-packages\dynaconf\vendor\click\core.py", line 288, in invoke
    if A.callback is not _A:return ctx.invoke(A.callback,**ctx.params)
  File "c:\users\eben\.virtualenvs\django-flex-user-ab_cvly8\lib\site-packages\dynaconf\vendor\click\core.py", line 170, in invoke
    with G:return A(*B,**E)
  File "c:\users\eben\.virtualenvs\django-flex-user-ab_cvly8\lib\site-packages\dynaconf\vendor\click\decorators.py", line 21, in A
    def A(*A,**B):return f(get_current_context(),*A,**B)
  File "c:\users\eben\.virtualenvs\django-flex-user-ab_cvly8\lib\site-packages\dynaconf\cli.py", line 257, in init
    click.echo("\u2699\ufe0f  Configuring your Dynaconf environment")
  File "c:\users\eben\.virtualenvs\django-flex-user-ab_cvly8\lib\site-packages\dynaconf\vendor\click\utils.py", line 82, in echo
    if A:B.write(A)
  File "C:\Users\eben\AppData\Local\Programs\Python\Python37\lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode characters in position 0-1: character maps to <undefined>

@rochacbruno
Copy link
Member

I suspect this problem is related to the emojis on https://github.com/rochacbruno/dynaconf/blob/master/dynaconf/cli.py#L283

We need to find a way to safely output the emojis on windows, make it optional or remove it completelly.

@endersonmenezes
Copy link
Contributor

I'm trying to reproduce this, and will go study to correct.

@endersonmenezes
Copy link
Contributor

endersonmenezes commented Oct 7, 2021

image

This issue seems to have been fixed in latest Windows updates. microsoft/terminal#190

I'm researching if this fix is only in Newer Builds. (I'm using Windows 11), and seeing how we can work around to not break the execution in older builds.

@ebenh What your windows version for me reproduce this in Virtual Machine?

@ebenh
Copy link
Author

ebenh commented Oct 7, 2021

@ebenh What your windows version for me reproduce this in Virtual Machine?

Windows 7 x64

@endersonmenezes
Copy link
Contributor

endersonmenezes commented Oct 8, 2021

@ebenh you are using python 37, 38 or 39?
I will testing with 3.7 :)

Edit: I managed to reproduce the error with Python379 on Windows 7 Ultimate x64, I will try to bypass it.

@endersonmenezes
Copy link
Contributor

@ebenh to bypass this error i found this solution:

image

You can use:

set PYTHONIOENCODING=utf-8

or

export PYTHONIOENCODING=utf-8

Source: Stackoverflow


@rochacbruno We can add this env directly into Dynaconf or put a manual for use on Windows systems with older builds, I think this can also happen on Windows 8 and Windows 8.1.
What do you prefer?

@rochacbruno
Copy link
Member

@endersonmenezes does it work if we put os.environ["PYTHONIOENCODING"] = "utf-8" on the top of the CLI file? of not then we can just add a note on the docs about this problem.

Or as a second option, remove all the emojis from the CLI output?

What do you think?

Probably getting rid of emojis is safer option pypa/pipenv#3131

@endersonmenezes
Copy link
Contributor

endersonmenezes commented Oct 8, 2021

I think we can keep the emojis and use PYTHONIOENCODING, as our String is passed by IO, just when it's not able to decode it will remove it, so we have the two solutions:

  • Emojis removed where they don't work.
  • Emojis working for those who like to put emojis on everything (like me) 😄

I think os.environ["PYTHONIOENCODING"] = "utf-8" the change will not change anything for newer devices that already have this by default, and helps users more, makes a learning curve for using dynaconf shorter, i can make a PR?

EDIT: The result would be similar to this change, simpler. https://github.com/pypa/pipenv/pull/3132/files

rochacbruno added a commit that referenced this issue Apr 15, 2022
Shortlog of commits since last release:

    Anderson Sousa (1):
          Document the usage with python -m (#710)

    Andressa Cabistani (2):
          Add unique label when merging lists to fix issue #653 (#661)
          Add new validation to fix issue #585 (#667)

    Armin Berres (1):
          Fix typo in error message

    Bruno Rocha (7):
          Release version 3.1.7
          Found this bug that was duplicating the generated envlist (#663)
          Add support for Python 3.10 (#665)
          Attempt to fix #555 (#669)
          Create update_contributors.yml
          Fixing pre-coomit and docs CI
          Added `dynaconf get` command to cli (#730)

    Caneco (2):
          improvement: add brand new logo to the project (#686)
          improvement: update socialcard to match the python way (#687)

    EdwardCuiPeacock (2):
          Feature: add @Jinja and @Format casting (#704)
          Combo converter doc (#735)

    Eitan Mosenkis (1):
          Fix FlaskConfig.setdefault (#706)

    Enderson Menezes (Mr. Enderson) (2):
          Force PYTHONIOENCODING to utf-8 to fix #664 (#672)
          edit: move discussions to github tab (#682)

    Eugene Triguba (1):
          Fix custom prefix link in envvar documentation (#680)

    Gibran Herrera (1):
          Fix Issue 662 Lazy validation (#675)

    Jitendra Yejare (2):
          Load vault secrets from environment less stores or which are not written by dynaconf (#725)
          Use default value when settings is blank (#729)

    Pavel Alimpiev (1):
          Update docs link (#678)

    Ugo Benassayag (1):
          Added validate_only_current_env to validator (issue #734) (#736)

    Waylon Walker (1):
          Docs Fix Spelling (#696)

    dependabot[bot] (3):
          Bump django from 2.1.5 to 2.2.26 in /example/django_pytest_pure (#711)
          Bump mkdocs from 1.1.2 to 1.2.3 (#715)
          Bump django from 2.2.26 to 2.2.27 in /example/django_pytest_pure (#717)

    github-actions[bot] (2):
          [automated] Update Contributors File (#691)
          [automated] Update Contributors File (#732)

    lowercase00 (1):
          Makes Django/Flask kwargs case insensitive (#721)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants