Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

2018.11.14 regression: UnicodeEncodeError: 'ascii' codec can't encode character u'\u2718' in position 0: ordinal not in range(128) #3223

Closed
jfly opened this issue Nov 15, 2018 · 8 comments

Comments

@jfly
Copy link
Contributor

@jfly jfly commented Nov 15, 2018

When running pipenv install --system --deploy with pipenv 2018.11.14 on Ubuntu 16.04, I see the following crash:

Traceback (most recent call last):
  File "/usr/local/bin/pipenv", line 7, in <module>
    from pipenv import cli
  File "/usr/local/lib/python2.7/dist-packages/pipenv/__init__.py", line 47, in <module>
    from .cli import cli
  File "/usr/local/lib/python2.7/dist-packages/pipenv/cli/__init__.py", line 3, in <module>
    from .command import cli
  File "/usr/local/lib/python2.7/dist-packages/pipenv/cli/command.py", line 18, in <module>
    from .. import environments
  File "/usr/local/lib/python2.7/dist-packages/pipenv/environments.py", line 278, in <module>
    PIPENV_SPINNER_FAIL_TEXT = fs_str(to_text(u"✘ {0}")) if not PIPENV_HIDE_EMOJIS else ("{0}")
  File "/usr/local/lib/python2.7/dist-packages/pipenv/vendor/vistir/compat.py", line 182, in fs_str
    return string.encode(_fs_encoding)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2718' in position 0: ordinal not in range(128)

(What follows is some light investigation)

This new PIPENV_SPINNER_FAIL_TEXT variable seems to have been introduced in 4dac167. The crash is happening in fs_str here:

def fs_str(string):
"""Encodes a string into the proper filesystem encoding
Borrowed from pip-tools
"""
if isinstance(string, str):
return string
assert not isinstance(string, bytes)
return string.encode(_fs_encoding)
. This crash seems to be happening because sys.getfilesystemencoding() returns "ANSI_X3.4-1968" on Ubuntu 16.04:

➜  ~ docker run -it ubuntu:16.04
root@72aab1a9c1e6:/# apt-get update && apt-get install python 
...
root@72aab1a9c1e6:/# python
Python 2.7.12 (default, Nov 12 2018, 14:36:49) 
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys; print(sys.getfilesystemencoding())
ANSI_X3.4-1968
>>> u'\u2718'.encode("ANSI_X3.4-1968")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2718' in position 0: ordinal not in range(128)

Hope this helps!

@frostming
Copy link
Contributor

@frostming frostming commented Nov 15, 2018

We are sorry for the trouble. As a workaround, you can set PIPENV_HIDE_EMOJIS=1.

@techalchemy
Copy link
Member

@techalchemy techalchemy commented Nov 15, 2018

What is your locale setting? Do you have LANG set? I think I’m using a better approach in most places...

@techalchemy
Copy link
Member

@techalchemy techalchemy commented Nov 15, 2018

Seriously need to consolidate the output code

@jfly
Copy link
Contributor Author

@jfly jfly commented Nov 15, 2018

This is an unaltered ubuntu:16.04 image from dockerhub, so you should be able to play with this locally if you've got docker! Some info for you:

➜  ~ docker run -it ubuntu:16.04
root@8aa33a1ac5e7:/# echo $LANG

root@8aa33a1ac5e7:/# env
HOSTNAME=8aa33a1ac5e7
TERM=xterm
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PWD=/
SHLVL=1
HOME=/root
_=/usr/bin/env
root@8aa33a1ac5e7:/# locale -a
C
C.UTF-8
POSIX
root@8aa33a1ac5e7:/# locale
LANG=
LANGUAGE=
LC_CTYPE="POSIX"
LC_NUMERIC="POSIX"
LC_TIME="POSIX"
LC_COLLATE="POSIX"
LC_MONETARY="POSIX"
LC_MESSAGES="POSIX"
LC_PAPER="POSIX"
LC_NAME="POSIX"
LC_ADDRESS="POSIX"
LC_TELEPHONE="POSIX"
LC_MEASUREMENT="POSIX"
LC_IDENTIFICATION="POSIX"
LC_ALL=
@techalchemy
Copy link
Member

@techalchemy techalchemy commented Nov 15, 2018

Until this release did this work at all?

@techalchemy
Copy link
Member

@techalchemy techalchemy commented Nov 15, 2018

Context: pipenv has required LANG to be set forever as far as I recall...

techalchemy added a commit that referenced this issue Nov 15, 2018
- Add spiner symbols to unicode translation map
- Update pythonfinder to master (ignore vendor update for now)
- Fixes #3223
- Fixes #3224

Signed-off-by: Dan Ryan <dan@danryan.co>
@jfly
Copy link
Contributor Author

@jfly jfly commented Nov 15, 2018

Yeah, things worked on pipenv 2018.10.13 and before.

Working

➜  ~/tmp/pipenv-3223 cat Dockerfile
FROM ubuntu:16.04

RUN apt-get update && apt-get install -y python python-pip

RUN pip install pipenv==2018.10.13
➜  ~/tmp/pipenv-3223 docker run $(docker build -q .) pipenv
Usage: pipenv [OPTIONS] COMMAND [ARGS]...
...

Broken

➜  ~/tmp/pipenv-3223 cat Dockerfile 
FROM ubuntu:16.04

RUN apt-get update && apt-get install -y python python-pip

RUN pip install pipenv==2018.11.14
➜  ~/tmp/pipenv-3223 docker run $(docker build -q .) pipenv
Traceback (most recent call last):
  File "/usr/local/bin/pipenv", line 7, in <module>
    from pipenv import cli
  File "/usr/local/lib/python2.7/dist-packages/pipenv/__init__.py", line 47, in <module>
    from .cli import cli
  File "/usr/local/lib/python2.7/dist-packages/pipenv/cli/__init__.py", line 3, in <module>
    from .command import cli
  File "/usr/local/lib/python2.7/dist-packages/pipenv/cli/command.py", line 18, in <module>
    from .. import environments
  File "/usr/local/lib/python2.7/dist-packages/pipenv/environments.py", line 278, in <module>
    PIPENV_SPINNER_FAIL_TEXT = fs_str(to_text(u"✘ {0}")) if not PIPENV_HIDE_EMOJIS else ("{0}")
  File "/usr/local/lib/python2.7/dist-packages/pipenv/vendor/vistir/compat.py", line 182, in fs_str
    return string.encode(_fs_encoding)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2718' in position 0: ordinal not in range(128)
@techalchemy
Copy link
Member

@techalchemy techalchemy commented Nov 15, 2018

(Waiting on a clean build of that PR but eventually it should be the right approach)

christek91 added a commit to christek91/pipenv that referenced this issue Nov 16, 2018
- Add spiner symbols to unicode translation map
- Update pythonfinder to master (ignore vendor update for now)
- Fixes pypa#3223
- Fixes pypa#3224

Signed-off-by: Dan Ryan <dan@danryan.co>
@techalchemy techalchemy added this to the November Bugfix Release milestone Nov 19, 2018
opnfv-github pushed a commit to opnfv/fuel that referenced this issue Nov 27, 2018
While at it, fix emoji issues with latest virtualenv [1].

JIRA: FUEL-398

[1] pypa/pipenv#3223

Change-Id: Ice5937222bf75c1ddadc6b9f1994635bc10faf57
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
opnfv-github pushed a commit to opnfv/fuel that referenced this issue Dec 1, 2018
While at it, fix emoji issues with latest virtualenv [1].

JIRA: FUEL-398

[1] pypa/pipenv#3223

Change-Id: Ice5937222bf75c1ddadc6b9f1994635bc10faf57
Signed-off-by: Alexandru Avadanii <Alexandru.Avadanii@enea.com>
(cherry picked from commit f03a33b)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.