Undisplayable \u sequence during pipenv install

[uranusjr]

> pipenv install
Creating a virtualenv for this project…
Using real prefix 'c:\\users\\uranusjr\\appdata\\local\\programs\\python\\python35'
New python executable in C:\Users\uranusjr\.virtualenvs\pipenv-QZcxVy6Y\Scripts\python.exe
Installing setuptools, pip, wheel...done.

Virtualenv location: C:\Users\uranusjr\.virtualenvs\pipenv-QZcxVy6Y
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Updated Pipfile.lock!
Installing dependencies from Pipfile.lock…
  ================================ 0/0 \u2014 00:00:00
To activate this project's virtualenv, run the following:
 $ pipenv shell

\u2014 is probably an en-dash, but is somehow not escaped. This likely happens only on Windows (and non-unicode consoles). I'm on Windows 10 + code page 932 (Japanese).

[kakulukia]

Try updating to the newest version .. it shall look like this now:

Installing dependencies from Pipfile.lock…
🐍 ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 23/57 — 00:00:18

[uranusjr]

I am at latest. You are not on Windows command prompt (it can’t display the snake either but that’s known and not fixable).

[kennethreitz]

Windows doesn't display the snake.

[kennethreitz]

I'm aware of the escape sequence. Will try to fix. Was waiting for someone to complain :)

[erinxocon]

@kennethreitz Did you address this? I can't reproduce using Powershell.exe and cmd.exe.

[nateprewitt]

@erinxocon did you change the character encoding for your shell when you tried to reproduce. This is an issue with Windows-31J having different character mappings than UTF-8, so U+2014 isn't a visually representable character.

It looks like moving from "Em Dash" to U+2015 "Horizontal Bar", which looks identical, may work across both encodings.

[erinxocon]

Ahh no I did not change my character encoding.

[uranusjr]

@nateprewitt Switching to U+2015 would fix things on CP932, but cause trouble on other code pages. CP950, for example, has U+2014 but not U+2015. Maybe it would be better to simply detect sys.stdout.encoding, and fallback to - (hyphen) when it’s not utf-8.

This would be more forward-compatible as well, since Python 3.6+ enforces UTF-8 output on Windows (PEP 528), and displays the en-dash fine.

[nateprewitt]

@uranusjr thanks for filling in the knowledge gaps here 😄! So that seems sensible to me although enforcing this uniformly across future strings may be tricky. I know Kenneth has a strong leaning towards using the em dash over hyphens, so it's likely we'll see more of them popping up in the future.

We could probably look at adding a utility function that does this though and adding it were it's needed. This seems like a pretty small edge case, but I do want to make sure we're keeping a quality user experience for as many users as reasonably possible.

[uranusjr]

There’s an app a library for that! https://pypi.python.org/pypi/Unidecode

>>> import unidecode
>>> unidecode.unidecode('\u2014')
'--'
>>> unidecode.unidecode('\u2015')
'--'

[kennethreitz]

i'm just going to make it a dash on windows.

[sirex]

Similar thing on Archlinux:

I'm using pipenv 10.0.0 version.

[jrd]

Same here, the first character is not available on any font…

[kennethreitz]

install emoji fonts

[sirex]

On Arch I fixed that by installing ttf-symbola:

sudo pacman -S ttf-symbola

Result:

[sirex]

https://bugs.archlinux.org/task/57649

[barthalion]

It definitely should be a dash by default, not just on Windows. You can't assume availability of emoji font on the system and dependency of CLI tool on font package is weird at best.

[kennethreitz]

emoji are normal nowadays — just fix your terminal

[eli-schwartz]

Oh well, if this was running on an iPhone where people look forward to new versions of iOS to see which new emoji Apple has added, you might have a point.

As far as desktop systems are concerned, are you telling us that:

• Arch Linux is required to install some arbitrary emoji font as part of the base group, or
• our pipenv package should depend on some arbitrary emoji font, or
• we should undergo a distro policy change whereby our font packages declare that they provide emojis, so we can sanely depend on them, or
• our pipenv package should work just like the PyPi version, inasmuch as users who were never told that emoji fonts are an obligatory part of every operating system on the planet without except will be startled to see undisplayable characters in their terminal when running CLI tools possibly used for server administration.

[kennethreitz]

this really isn't open for discussion, but i appreciate the feedback

[Foxboron]

@kennethreitz As a downstream package maintainer for pipenv, please come with a recommendation for what font you develop with and target. Arbitrarily depending on emoji fonts won't really help me much.

[Sim4n6]

I installed ttf-ancient-fonts-symbola and it is working on Debian 9

apt install ttf-ancient-fonts-symbola

or even better

apt install fonts-noto-color-emoji