prettytoml.tokens.errors.BadEscapeCharacter on Pipfile

[nicoxxl]

Issue description

There is a prettytoml.tokens.errors.BadEscapeCharacter when I did a pipenv install -r requirements.txt on this project https://github.com/cs01/gdbgui.

Expected result

Working stuff

Actual result

This :

Requirements file provided! Importing into Pipfile...
Traceback (most recent call last):
  File "/usr/bin/pipenv", line 11, in <module>
    load_entry_point('pipenv==2018.7.1', 'console_scripts', 'pipenv')()
  File "/usr/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/lib/python3.7/site-packages/pipenv/vendor/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/lib/python3.7/site-packages/pipenv/cli.py", line 435, in install
    selective_upgrade=selective_upgrade,
  File "/usr/lib/python3.7/site-packages/pipenv/core.py", line 1890, in do_install
    section = project.editable_packages if not dev else project.editable_dev_packages
  File "/usr/lib/python3.7/site-packages/pipenv/project.py", line 524, in editable_packages
    return self._get_editable_packages(dev=False)
  File "/usr/lib/python3.7/site-packages/pipenv/project.py", line 508, in _get_editable_packages
    for k, v in self.parsed_pipfile.get(section, {}).items()
  File "/usr/lib/python3.7/site-packages/pipenv/project.py", line 509, in <dictcomp>
    if is_editable(v)
  File "/usr/lib/python3.7/site-packages/pipenv/utils.py", line 584, in is_editable
    pipfile_entry.get(key) for key in ('file', 'path') + VCS_LIST
  File "/usr/lib/python3.7/site-packages/pipenv/utils.py", line 584, in <genexpr>
    pipfile_entry.get(key) for key in ('file', 'path') + VCS_LIST
  File "/usr/lib/python3.7/site-packages/pipenv/patched/prettytoml/elements/abstracttable.py", line 73, in get
    return self[key]
  File "/usr/lib/python3.7/site-packages/pipenv/patched/prettytoml/elements/abstracttable.py", line 66, in __getitem__
    for key, value in self.items():
  File "/usr/lib/python3.7/site-packages/pipenv/patched/prettytoml/elements/abstracttable.py", line 34, in items
    yield key.value, value.value
  File "/usr/lib/python3.7/site-packages/pipenv/patched/prettytoml/elements/atomic.py", line 40, in value
    return toml2py.deserialize(self._tokens[self._value_token_index()])
  File "/usr/lib/python3.7/site-packages/pipenv/patched/prettytoml/tokens/toml2py.py", line 33, in deserialize
    return _to_string(token)
  File "/usr/lib/python3.7/site-packages/pipenv/patched/prettytoml/tokens/toml2py.py", line 86, in _to_string
    return _unescape_str(escaped)
  File "/usr/lib/python3.7/site-packages/pipenv/patched/prettytoml/tokens/toml2py.py", line 46, in _unescape_str
    raise BadEscapeCharacter
prettytoml.tokens.errors.BadEscapeCharacter

Steps to replicate

Do a pipenv install -r requirements.txt on a dir with non ascii characters (I think it might be the source)

---

$ pipenv --support

Pipenv version: '2018.7.1'

Pipenv location: '/usr/lib/python3.7/site-packages/pipenv'

Python location: '/usr/bin/python'

Other Python installations in PATH:

• 2.7: /usr/bin/python2.7

• 2.7: /usr/bin/python2.7

• 3.7: /usr/bin/python3.7m

• 3.7: /usr/bin/python3.7

• 3.7.0: /usr/bin/python

• 2.7.15: /usr/bin/python2

• 3.7.0: /usr/bin/python3

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.7.0',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '4.17.14-arch1-1-ARCH',
 'platform_system': 'Linux',
 'platform_version': '#1 SMP PREEMPT Thu Aug 9 11:56:50 UTC 2018',
 'python_full_version': '3.7.0',
 'python_version': '3.7',
 'sys_platform': 'linux'}

System environment variables:

• LANG
• DISPLAY
• SHLVL
• LOGNAME
• XDG_VTNR
• PWD
• MOZ_PLUGIN_PATH
• ANT_HOME
• XDG_SESSION_ID
• GNOME_TERMINAL_SCREEN
• DESKTOP_SESSION
• XDG_SESSION_DESKTOP
• GDMSESSION
• GNOME_DESKTOP_SESSION_ID
• USERNAME
• WAYLAND_DISPLAY
• DBUS_SESSION_BUS_ADDRESS
• VTE_VERSION
• MAIL
• GNOME_TERMINAL_SERVICE
• GJS_DEBUG_TOPICS
• COLORTERM
• XDG_MENU_PREFIX
• OLDPWD
• GDM_LANG
• XDG_SESSION_TYPE
• GRADLE_HOME
• GJS_DEBUG_OUTPUT
• TERM
• SESSION_MANAGER
• XDG_CURRENT_DESKTOP
• SSH_AUTH_SOCK
• PATH
• QT_IM_MODULE
• HOME
• XDG_SEAT
• XMODIFIERS
• SHELL
• XDG_RUNTIME_DIR
• USER
• _
• PYTHONDONTWRITEBYTECODE
• PIP_PYTHON_PATH

Pipenv–specific environment variables:

Debug–specific environment variables:

• PATH: /usr/local/bin:/usr/local/sbin:/usr/bin:/opt/cuda/bin:/usr/lib/jvm/default/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl
• SHELL: /usr/bin/zsh
• LANG: fr_FR.UTF-8
• PWD: /home/nicoxxl/Projets/Mèche/gdbgui

---

Contents of Pipfile ('/home/nicoxxl/Projets/Mèche/gdbgui/Pipfile'):

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
gdbgui = {editable = true, file = "file:///home/nicoxxl/Projets/M\xe8che/gdbgui"}

[dev-packages]

[requires]
python_version = "3.7"

[[source]]
url = "https://pypi.org/simple/"
verify_ssl = true

[uranusjr]

As you may have noticed in the syntax highlighted block, the \x sequence with red background is causing the problem. Try modifying it to the actual path (with the è character).

[nicoxxl]

It didn't solve the root cause which is that the Pipfile is automatically generated from my pipenv install -r requirements.txt, plus the Pipfile is re-written.

[techalchemy]

pipenv as it is doesn't currently support non-unicode paths, we have another issue open on the topic, but that is absolutely the problem here.

[frostming]

@nicoxxl It is unfortunate but TOML spec doesn't support \x strings. It used to, but has removed the support on later spec version.

The problem is that TOML encoder should convert the string into unicode.