Unable to load non-ASCII metadata on Python 2

[jaraco]

In GitLab by @jaraco on Dec 5, 2018, 18:32

Attempting to load non-ascii metadata on Python 2.7 fails with a UnicodeEncodeError and this traceback:

    desc = self._find_local_distribution().metadata['Description']
.tox/python/lib/python2.7/site-packages/importlib_metadata/api.py:71: in metadata
    self.read_text('METADATA') or self.read_text('PKG-INFO')
/opt/python/2.7.15/lib/python2.7/email/__init__.py:57: in message_from_string
    return Parser(*args, **kws).parsestr(s)
/opt/python/2.7.15/lib/python2.7/email/parser.py:82: in parsestr
    return self.parse(StringIO(text), headersonly=headersonly)
E   UnicodeEncodeError: 'ascii' codec can't encode character u'\xb7' in position 905: ordinal not in range(128)

Discovered this on the portend project in this job, which is using pytest-checkdocs to ensure the long-description is valid ReStructuredText.

[jaraco]

In GitLab by @jaraco on Dec 5, 2018, 18:35

Replicate the issue readily thus:

$ python2.7 -m pip-run portend importlib_metadata -- -c "import importlib_metadata; importlib_metadata.metadata('portend')"
Collecting portend
  Using cached https://files.pythonhosted.org/packages/81/43/21afd5914b74d4271184ee76f4093b45aa6a580dc6627d72dfc33664c6ac/portend-2.3-py2.py3-none-any.whl
Collecting importlib_metadata
  Using cached https://files.pythonhosted.org/packages/d3/50/9c436e4ac464f16e271e34e6c79b9c753e7e1a80b0ebd86da9b99de182ea/importlib_metadata-0.7-py2.py3-none-any.whl
Collecting tempora>=1.8 (from portend)
  Using cached https://files.pythonhosted.org/packages/6a/73/22900a52243fdcb2251a10bdb7c6a75fc8d40ab59ec25c01e26823af5126/tempora-1.14-py2.py3-none-any.whl
Collecting pathlib2; python_version < "3" (from importlib_metadata)
  Using cached https://files.pythonhosted.org/packages/2a/46/c696dcf1c7aad917b39b875acdc5451975e3a9b4890dca8329983201c97a/pathlib2-2.3.3-py2.py3-none-any.whl
Collecting contextlib2; python_version < "3" (from importlib_metadata)
  Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl
Collecting pytz (from tempora>=1.8->portend)
  Using cached https://files.pythonhosted.org/packages/f8/0e/2365ddc010afb3d79147f1dd544e5ee24bf4ece58ab99b16fbb465ce6dc0/pytz-2018.7-py2.py3-none-any.whl
Collecting jaraco.functools>=1.20 (from tempora>=1.8->portend)
  Using cached https://files.pythonhosted.org/packages/c2/f3/ce1bd8d5434227d26c2c2e07a9f89229a6b81122437452871d7e67f0ec76/jaraco.functools-1.20-py2.py3-none-any.whl
Collecting six (from tempora>=1.8->portend)
  Using cached https://files.pythonhosted.org/packages/67/4b/141a581104b1f6397bfa78ac9d43d8ad29a7ca43ea90a2d863fe3056e86a/six-1.11.0-py2.py3-none-any.whl
Collecting scandir; python_version < "3.5" (from pathlib2; python_version < "3"->importlib_metadata)
Collecting more-itertools (from jaraco.functools>=1.20->tempora>=1.8->portend)
  Using cached https://files.pythonhosted.org/packages/fb/d3/77f337876600747ae307ea775ff264c5304a691941cd347382c7932c60ad/more_itertools-4.3.0-py2-none-any.whl
Collecting backports.functools-lru-cache>=1.0.3; python_version == "2.7" (from jaraco.functools>=1.20->tempora>=1.8->portend)
  Using cached https://files.pythonhosted.org/packages/03/8e/2424c0e65c4a066e28f539364deee49b6451f8fcd4f718fefa50cc3dcf48/backports.functools_lru_cache-1.5-py2.py3-none-any.whl
Installing collected packages: pytz, six, more-itertools, backports.functools-lru-cache, jaraco.functools, tempora, portend, scandir, pathlib2, contextlib2, importlib-metadata
Successfully installed backports.functools-lru-cache-1.5 contextlib2-0.5.5 importlib-metadata-0.7 jaraco.functools-1.20 more-itertools-4.3.0 pathlib2-2.3.3 portend-2.3 pytz-2018.7 scandir-1.9.0 six-1.11.0 tempora-1.14
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-run-gx8RI6/importlib_metadata/api.py", line 95, in metadata
    return Distribution.from_name(package).metadata
  File "/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pip-run-gx8RI6/importlib_metadata/api.py", line 71, in metadata
    self.read_text('METADATA') or self.read_text('PKG-INFO')
  File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/email/__init__.py", line 57, in message_from_string
    return Parser(*args, **kws).parsestr(s)
  File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/email/parser.py", line 82, in parsestr
    return self.parse(StringIO(text), headersonly=headersonly)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xb7' in position 905: ordinal not in range(128)

[jaraco]

In GitLab by @jaraco on Dec 5, 2018, 18:59

mentioned in commit 514f317433996d70d3919048c9a4abfbd857aa84

[jaraco]

In GitLab by @jaraco on Dec 5, 2018, 19:07

mentioned in commit 3568e65a642e4e27ca6dd1e13419b120ca4b73fb

[jaraco]

In GitLab by @jaraco on Dec 5, 2018, 19:07

mentioned in commit 523e04a48b79a3878cdaece1244f88c81db3ee7b

[jaraco]

In GitLab by @jaraco on Dec 5, 2018, 19:18

mentioned in commit 7b569f9725f2e109b6f628af502dfb83fefde537

[jaraco]

In GitLab by @jaraco on Dec 5, 2018, 19:53

mentioned in commit ea520ac

[jaraco]

In GitLab by @jaraco on Dec 5, 2018, 19:53

mentioned in commit d58bf2a

[jaraco]

In GitLab by @jaraco on Dec 5, 2018, 19:53

mentioned in commit c687f37

[jaraco]

In GitLab by @jaraco on Dec 6, 2018, 03:08

closed via merge request !30

[jaraco]

In GitLab by @jaraco on Dec 6, 2018, 03:08

closed via commit c687f37