Project description does not display on PyPI, although present in metadata

[vsajip]

Describe the bug

Under the "Project description" for distlib 0.3.1, no description is shown - instead,

The author of this package has not provided a project description

is shown.

Expected behavior

The description hasn't changed since the previous release, 0.3.0. Under that release, the description shows

Low-level components of distutils2/packaging, augmented with higher-level APIs for making packaging easier.

To Reproduce

As given above under "Describe the bug".

My Platform

The problem isn't platform-specific.

Additional context

The uploaded files contain the relevant metadata under the "Description:" key in PKG-INFO (source archive) and METADATA (wheel).

[jamadden]

What does twine check tell you?

[di]

Looking at the 0.3.1 wheel, it looks like pkginfo is not able to read the Description for some reason:

>>> from pkginfo import Wheel
>>> w = Wheel('distlib-0.3.1-py2.py3-none-any.whl')
>>> w.description
''

[vsajip]

$ twine check dist/distlib-0.3.1.zip
Checking dist/distlib-0.3.1.zip: PASSED, with warnings
  warning: `long_description_content_type` missing. defaulting to `text/x-rst`.
$ twine check dist/distlib-0.3.1-py2.py3-none-any.whl
Checking dist/distlib-0.3.1-py2.py3-none-any.whl: FAILED
  `long_description` has syntax errors in markup and would not be rendered on PyPI.
  warning: `long_description_content_type` missing. defaulting to `text/x-rst`.

The distlib project uses a simple long_description in setup.py - here's what's passed to setup();

long_description = ('Low-level components of distutils2/packaging, '
                    'augmented with higher-level APIs for making '
                    'packaging easier.'),

This has remained unchanged over many previous releases, so the problem would appear to be due to a regression. It's not clear that that simple string would lead to RST markup errors (given just that input, for example, Sphinx merely gives a warning, not an error - Explicit markup ends without a blank line; unexpected unindent.

Perhaps one should not be forced to update old code with plain-text descriptions to use Markdown or ReST.

[vsajip]

Also, parsing that bare description text with docutils appears to give no errors/warnings:

$ docutils-parse test.rst
<document: <paragraph...>>
  <paragraph: <#text: 'Low-level comp ...'>>
    <#text: 'Low-level components of distutils2/packaging, augmented with hig ...'>
$ cat test.rst
Low-level components of distutils2/packaging, augmented with higher-level APIs for making packaging easier.

[jamadden]

(tl;dr: distlib is using an older metadata format and placing the description in a header, not the body. pkginfo only looks at the body, not the header, in wheel files. This may be a bug in pkginfo.)

It looks like distlib itself was used to generate that wheel:

$  cat cat -p  distlib-0.3.1.dist-info/WHEEL
Wheel-Version: 1.0
Generator: distlib 0.3.1.dev0
Root-Is-Purelib: true
Tag: py2-none-any
Tag: py3-none-any

Compared with twine:

$ cat -p cat -p twine-3.2.0.dist-info/WHEEL
Wheel-Version: 1.0
Generator: bdist_wheel (0.34.2)
Root-Is-Purelib: true
Tag: py3-none-any

The METADATA for the distlib wheel is an older version, and specifies the description as a header:

$  cat -p distlib-0.3.1.dist-info/METADATA
Metadata-Version: 1.1
Name: distlib
Version: 0.3.1
Summary: Distribution utilities
Description: Low-level components of distutils2/packaging, augmented with higher-level APIs for making packaging easier.
Home-page: https://bitbucket.org/pypa/distlib
Author: Vinay Sajip
Author-email: vinay_sajip@red-dove.com
License: Python license
Classifier: Development Status :: 5 - Production/Stable
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Python Software Foundation License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 2
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 2.7
Classifier: Programming Language :: Python :: 3.5
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Download-URL: https://bitbucket.org/pypa/distlib/downloads/distlib-0.3.1.zip

Again comparing with twine, where the long-description data is in the body of the METADATA:

$ cat -p twine-3.2.0.dist-info/METADATA
Metadata-Version: 2.1
Name: twine
Version: 3.2.0
Summary: Collection of utilities for publishing packages on PyPI
Home-page: https://twine.readthedocs.io/
Author: Donald Stufft and individual contributors
Author-email: donald@stufft.io
License: UNKNOWN
Project-URL: Packaging tutorial, https://packaging.python.org/tutorials/distributing-packages/
Project-URL: Travis CI, https://travis-ci.org/pypa/twine/
Project-URL: Twine documentation, https://twine.readthedocs.io/en/latest/
Project-URL: Twine source, https://github.com/pypa/twine/
Platform: UNKNOWN
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Natural Language :: English
Classifier: Operating System :: MacOS :: MacOS X
Classifier: Operating System :: POSIX
Classifier: Operating System :: POSIX :: BSD
Classifier: Operating System :: POSIX :: Linux
Classifier: Operating System :: Microsoft :: Windows
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: Implementation :: CPython
Requires-Python: >=3.6
Requires-Dist: pkginfo (>=1.4.2)
Requires-Dist: readme-renderer (>=21.0)
Requires-Dist: requests (>=2.20)
Requires-Dist: requests-toolbelt (!=0.9.0,>=0.8.0)
Requires-Dist: setuptools (>=0.7.0)
Requires-Dist: tqdm (>=4.14)
Requires-Dist: keyring (>=15.1)
Requires-Dist: rfc3986 (>=1.4.0)
Requires-Dist: colorama (>=0.4.3)
Requires-Dist: importlib-metadata ; python_version < "3.8"

.. image:: https://img.shields.io/pypi/v/twine.svg
    :target: https://pypi.org/project/twine

.. image:: https://img.shields.io/pypi/pyversions/twine.svg
    :target: https://pypi.org/project/twine

.. image:: https://img.shields.io/readthedocs/twine
    :target: https://twine.readthedocs.io

.. image:: https://img.shields.io/travis/com/pypa/twine
    :target: https://travis-ci.org/pypa/twine
…

According to the specification for description, the data can either be in the Description field, or the body. This was a change in version 2.1 of the specification.

In a wheel file only the body is being looked at regardless of the metadata version. (Compare this to the sdist parsing, which will take the body in precedence to the description header, but only if it is non-empty.)

so the problem would appear to be due to a regression.

The previous release of distlib only uploaded a .zip file. The description can be found in the PKG-INFO file found in that .zip file (as well as the .zip file of 0.3.1):

$ pkginfo ./distlib-0.3.1.zip | grep description
description: Low-level components of distutils2/packaging, augmented with higher-level APIs for making packaging easier.

But the wheel file's description is empty, apparently due to the discrepancy noted above:

$ pkginfo ./distlib-0.3.1-py2.py3-none-any.whl | grep description
description:

The first file uploaded sets the metadata for a release. So if the .whl was uploaded first, that would explain why there's no description for this release.

[di]

The wheel was uploaded first.

[vsajip]

Good detective work. I actually did the upload using a single twine command and a wildcard, which led to the wheel being picked before the zip.

Since the spec allows the description to be in the Description field and it is there, this does seem to point to a problem in pkginfo.

Is there any way I can ask someone to update the description on PyPI to the correct value?

[di]

Is there any way I can ask someone to update the description on PyPI to the correct value?

Sorry, as a matter of policy we don't manually edit any metadata of any release on PyPI.

One alternative would be making a postrelease and yanking the existing release.

[di]

I'm going to close this as it's clear the bug is in pkginfo and not with PyPI, but let us know if you need any additional help here. Thanks for the report!