Updates for Metadata 2.1 (PEP 566)

[di]

This PR makes the necessary updates for Metadata 2.1 (PEP 566):

• Handles the new Description-Content-Type and Provides-Extra fields
  • The former was already added in Add new long_description_content_type kwarg #1075, I made a slight tweak from Add setup.cfg support for long_description_content_type #1207 to only set this field if it is present
• Correctly sets the Version field to 2.1 if either of the new fields are present

[di]

Hi @jaraco, anything I can do to help get this merged? Since pkginfo 1.4.2 was released today, and since it only provides these fields if the Metadata-Version is 2.1, this PR is soon going to become a blocker for anyone wanting to use them. Thanks!

[jaraco]

These changes look to be backwards-compatible. Can you add an entry to the CHANGES.rst with an 0.1 bump from the last? Then we can pull that and cut a release.

[brainwane]

I created a virtualenv, installed @di's branches of twine & setuptools, added a description_content_type to the setup.py in my project, and created and uploaded a package to Test PyPI successfully. I have two questions:

1. The warning /usr/local/lib/python3.6/distutils/dist.py:261: UserWarning: Unknown distribution option: 'description_content_type' comes from readme_renderer, right? I presume it does because:

$ python setup.py check -r -s
/usr/local/lib/python3.6/distutils/dist.py:261: UserWarning: Unknown distribution option: 'description_content_type'
  warnings.warn(msg)
running check

2. The wheel building step still makes a DESCRIPTION.rst (Forms990_analysis-1.0.5.dev1.dist-info/DESCRIPTION.rst). This feels suboptimal to me given that, when I inspect the DESCRIPTION.rst here, it's my Markdown README. Should this be addressed in this PR?

Log:

(dustin-setuptools-test) sumanah@momentum:~/test/form990s$ python setup.py sdist bdist_wheel
/usr/local/lib/python3.6/distutils/dist.py:261: UserWarning: Unknown distribution option: 'description_content_type'
  warnings.warn(msg)
/home/sumanah/test/psf/setuptools/setuptools/dist.py:366: UserWarning: Normalizing '1.0.5dev1' to '1.0.5.dev1'
  normalized_version,
running sdist
running egg_info
writing Forms990_analysis.egg-info/PKG-INFO
writing dependency_links to Forms990_analysis.egg-info/dependency_links.txt
writing requirements to Forms990_analysis.egg-info/requires.txt
writing top-level names to Forms990_analysis.egg-info/top_level.txt
reading manifest file 'Forms990_analysis.egg-info/SOURCES.txt'
writing manifest file 'Forms990_analysis.egg-info/SOURCES.txt'
running check
creating Forms990-analysis-1.0.5.dev1
creating Forms990-analysis-1.0.5.dev1/Forms990_analysis.egg-info
copying files to Forms990-analysis-1.0.5.dev1...
copying README.rst -> Forms990-analysis-1.0.5.dev1
copying setup.cfg -> Forms990-analysis-1.0.5.dev1
copying setup.py -> Forms990-analysis-1.0.5.dev1
copying Forms990_analysis.egg-info/PKG-INFO -> Forms990-analysis-1.0.5.dev1/Forms990_analysis.egg-info
copying Forms990_analysis.egg-info/SOURCES.txt -> Forms990-analysis-1.0.5.dev1/Forms990_analysis.egg-info
copying Forms990_analysis.egg-info/dependency_links.txt -> Forms990-analysis-1.0.5.dev1/Forms990_analysis.egg-info
copying Forms990_analysis.egg-info/requires.txt -> Forms990-analysis-1.0.5.dev1/Forms990_analysis.egg-info
copying Forms990_analysis.egg-info/top_level.txt -> Forms990-analysis-1.0.5.dev1/Forms990_analysis.egg-info
Writing Forms990-analysis-1.0.5.dev1/setup.cfg
Creating tar archive
removing 'Forms990-analysis-1.0.5.dev1' (and everything under it)
running bdist_wheel
running build
installing to build/bdist.linux-x86_64/wheel
running install
running install_egg_info
Copying Forms990_analysis.egg-info to build/bdist.linux-x86_64/wheel/Forms990_analysis-1.0.5.dev1-py3.6.egg-info
running install_scripts
creating build/bdist.linux-x86_64/wheel/Forms990_analysis-1.0.5.dev1.dist-info/WHEEL
creating '/home/sumanah/test/form990s/dist/Forms990_analysis-1.0.5.dev1-py3-none-any.whl' and adding '.' to it
adding 'Forms990_analysis-1.0.5.dev1.dist-info/DESCRIPTION.rst'
adding 'Forms990_analysis-1.0.5.dev1.dist-info/metadata.json'
adding 'Forms990_analysis-1.0.5.dev1.dist-info/top_level.txt'
adding 'Forms990_analysis-1.0.5.dev1.dist-info/WHEEL'
adding 'Forms990_analysis-1.0.5.dev1.dist-info/METADATA'
adding 'Forms990_analysis-1.0.5.dev1.dist-info/RECORD'
(dustin-setuptools-test) sumanah@momentum:~/test/form990s$

[di]

The warning /usr/local/lib/python3.6/distutils/dist.py:261: UserWarning: Unknown distribution option: 'description_content_type' comes from readme_renderer, right? I presume it does because:

It comes from distutils: https://github.com/python/cpython/blob/8837dd092fe5ad5184889104e8036811ed839f98/Lib/distutils/dist.py#L273

It can be safely ignored.

The wheel building step still makes a DESCRIPTION.rst (Forms990_analysis-1.0.5.dev1.dist-info/DESCRIPTION.rst). This feels suboptimal to me given that, when I inspect the DESCRIPTION.rst here, it's my Markdown README.

This is a good catch, but I'm pretty sure this happens in wheel, not setuptools.

[di]

Yeah, here's the relevant lines in wheel: https://github.com/pypa/wheel/blob/master/wheel/bdist_wheel.py#L428-L436

I'm not really sure what that's used for since the description also exists as the message body in /METADATA as well, and IIRC that's where twine gets it from.

[di]

@jaraco Added a changelog entry, thanks!

[jaraco]

Releasing now as 38.6.0.

[di]

Oops, misread your request for a version bump. Thanks!

[wimglenn]

This caused a regression for me. PR incoming: #1293

[sayanarijit]

Hi

I notices it loads normal markdown as usual. But I believe lots of markdowns are in GFM (GitHub flavored) format. For e.g. it supports

print('Hello')

# ```
# print('Hello')
# ```

But not

print('Hello')

# ```python
# print('Hello')
# ```

We can think about code highlighting and all the advanced features later but for now I think just adding the support to render it as normal markdown would be nice.

[brainwane]

@sayanarijit Thank you for the comment! Several people have asked about Warehouse supporting GitHub-flavored Markdown -- for that let's talk in pypa/packaging-problems#126, as this is a request that spans a few different codebases.