New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

long_description_content_type=text/markdown doesn't always work #3664

Closed
simonw opened this Issue Apr 13, 2018 · 18 comments

Comments

Projects
None yet
7 participants
@simonw
Copy link

simonw commented Apr 13, 2018

This page does not show correctly rendered markdown: https://pypi.org/project/datasette/

2018-04-13 at 2 47 pm

The setup.py file looks like this:

https://github.com/simonw/datasette/blob/fb988ace7c7e2bee5ac142a0eab22431d0675a77/setup.py#L13-L18

setup(
    name='datasette',
    description='An instant JSON API for your SQLite databases',
    long_description=get_long_description(),
    long_description_content_type='text/markdown',
    author='Simon Willison',

According to @ambv on Twitter this is caused by a bug: https://twitter.com/llanga/status/984866099886895104 "There's a bug now where you need to upload just the tar.gz first with Twine and only then upload the rest of the files."

@ambv

This comment has been minimized.

Copy link

ambv commented Apr 13, 2018

@di will know which bug I'm talking about.

@di

This comment has been minimized.

Copy link
Member

di commented Apr 13, 2018

Yep, this is due to building with an old version of wheel which incorrectly sets the Metadata-Version field. Should be fixed by a pip install wheel==0.31.0 and then building/uploading a new distribution.

@simonw

This comment has been minimized.

Copy link
Author

simonw commented Apr 14, 2018

That worked - I ran pip install wheel==0.31.0 before doing python setup.py bdist_wheel and the result was this page with the markdown correctly formatted: https://pypi.org/project/datasette/

@simonw simonw closed this Apr 14, 2018

@rgieseke

This comment has been minimized.

Copy link

rgieseke commented Apr 24, 2018

I might have a similar problem: https://pypi.org/project/shortcountrynames/
The markdown rendering worked for me in other projects and I think I have wheel=0.31.0

Requirement already up-to-date: wheel in ./venv/lib/python3.6/site-packages (from -r requirements.txt (line 3)) (0.31.0)

Repo: https://github.com/rgieseke/shortcountrynames

@di

This comment has been minimized.

Copy link
Member

di commented Apr 24, 2018

@rgieseke Lookng at the latest release for that project, the Description-Content-Type field is not making it to PyPI.

Can you confirm that you have setuptools>=38.6.0 and twine>=1.11.0 as well?

@rspeer

This comment has been minimized.

Copy link

rspeer commented Apr 24, 2018

I have the same problem: https://pypi.org/project/ConceptNet/5.6.2/ . Repo: https://github.com/commonsense/conceptnet5

I have setuptools 39.0.1, twine 1.11.0, and wheel 0.31.0, and I ran python setup.py sdist upload.

I even deleted ConceptNet 5.6.1 and incremented the version to 5.6.2 when I updated all the above dependencies. However, I'm not just going to keep incrementing the version number for the sake of PyPI.

@rgieseke

This comment has been minimized.

Copy link

rgieseke commented Apr 24, 2018

@di Updating setuptools did the trick! Thanks for the quick reply and so much for the PyPI update!

@rspeer You can use the https://test.pypi.org/ instance to test as well ... (and delete tags later or re-try)

@rspeer

This comment has been minimized.

Copy link

rspeer commented Apr 24, 2018

Apparently python setup.py sdist upload is the problem. I will open a new issue about the fact that it's extremely non-obvious that you have to use "twine". (I know there's documentation that says so... but it's not the documentation that's linked to when you're looking for how to set your project description.)

@di

This comment has been minimized.

Copy link
Member

di commented Apr 24, 2018

@rspeer Can you link us to the documentation that showed you to use python setup.py sdist upload?

@rspeer

This comment has been minimized.

Copy link

rspeer commented Apr 24, 2018

https://docs.python.org/3.6/distutils/packageindex.html#the-upload-command

Also a decade of habit. If you want to change people's habits, you could at least tell them they need to change.

@rspeer

This comment has been minimized.

Copy link

rspeer commented Apr 24, 2018

To clarify something: I'm not claiming that PyPI is linking to the Python documentation. PyPI links to packaging.python.org, of course. However, one way that people learn about PyPI in the first place is the Python documentation, where PyPI is described as that place that setup.py upload uploads to.

If you want to update people's understanding about how to upload Python packages, talking about it on your own blog and mentioning Twine twice on a very long documentation page isn't enough. You need to make getting the new tools part of the new PyPI user experience.

Lawouach added a commit to chaostoolkit/chaostoolkit that referenced this issue Sep 19, 2018

Fix pypi readme rendering
As per pypa/warehouse#3664

Signed-off-by: Sylvain Hellegouarch <sh@defuze.org>

heilaaks added a commit to heilaaks/snippy that referenced this issue Oct 17, 2018

Add pyproject.toml for build system dependencies
The intention is to provide requirements for the Python
installer. There are features in the setup process which
require specific setuptools, wheels and pip versions.

Currently it seems that also older versions are able to
handle the install process.

The topic is complicated and not fully understood by
the author. But the intention is to provide more robust
and well defined installation and to understand the
concept better.

Required functionalities

PEP 508 defines python_version as one of the environment
markers which requires setuptools 36.2.0 (released Jul 13,
2017) or later [1]. This is used to select PyYAML version
as well as one test dependency.

PEP 566 added new metadata fields including Description-
Content-Type which requires setuptools 38.6.0 (released
Mar 15, 2018) or later [1]. This is used to define the
long description as rst formatted text. This seems to
require also wheel version 0.31.0 of greater [4].

Pip 9.1 added support for pyproject.toml.

[1] https://setuptools.readthedocs.io/en/latest/history.html
[2] https://pip.pypa.io/en/stable/news/
[3] https://hynek.me/articles/conditional-python-dependencies/
[4] pypa/warehouse#3664 (comment)

Signed-off-by: Heikki Laaksonen <laaksonen.heikki.j@gmail.com>
@osilkin98

This comment has been minimized.

Copy link

osilkin98 commented Oct 17, 2018

I actually have this problem right now, and using both wheel==0.31.0 and wheel==0.32.1 (latest at the time of posting) both break the markdown formatting

@di

This comment has been minimized.

Copy link
Member

di commented Oct 17, 2018

Hi @osilkin98, can you link to the release with a broken description? Can you tell us how you uploaded the release to PyPI?

@osilkin98

This comment has been minimized.

Copy link

osilkin98 commented Oct 18, 2018

@di I was typing out a response documenting how to replicate the issue and in doing so I realized that the version of wheel that was being used during the setup was installed to python3.6's packages, but the one I installed and upgraded was installed to python3.7.

I fixed the issue simply by verbosely running python -m pip install --upgrade wheel in my virtual environment. My apologies for any confusion.

@scruffystuffs

This comment has been minimized.

Copy link

scruffystuffs commented Nov 16, 2018

This problem is still occurring when running python setup.py sdist bdist_wheel and twine upload from a windows machine.

Tested ubuntu 18.04 and windows 10. Ubuntu uploads everything fine, but running the exact same commands from a windows machine causes the above issue to appear with the exact same files.

setuptools, wheel, and twine are all at or beyond the levels specified in the above comments, and are at the same level for both machines.

@di

This comment has been minimized.

Copy link
Member

di commented Nov 16, 2018

@scruffystuffs Can you point me to a distribution on PyPI that you've uploaded that has this problem?

@scruffystuffs

This comment has been minimized.

Copy link

scruffystuffs commented Nov 16, 2018

Uploaded two example cases to test.pypi.org.

Uploaded from Windows
Uploaded from Ubuntu

The files for these distributions should be identical, other than the version number, yet they render entirely differently on the project page.

If any more info is needed, let me know and I will be happy to provide it.

@di

This comment has been minimized.

Copy link
Member

di commented Nov 16, 2018

@scruffystuffs Looking at the metadata of the source distribution of your package, it has Windows-style line endings in the Description, so it looks like you've having the same issue as pypa/setuptools#1440

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment