Packaging instructions are not optimal about README.MD

[vanrein]

Thank you for providing feedback on Python packaging!

To help us help you, please fill out as much of the following as you can. If a question is not relevant, feel free to skip it.

1. What is your operating system and version?

MacOS 10.15.3

2. What is your Python version?

Python 3.7.3

3. What version of pip do you have?

pip 19.0.3

4. If following an online tutorial or guide, please provide a link to the page or section giving you trouble:

https://packaging.python.org/tutorials/packaging-projects/#creating-setup-py

5. Could you describe your issue in as much detail as possible?

The instructions provide an adagio for a long readme and claim it is normal:

with open("README.md", "r") as fh:
    long_description = fh.read()

This however, will only run from the directory containing this file. It is better to construct a path:

here = path.dirname (path.realpath (__file__))
readme = open (path.join (here, 'README.MD')).read ()

Both forms still leave a problem. When setup.py is run during pip install, it will try to open README.md once more, while it is being installed. (This was after uploading an sdist, not a bdist.)

Collecting arpa2.wsgi

  Downloading https://files.pythonhosted.org/packages/71/87/fe38499da24d4bfcd04d301aa20a03af35bfdf379e8b739069b8a726af2a/arpa2.wsgi-0.4.0.tar.gz

    Complete output from command python setup.py egg_info:

    Traceback (most recent call last):

      File "<string>", line 1, in <module>

      File "/private/var/folders/5_/dsls_kvx7tq171bf6chfm3x80000gn/T/pip-install-zcq3d6v9/arpa2.wsgi/setup.py", line 19, in <module>

        readme = open (path.join (here, 'README.MD')).read ()

    FileNotFoundError: [Errno 2] No such file or directory: '/private/var/folders/5_/dsls_kvx7tq171bf6chfm3x80000gn/T/pip-install-zcq3d6v9/arpa2.wsgi/README.MD'

As far as I can see, I am not supposed to include the README.MD and it is only needed because of the packaging instruction -- which may be off. Not sure what a good fix would be, otherwise I'd suggest it.

[di]

I agree that a proper path to the README would be idea. Here's what we suggest on https://github.com/pypa/sampleproject:

here = path.abspath(path.dirname(__file__))

# Get the long description from the README file
with open(path.join(here, 'README.md'), encoding='utf-8') as f:
    long_description = f.read()

As far as I can see, I am not supposed to include the README.MD and it is only needed because of the packaging instruction

You should include the README in your distribution, if not for this reason, because your users might want to read it along with your source.

[pastelmind]

I used the code from https://github.com/pypa/sampleproject/blob/master/setup.py (currently at d09af3dbd851d385e56f0aed29875bfa3d3df230) to update the sample setup.py, and also explain why README.md needs to be added to the package. Please check #711

Edit: According to @merwok (#711 (comment)), setup.py is always intended to be called from the directory it lives in, since it otherwise would break "other things"). So the modified setup.py code has been excluded from the PR.

[matthewfeickert]

As I'm (slowly) going through and picking through "good first issue" Issues, should this one still be open / in need of a PR? Or from @pastelmind's comment

setup.py is always intended to be called from the directory it lives in, since it otherwise would break "other things")

can this be closed?

[pastelmind]

Looking back, I realized that my PR does not adequately address the two concerns raised by @vanrein. I'll reiterate them here and attempt to answer them:

1. Why not use __path__ to load README.md regardless of the cwd?
   • Because setup.py is meant to be run at the project root anyway, so there's little point in providing flexibility for infeasible cases. (This was
     @merwok's argument; I'm not knowledged enough to verify this claim.)
2. Why do we have to load README.md during pip install? Isn't that suboptimal?
   • Yes, it is. It's one of the reasons why we are moving away from setup.py in favor of pyproject.toml and tools that use them.

I believe these answers are not appropriate for adding to the guide, because (1) they're incomplete and (2) beyond the scope of what the guide is about. Just personal opinions, so I'm fine if someone else adds them to the guide.

[vanrein]

FYI, this never really worked for me; I now refrain from including READMEs and am seriously unhappy with Python packaging -- mostly because there are so many versions of it, and all are slightly incompatible. I have given up on half of the Python Packaging Promises, basically, and cut down to bare essentials. What would be useful is a clear indication of what works on what packaging system. Now all that developers can find is a confusing image of the world of Python Packaging.

[pastelmind]

I agree that the state of Python packaging is still far from becoming streamlined after all these years. It would have saved me many hours if someone had told me "Yeah this is tough. Just use Flit or Poetry."

It reminds me of how, in the JavaScript world, NPM (the default package manager, dependency manager and publishing tool) lagged behind competitiors like Yarn for years--though NPM has been catching up recently. The JS community is not afraid to teach newcomers "NPM is good for entry-level use cases. When it fails, use Yarn, JSPM, ..."

[matthewfeickert]

FYI, this never really worked for me

Without going off the topic of this Issue, and what a PR resolution might be, too much — I'm sorry to hear that you've had such a bad experience here @vanrein. If your library that you're trying to package is public have you reported your specific issues as a GitHub issue over on pypa/packaging-problems? (Edit: Whoops, this is where it came from to begin with! Sorry for not paying close attention.)

[merwok]

Re-reading the original message: the guides all talk about README.md, as that’s the common name for such a file in a github-dominated landscape, but your code snippet and the error message show README.MD. Do you have a readme file exactly named that way (caps)?

[vanrein]

Yes I do. I only recently learnt about the capitalisation of the extension, find it ugly, but should work towards compliance. But the file is not available, I hardly think this is practical?

[matthewfeickert]

But the file is not available, I hardly think this is practical?

I'm not sure what you mean here that the README file is "not available". Can you link to your project if it is public so that we can see?

With regards to the Issue of revising the tutorial, I agree with @pastelmind's thoughts

I believe these answers are not appropriate for adding to the guide, because (1) they're incomplete and (2) beyond the scope of what the guide is about

RE: being beyond the scope of the "Packaging Python Projects" tutorial. I think that given @merwok's point in PR #711 (comment) the suggestion for revision to the tutorial to include paths seems to be no longer relevant. If maintainers (like @di) agree, then I would suggest that once @vanrein's specific problem has been resolved this Issue get closed — IMO PR #711 resolved it.

[pradyunsg]

The JS community is not afraid to teach newcomers "NPM is good for entry-level use cases. When it fails, use Yarn, JSPM, ..."

Right, and our problem is quite the opposite.

Existing tooling is great for handling complicated workflows and usecases, and all the new tooling is for simpler packaging usecases with a nicer UX (so far).

None the less, the documentation definitely needs some TLC and thought.