poetry publish fails for package with . in its name.

[nekitdev]

• I am on the latest Poetry version.

• I have searched the issues of this repo and believe that this is not a duplicate.

• OS version and name: windows 11, also fails on linux

• Poetry version: 1.2.0-beta.1

• Link of a Gist with the contents of your pyproject.toml file: https://gist.github.com/nekitdev/99b0cd36e9fd32712f38401439823cc1

Issue

For packages with . in the name:

[tool.poetry]
name = "example.py"

Poetry correctly builds wheels and sdists with example.py in the file name.
However, when uploading to PyPI with the canonical name (i.e. example-py), poetry publish fails with the following error:

HTTP Error 400: Start filename for 'example-py' with 'example-py'.

[mkniewallner]

According to https://packaging.python.org/en/latest/specifications/binary-distribution-format/#escaping-and-unicode, it seems that the right thing to do for wheels would be to replace . with _ for the distribution name.

Though even if doing so, a bug in warehouse (the backend behind PyPI) may prevent the upload from working: pypi/warehouse#10030.

[nekitdev]

It seems as though Poetry attempts to publish example-py with files containing example.py, causing PyPI to return the error above. I have hooked canonicalize_name function to see if PyPI allows uploading with example.py as the name, which it apparently does. So I'm assuming the mismatch is what causes the error.

[nekitdev]

Once again, I'm not exactly sure how to solve this issue, though.

[neersighted]

Once again, I'm not exactly sure how to solve this issue, though.

We've already got a PR under review that uses the same logic pip does to canonicalize wheel names.

[nekitdev]

Oh, I see, thanks! I'm looking forward to it. :)

[NiklasRosenstein]

Hello, what's the progress on the PR @neersighted? I'm running into the same issue.

I'm a bit surprised because on 20th of June (so only two weeks back), I was able to publish a package with . in the name: https://pypi.org/project/databind.core/2.0.7/#files

But today I get

400 Client Error: Start filename for 'databind-core' with 'databind-core'. for url: https://upload.pypi.org/legacy/

I don't see any release to poetry-core since February though.

[NiklasRosenstein]

I just realized that this is an issue with poetry-core==1.1.0b2, but not with the latest final release poetry-core==1.0.8.
The way I had set up my build process was that I ended up using the a poetry-core beta release.

I also confirmed the issue with 1.1.0b3.

[aaronhnsy]

I am still getting this issue with poetry==1.2.0 and poetry-core==1.1.0.
Building works fine but I am unable to publish the sdist file and publishing the wheel file results in the package having a dash in the name on pypi which is undesirable.

[dimbleby]

Situation:

• poetry always normalizes the package name at upload
• poetry-core normalizes package names when building .whl but not when building .tar.gz
• pypi insists that uploaded files start with a prefix that matches the package name

So per this issue, folk who try for un-normalized package names find that poetry normalizes their package name at upload, and they can't upload their source distribution because it has the wrong prefix.

Probably the sensible next step is to normalize .tar.gz names in poetry-core (and also make the corresponding change in poetry itself - the uploader class currently expects unnormalized source distributions, because it 'knows' what poetry-core did).

That gets to a place where poetry insists on normalization, but at least it is possible to upload sdists.

Not planning on doing this myself, no doubt MRs are welcome.

[mkniewallner]

Probably the sensible next step is to normalize .tar.gz names in poetry-core (and also make the corresponding change in poetry itself - the uploader class currently expects unnormalized source distributions, because it 'knows' what poetry-core did).

Last time I checked, there was no accepted PEP formally stating that sdists should be canonicalised, though the closest thing there is to that is this packaging specification which states:

The file name of a sdist is not currently standardised, although the de facto form is {name}-{version}.tar.gz, where {name} is the canonicalized form of the project name (see PEP 503 for the canonicalization rules) with - characters replaced with _, and {version} is the canonicalized form of the project version (see Version specifiers).

There is PEP 625 about canonicalising the name the same way as for wheels, but it is still in draft.

Although PEP or not, it would seem more sensible to canonicalise sdists the same way as Poetry already does for wheels, for better consistency, especially since this (probably) shouldn't be a problem for any tool consuming the sdists.

[dimbleby]

Sure, the proposal to canonicalize sdists in the same way as bdists wasn't intended to rule out the possibility of supporting un-canonicalized project names at some later point (nor to rule it in, come to that!).

I suggest sdist canonicalization as a relatively low risk next step that can be taken without getting into that discussion at all. It's clearly a bug at the moment that some packages can't upload their sdists, start by solving that.

Then later debate the merits or otherwise of supporting un-canonicalized names.

[neersighted]

If anything this bug is PyPI being too strict in validation when different names are allowed by the spec -- technically Poetry is doing nothing wrong. However, from a practical standpoint I believe they use filenames to identify releases, so the requirement to match makes sense (even if it's undocumented/not required by spec).

[SvenBecker]

Is there some way to opt out of the package name normalization or would it be possible to provide such an option? We are using some internal PyPI so I'm not sure if those PyPI validations even affect us.

[dimbleby]

No. You will either need to put together an MR changing the code, or hope that someone else does.

[NiklasRosenstein]

What worked for me is to pin poetry-core==1.0.8:

[build-system]
requires = ["poetry-core==1.0.8"]
build-backend = "poetry.core.masonry.api"

But I build with Poetry build and publish with Twine; not sure if this still works with poetry publish.

[neersighted]

This will not work with poetry publish as Poetry does not use the requires = field.

[neersighted]

Closing as python-poetry/poetry-core#484 is merged.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.