Repository URL weirdness

[InAnimaTe]

It seems there might be some weirdness with defining the pypi repo url related to #203 changes which exist in 1.8.1

Case 1

With the following variables set:

export TWINE_REPOSITORY_URL=https://pypi.example.domain
export TWINE_PASSWORD=hahagtfo
export TWINE_USERNAME=project-user

and a ~/.pypirc either with or without contents (this one is empty), and upload works just fine:

└[~/arrovenv/project]> twine upload dist/*
Uploading distributions to https://pypi.example.domain/
Uploading project-0.1b1.dev0-py35-none-any.whl
[================================] 21581/21581 - 00:00:01

However, if I rm the ~/.pypirc file (leaving all variables the same), I get:

└[~/arrovenv/project]> twine upload dist/*
Uploading distributions to https://upload.pypi.org/legacy/
Uploading project-0.1b1.dev0-py35-none-any.whl
[================================] 21581/21581 - 00:00:00
HTTPError: 403 Client Error: Invalid or non-existent authentication information. for
url: https://upload.pypi.org/legacy/

From this, it seems like twine won't even read the variables if a .pypirc file doesn't exist.

Case 2

Additionally, if I have the following variables defined:

export TWINE_REPOSITORY=https://pypi.example.domain
export TWINE_REPOSITORY_URL=https://pypi.example.domain
export TWINE_PASSWORD=hahagtfo
export TWINE_USERNAME=project-user

and no ~/.pypirc in existence, upload works just fine.

Case 3

Well ok...what if just TWINE_REPOSITORY is defined with my url?

Either with or without a ~/.pypirc in existence,

export TWINE_REPOSITORY=https://pypi.example.domain
export TWINE_PASSWORD=hahagtfo
export TWINE_USERNAME=project-user

I receive an error about a problem with configuration:

└[~/arrovenv/project]> twine upload dist/*
KeyError: "Missing 'https://pypi.example.domain/' section from the configuration
 file\nor not a complete URL in --repository.\nMaybe you have a out-dated '~/.pypirc'
 format?\nmore info: https://docs.python.org/distutils/packageindex.html#pypirc\n"

Conclusion

There is definitely something going on here. If all I should need, given I don't have a ~/.pypirc, is a url+credentials, why do I have to provide the URL in two separate variables?
In my opinion, Case 1 should work without a ~/.pypirc, Case 2 shouldn't ever be necessary, and Case 3 should work.

I want to thank @sigmavirus24 for his work on getting this feature merged and in 1.8.1. Hopefully with a few minor changes, we can make it work swimmingly for everyone ;)

[fruechel]

I looked a bit into this and the issue lies in the following problem: If you don't have a .pypirc and also don't specify --repository on the command line, then it will default to the repo to pypi (see upload.py).

It will then go to utils.py and try to get the current config file. If that doesn't exist, it will return a default one which contains pypi. Since it also defaulted to pypi above, it will find and use that and ignore anything in --repository-url. If it happens to find a .pypirc which doesn't contain a pypi section then it will fail on the lookup and continue to use the URL.

The fix pretty much depends on the behaviour expected from the program. I feel like it would be expected that --repository-url overrides any --repository choices. Alternatively, it could just be an alias since there is no sense in providing both. Except, of course, --repository-url pypi doesn't really make sense but --repository URL would.

I also feel like you shouldn't be removing the pypi section (implicitly) just by creating your own .pypirc. Instead the default should be those two sections (pypi and pypitest) + anything you add in that file (while that file then overrides those two if specified with the same name).

Having said all of that, I created a simple commit that only changes the behaviour of having the URL take precedence over a repo name. This clearly changes the behaviour of the program and is nut just a bugfix. That is, if you rely on this where you provide --repository and --repository-url and then expect it to only fall back then this will break. For this reason, I did not make a PR for this yet.

@sigmavirus24 you implemented this feature it appears so maybe you could chime in on whether you wanted this to be in here explicitly. If the decision is to make the URL only a fallback, then we could discuss only overriding with the URL if the name for the repo is the default.

In any case, I would like to get this feature improved, because the current fix basically relies on creating dummy files or providing invalid parameters and none of that is very clean. I'll happily provide PRs for expected behaviour :)

[ioggstream]

@InAnimaTe thx for your export tip, it worked for me where neither the setting the proper uri in .pypirc and removing the file worked.

imho this kind of issues are annoying as they are noticed... at release time ;)

We could manage to:

• better document it ( find nothing on that on https://pypi.python.org/pypi/twine)
• add migration infos properly.

[zirpu]

Case N. 2 doesn't work for me for https://upload.pypi.io nor with /legacy/

Using twine 1.8.1

[zirpu]

Update: turns out my password having '#' and '!' characters seems to have been the problem. weirdness.

[gdubicki]

Thank you @zirpu , your comment about non-accepted characters helped me!

It may be worth mentioning that I had this issue when trying to use Travis CI to deploy to PyPi (https://github.com/egnyte/gitlabform/blob/master/.travis.yml).