Problem with register, upload #191

Closed
delijati opened this Issue Nov 7, 2016 · 9 comments

Projects

None yet

4 participants

@delijati
delijati commented Nov 7, 2016

I run into problems when i run fullrelease:

INFO: Starting prerelease.
Changelog entries for version 0.2:

0.1.1 (unreleased)
------------------

- Pypi release.


0.1 (2016-11-05)
----------------
Enter version [0.2]: 
INFO: Set setup.py's version to '0.2'
INFO: Changed version from 0.2.dev0 to 0.2
INFO: History file CHANGES.rst updated.
INFO: The 'git diff':

diff --git a/CHANGES.rst b/CHANGES.rst
index bd7668b..eaff8c9 100644
--- a/CHANGES.rst
+++ b/CHANGES.rst
@@ -1,8 +1,8 @@
 CHANGES
 =======
 
-0.1.1 (unreleased)
-------------------
+0.2 (2016-11-07)
+----------------
 
 - Pypi release.
 
diff --git a/setup.py b/setup.py
index 3d366ff..14dec22 100755
--- a/setup.py
+++ b/setup.py
@@ -15,7 +15,7 @@ requires = [
 
 setup(
     name="cornice_swagger",
-    version='0.2.dev0',
+    version='0.2',
     description="Generate swagger from a Cornice application",
     long_description=README + "\n\n" + CHANGES,
     license="Apache License (2.0)",


OK to commit this (Y/n)? 
INFO: [master 06d8539] Preparing release 0.2
 2 files changed, 3 insertions(+), 3 deletions(-)

INFO: Starting release.
Tag needed to proceed, you can use the following command:
git tag 0.2 -m "Tagging 0.2"
Run this command (Y/n)? 

Check out the tag (for tweaks or pypi/distutils server upload) (Y/n)? 
INFO: Doing a checkout...
Note: checking out '0.2'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

git checkout -b <new-branch-name>

HEAD ist jetzt bei 06d8539... Preparing release 0.2
INFO: Tag checkout placed in /tmp/cornice_swagger-0.2-kc2fxktg/gitclone
INFO: Changing to sub directory in tag checkout: /tmp/cornice_swagger-0.2-kc2fxktg/gitclone
INFO: Making a source distribution of a fresh tag checkout (in /tmp/cornice_swagger-0.2-kc2fxktg/gitclone).
Showing first few lines...

Installed /tmp/cornice_swagger-0.2-kc2fxktg/gitclone/.eggs/setuptools_git-1.1-py3.5.egg
running sdist
running egg_info
creating cornice_swagger.egg-info
...
Showing last few lines...
Writing cornice_swagger-0.2/setup.cfg
creating dist
Creating tar archive
removing 'cornice_swagger-0.2' (and everything under it)

Showing first few lines...

Installed /tmp/cornice_swagger-0.2-kc2fxktg/gitclone/.eggs/setuptools_git-1.1-py3.5.egg
running sdist
running egg_info
creating cornice_swagger.egg-info
...
Showing last few lines...
Writing cornice_swagger-0.2/setup.cfg
creating dist
Creating tar archive
removing 'cornice_swagger-0.2' (and everything under it)

INFO: This package is NOT registered on PyPI.
Register and upload to pypi (y/N)? yes
INFO: Registering...
Registering cornice_swagger-0.2.tar.gz
INFO: Starting new HTTPS connection (1): upload.pypi.org
INFO: Starting new HTTPS connection (1): pypi.python.org
Traceback (most recent call last):
  File "/opt/venvs/py3/bin/fullrelease", line 11, in <module>
    sys.exit(main())
  File "/opt/venvs/py3/lib/python3.5/site-packages/zest/releaser/fullrelease.py", line 27, in main
    releaser.run()
  File "/opt/venvs/py3/lib/python3.5/site-packages/zest/releaser/baserelease.py", line 308, in run
    self.execute()
  File "/opt/venvs/py3/lib/python3.5/site-packages/zest/releaser/release.py", line 71, in execute
    self._release()
  File "/opt/venvs/py3/lib/python3.5/site-packages/zest/releaser/release.py", line 311, in _release
    self._upload_distributions(package)
  File "/opt/venvs/py3/lib/python3.5/site-packages/zest/releaser/release.py", line 172, in _upload_distributions
    self._retry_twine('upload', server, filename)
  File "/opt/venvs/py3/lib/python3.5/site-packages/zest/releaser/release.py", line 198, in _retry_twine
    if repository.package_is_uploaded(package_file):
  File "/opt/venvs/py3/lib/python3.5/site-packages/twine/repository.py", line 176, in package_is_uploaded
    releases = response.json()['releases']
  File "/opt/venvs/py3/lib/python3.5/site-packages/requests/models.py", line 826, in json
    return complexjson.loads(self.text, **kwargs)
  File "/opt/venvs/py3/lib/python3.5/site-packages/simplejson/__init__.py", line 516, in loads
    return _default_decoder.decode(s)
  File "/opt/venvs/py3/lib/python3.5/site-packages/simplejson/decoder.py", line 374, in decode
    obj, end = self.raw_decode(s)
  File "/opt/venvs/py3/lib/python3.5/site-packages/simplejson/decoder.py", line 404, in raw_decode
    return self.scan_once(s, idx=_w(s, idx).end())
simplejson.scanner.JSONDecodeError: Expecting value: line 2 column 1 (char 1)
fullrelease  7,35s user 0,41s system 36% cpu 21,419 total
@mauritsvanrees
Member

What is a bit suspect here, is that you have simplejson. I normally don't use that, unless I need to support older Python versions that do not have the standard json library. So maybe you could try to remove simplejson and try again. But that may not be the problem.

It goes wrong in the twine library that we are using. Or really in the requests library that it is using. It makes a request to PyPI and expects json as response, and fails to parse it. So maybe it wrongly got an html page back from PyPI. It could be a temporary PyPI failure. Can you simply try again?

If it still fails, you can try it without zest.releaser, to see if the failure is with us or upstream. After failure, zest.releaser should print a directory where it has checked out the new tag of your package. Go to that directory and do twine upload dist/*. Or in the working directory of your repository, call python setup.py sdist and then the same twine command.

If it fails with zest.releaser and works with plain twine, zest.releaser may be creating the package in a wrong way.

BTW, I see your package uses setuptools_git to make sure the correct files end up in the release. I always prefer not to rely on such a helper package, but instead add a correct MANIFEST.in. But that is unlikely to be the problem here.

@delijati
delijati commented Nov 8, 2016

@mauritsvanrees i removed simplejsonbut the error still exists. I tried the twine directly and it worked:

(py3) delicj@lupad ../cornice_swagger-0.2-uyfmy033/gitclone (git)-[0.2] % twine upload dist/*
Uploading distributions to https://upload.pypi.org/legacy/
Uploading cornice_swagger-0.2.tar.gz
[================================] 29170/29170 - 00:00:02
@delijati
delijati commented Nov 8, 2016 edited

I also noticed that registering a egg is not necessary any more.

@mauritsvanrees
Member

What zest.releaser is doing here, is basically:

  • call python setup.py sdist
  • create a Repository instance fromtwine
  • create a Packagefile instance from twine, using the created sdist
  • call repository.package_is_uploaded(package_file), and here it fails.

I don't see what we could be doing differently. When I try it myself on a clone of https://github.com/Cornices/cornice.ext.swagger the same call works fine now:

INFO: This package is registered on PyPI.
Upload to pypi (Y/n)? 
INFO: Starting new HTTPS connection (1): pypi.python.org
WARNING: A file dist/cornice_swagger-0.2.tar.gz has already been uploaded. Ignoring.

Ah, but if I change the name of the package to maurits_cornice_swagger and try it again, it fails in the same way. It fails when interpreting the contents of https://pypi.python.org/pypi/maurits-cornice-swagger/json. That gives an html page with a 'not found' message.

Ah, and core twine has the same problem if you pass it the skip-existing option, like this:

twine upload --skip-existing dist/maurits_cornice_swagger-0.3.tar.gz

Without this option, the package_is_uploaded method never gets called. I think it should get called always. The twine command line skip-existing option currently has multiple effects:

skip-existing = False:

  • don't check if package file has been uploaded already
  • just always upload, and if this fails and the error reason is that the package file was already uploaded, print a warning and continue with the next file.

skip-existing = True:

  • check if package file has been uploaded already
  • if already uploaded, print a message that it gets skipped
  • if not yet uploaded, upload the file, and throw an error when this fails.

So this error boils down to: if you call twine upload --skip-existing for a package that has not been registered yet, you get an error. I will open an issue in twine.

@gyst
gyst commented Nov 24, 2016

Issue confirmed. Workaround: register the package manually on pypi via the web form first. Then run release.

@tkimnguyen
Contributor

Yup, just ran into this. My first time using zest.releaser :)

@tkimnguyen
Contributor

Trying this with Plone 4.3.11 I had to add twine explicitly to my eggs in buildout.cfg, and I had to manually run twine upload dist/* after the above failure (but the twine command did succeed).

@mauritsvanrees
Member

This is fixed in zest.releaser 6.8.

@mauritsvanrees
Member

I have released 6.8.1 with a related fix that may still be needed for this one. See issue #206.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment