Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

pip crashes when server does not send content-type header #32

Closed
vbabiy opened this Issue Mar 15, 2011 · 14 comments

Comments

Projects
None yet
6 participants
Contributor

vbabiy commented Mar 15, 2011

I tried to work around the broken werkzeug pypi infos by:

   dependency_links = [
        'https://nodeload.github.com/mitsuhiko/werkzeug/tarball/master#egg=Werkzeug-0.7dev',
    ],
    install_requires=[
        'Werkzeug==0.7dev', # wsgi toolkit, minimum rev 0.7dev 2010/12
...
    ],

Problem: github does not send a content-type header 8(

Downloading/unpacking Werkzeug==0.7dev (from moin)
Exception:
Traceback (most recent call last):
  File "/home/tw/w/mm20dev/env/lib/python2.6/site-packages/pip-0.7.2-py2.6.egg/pip/basecommand.py", line 120, in main
    self.run(options, args)
  File "/home/tw/w/mm20dev/env/lib/python2.6/site-packages/pip-0.7.2-py2.6.egg/pip/commands/install.py", line 161, in run
    requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
  File "/home/tw/w/mm20dev/env/lib/python2.6/site-packages/pip-0.7.2-py2.6.egg/pip/req.py", line 879, in prepare_files
    self.unpack_url(url, location, self.is_download)
  File "/home/tw/w/mm20dev/env/lib/python2.6/site-packages/pip-0.7.2-py2.6.egg/pip/req.py", line 1035, in unpack_url
    content_type = resp.info()['content-type']
  File "/usr/lib/python2.6/rfc822.py", line 388, in __getitem__
    return self.dict[name.lower()]
KeyError: 'content-type'

Suggestion: maybe catch this case and just look at the filename you get and use mimetypes.guess_type.


Contributor

vbabiy commented Mar 15, 2011

Couldn't reproduce this in my own tests with

pip install

https://github.com/mitsuhiko/werkzeug/tarball/master#egg=Werkzeug-0.7dev

Not sure whether if Github fixed something or there is a difference between
using nodeload in the URL.


Original Comment By: Jannis Leidel
Contributor

vbabiy commented Mar 15, 2011

github obviously changed behaviour (I also reported a bug there about the
missing content-type header).

but still pip needs fixing, because it'll still crash if it doesn't get a c-t
header.


Original Comment By: Thomas Waldmann
Contributor

vbabiy commented Mar 15, 2011

Thomas, it seems you are running an older version of pip (0.7.2). Is it
possible for you to upgrade to pip 8.2? I am curious if you would still get
this error running pip 8.2


Original Comment By: Kelsey Hightower
Contributor

vbabiy commented Mar 15, 2011

I can't reproduce this anymore but there was an issue with Github due not
sending a content type header. Following command failed for me with the same
error as above.

pip install https://github.com/divio/django-cms/tarball/master

I also think that pip should handle missing content type headers more
graceful. I use and used pip 0.8.2.


Original Comment By: Anonymous
Contributor

vbabiy commented Mar 15, 2011

Thanks for the reply, I will work up a patch and get it ready for review.


Original Comment By: Kelsey Hightower
Contributor

vbabiy commented Mar 15, 2011

Fixed in my fork:

https://bitbucket.org/khightower/pip/changeset/1bb3bf439fdb

The Solution

If the HTTP response header does not define the content-type, set it to an
empty string which supports the 'lower' attribute required by
mimetypes.guess_extension()


Original Comment By: Kelsey Hightower
Contributor

vbabiy commented Mar 15, 2011

Created a new fork without named branches.

https://bitbucket.org/khightower/pip-issue-207/changeset/d486e99d8da4


Original Comment By: Kelsey Hightower
Member

hltbra commented Mar 24, 2011

Attaching gist for Kelsey's patch: https://gist.github.com/886024

(repository pip-issue-207 was removed)

Contributor

igorsobreira commented Mar 20, 2013

I'm having the same issue here:

$ pip install https://github.com/pypa/pip/archive/develop.zip
Downloading/unpacking https://github.com/pypa/pip/archive/develop.zip
Exception:
Traceback (most recent call last):
  File "/Users/igor/.envs/removeme/lib/python2.7/site-packages/pip/basecommand.py", line 139, in main
    status = self.run(options, args)
  File "/Users/igor/.envs/removeme/lib/python2.7/site-packages/pip/commands/install.py", line 266, in run
    requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
  File "/Users/igor/.envs/removeme/lib/python2.7/site-packages/pip/req.py", line 1033, in prepare_files
    self.unpack_url(url, location, self.is_download)
  File "/Users/igor/.envs/removeme/lib/python2.7/site-packages/pip/req.py", line 1161, in unpack_url
    retval = unpack_http_url(link, location, self.download_cache, self.download_dir)
  File "/Users/igor/.envs/removeme/lib/python2.7/site-packages/pip/download.py", line 535, in unpack_http_url
    content_type = resp.info()['content-type']
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/rfc822.py", line 388, in __getitem__
    return self.dict[name.lower()]
KeyError: 'content-type'

$ pip --version
pip 1.3.1 from /Users/igor/.envs/removeme/lib/python2.7/site-packages (python 2.7)

I was having this same issue earlier today with installing https://github.com/django/django-localflavor-us/archive/e81479b3a4.zip , but github is now sending the correct header to "fix" it.

Strange that it seems to be so intermittent...

Contributor

igorsobreira commented Mar 20, 2013

I've sent an email to github support, they said it was really a bug. Already fixed!

Contributor

pnasrat commented Mar 21, 2013

We should certainly fail a bit more gracefully.

@ghost ghost assigned hltbra Mar 28, 2013

hltbra added a commit to hltbra/pip that referenced this issue Mar 28, 2013

Fix issue #32: pip crashes when server does not send content-type header
Thanks Kelsey Hightower for pointing what to change.
Member

hltbra commented Mar 29, 2013

Closing this issue because of pull request #872 with the fix.

@hltbra hltbra closed this Mar 29, 2013

pnasrat added a commit that referenced this issue Mar 30, 2013

Merge pull request #872 from hltbra/issue-32
Fix issue #32: pip crashes when server does not send content-type header
Contributor

davvid commented Jul 13, 2013

I ran into this issue today on the "develop" branch. PR forthcoming.

davvid added a commit to davvid/pip that referenced this issue Jul 13, 2013

pip.index: fix crash getting Content-Type header
Returned headers may not always have the "Content-Type"
field available.  Avoid KeyError exceptions by using
dict.get() instead of blindly grabbing the header value.

Closes #32

dstufft added a commit that referenced this issue Jul 15, 2013

pip.index: fix crash getting Content-Type header
Returned headers may not always have the "Content-Type"
field available.  Avoid KeyError exceptions by using
dict.get() instead of blindly grabbing the header value.

Closes #32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment