New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Installation fails in Cygwin 64 bit #1547

Closed
gregersn opened this Issue Aug 21, 2013 · 54 comments

Comments

Projects
None yet
@gregersn

gregersn commented Aug 21, 2013

When I try a pip install requests in Cygwin 64 bit the installation fails.

Here's the end of the log file:

  Downloading from URL https://pypi.python.org/packages/source/r/requests/requests-1.2.3.tar.gz#md5=adbd3f18445f7fe5e77f65c502e264fb (from https://pypi.python.org/simple/requests/)
  Running setup.py egg_info for package requests

Cleaning up...

  Removing temporary dir /cygdrive/d/home/greger/cygwin/projects/myproject/build...
No files/directories in /cygdrive/d/home/greger/cygwin/projects/myproject/build/requests/pip-egg-info (from PKG-INFO)

Exception information:
Traceback (most recent call last):
  File "/cygdrive/d/home/greger/cygwin/projects/myproject/lib/python2.7/site-packages/pip/basecommand.py", line 134, in main
    status = self.run(options, args)
  File "/cygdrive/d/home/greger/cygwin/projects/myproject/lib/python2.7/site-packages/pip/commands/install.py", line 236, in run
    requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
  File "/cygdrive/d/home/greger/cygwin/projects/myproject/lib/python2.7/site-packages/pip/req.py", line 1139, in prepare_files
    req_to_install.assert_source_matches_version()
  File "/cygdrive/d/home/greger/cygwin/projects/myproject/lib/python2.7/site-packages/pip/req.py", line 394, in assert_source_matches_version
    version = self.installed_version
  File "/cygdrive/d/home/greger/cygwin/projects/myproject/lib/python2.7/site-packages/pip/req.py", line 390, in installed_version
    return self.pkg_info()['version']
  File "/cygdrive/d/home/greger/cygwin/projects/myproject/lib/python2.7/site-packages/pip/req.py", line 357, in pkg_info
    data = self.egg_info_data('PKG-INFO')
  File "/cygdrive/d/home/greger/cygwin/projects/myproject/lib/python2.7/site-packages/pip/req.py", line 293, in egg_info_data
    filename = self.egg_info_path(filename)
  File "/cygdrive/d/home/greger/cygwin/projects/myproject/lib/python2.7/site-packages/pip/req.py", line 330, in egg_info_path
    raise InstallationError('No files/directories in %s (from %s)' % (base, filename))
InstallationError: No files/directories in /cygdrive/d/home/greger/cygwin/projects/myproject/build/requests/pip-egg-info (from PKG-INFO)

Don't know if that is a problem with the installation package, or cygwin itself. But thought I'd try here first.

If I try to use easy_install it just leaves all the files in the tmp-dir. And if I from there try to do a python setup.py install, and then try to import requests from the python shell, it just exits without and output.

@Lukasa

This comment has been minimized.

Show comment
Hide comment
@Lukasa

Lukasa Aug 27, 2013

Member

Hmm, I can't reproduce this. =( But I wonder if you're using a Windows virtualenv with a Windows interpreter in Cygwin. It's not clear (to me) how well that would work.

Member

Lukasa commented Aug 27, 2013

Hmm, I can't reproduce this. =( But I wonder if you're using a Windows virtualenv with a Windows interpreter in Cygwin. It's not clear (to me) how well that would work.

@gregersn

This comment has been minimized.

Show comment
Hide comment
@gregersn

gregersn Aug 27, 2013

You are right, I did try through VirtualEnv and with VirtualenvWrapper. Tried again now from just regular cygwin prompt, and here's the log, again:

  Downloading from URL https://pypi.python.org/packages/source/r/requests/requests-1.2.3.tar.gz#md5=adbd3f18445f7fe5e77f65c502e264fb (from https://pypi.python.org/simple/requests/)
  Running setup.py egg_info for package requests

Cleaning up...

  Removing temporary dir /tmp/pip_build_greger...
No files/directories in /tmp/pip_build_greger/requests/pip-egg-info (from PKG-INFO)

Exception information:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/pip-1.4.1-py2.7.egg/pip/basecommand.py", line 134, in main
    status = self.run(options, args)
  File "/usr/lib/python2.7/site-packages/pip-1.4.1-py2.7.egg/pip/commands/install.py", line 236, in run
    requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
  File "/usr/lib/python2.7/site-packages/pip-1.4.1-py2.7.egg/pip/req.py", line 1139, in prepare_files
    req_to_install.assert_source_matches_version()
  File "/usr/lib/python2.7/site-packages/pip-1.4.1-py2.7.egg/pip/req.py", line 394, in assert_source_matches_version
    version = self.installed_version
  File "/usr/lib/python2.7/site-packages/pip-1.4.1-py2.7.egg/pip/req.py", line 390, in installed_version
    return self.pkg_info()['version']
  File "/usr/lib/python2.7/site-packages/pip-1.4.1-py2.7.egg/pip/req.py", line 357, in pkg_info
    data = self.egg_info_data('PKG-INFO')
  File "/usr/lib/python2.7/site-packages/pip-1.4.1-py2.7.egg/pip/req.py", line 293, in egg_info_data
    filename = self.egg_info_path(filename)
  File "/usr/lib/python2.7/site-packages/pip-1.4.1-py2.7.egg/pip/req.py", line 330, in egg_info_path
    raise InstallationError('No files/directories in %s (from %s)' % (base, filename))
InstallationError: No files/directories in /tmp/pip_build_greger/requests/pip-egg-info (from PKG-INFO)

gregersn commented Aug 27, 2013

You are right, I did try through VirtualEnv and with VirtualenvWrapper. Tried again now from just regular cygwin prompt, and here's the log, again:

  Downloading from URL https://pypi.python.org/packages/source/r/requests/requests-1.2.3.tar.gz#md5=adbd3f18445f7fe5e77f65c502e264fb (from https://pypi.python.org/simple/requests/)
  Running setup.py egg_info for package requests

Cleaning up...

  Removing temporary dir /tmp/pip_build_greger...
No files/directories in /tmp/pip_build_greger/requests/pip-egg-info (from PKG-INFO)

Exception information:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/pip-1.4.1-py2.7.egg/pip/basecommand.py", line 134, in main
    status = self.run(options, args)
  File "/usr/lib/python2.7/site-packages/pip-1.4.1-py2.7.egg/pip/commands/install.py", line 236, in run
    requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
  File "/usr/lib/python2.7/site-packages/pip-1.4.1-py2.7.egg/pip/req.py", line 1139, in prepare_files
    req_to_install.assert_source_matches_version()
  File "/usr/lib/python2.7/site-packages/pip-1.4.1-py2.7.egg/pip/req.py", line 394, in assert_source_matches_version
    version = self.installed_version
  File "/usr/lib/python2.7/site-packages/pip-1.4.1-py2.7.egg/pip/req.py", line 390, in installed_version
    return self.pkg_info()['version']
  File "/usr/lib/python2.7/site-packages/pip-1.4.1-py2.7.egg/pip/req.py", line 357, in pkg_info
    data = self.egg_info_data('PKG-INFO')
  File "/usr/lib/python2.7/site-packages/pip-1.4.1-py2.7.egg/pip/req.py", line 293, in egg_info_data
    filename = self.egg_info_path(filename)
  File "/usr/lib/python2.7/site-packages/pip-1.4.1-py2.7.egg/pip/req.py", line 330, in egg_info_path
    raise InstallationError('No files/directories in %s (from %s)' % (base, filename))
InstallationError: No files/directories in /tmp/pip_build_greger/requests/pip-egg-info (from PKG-INFO)
@gregersn

This comment has been minimized.

Show comment
Hide comment
@gregersn

gregersn Aug 27, 2013

Tested again in a clean installed Cygwin64 bit, same error.
It does work in 32 bit version of Cygwin, though.

gregersn commented Aug 27, 2013

Tested again in a clean installed Cygwin64 bit, same error.
It does work in 32 bit version of Cygwin, though.

@Lukasa

This comment has been minimized.

Show comment
Hide comment
@Lukasa

Lukasa Aug 27, 2013

Member

That's truly bizarre. It looks like pip is being installed as part of Cygwin, so you shouldn't have any 32/64 bit problems, but I kinda have to believe you are.

Member

Lukasa commented Aug 27, 2013

That's truly bizarre. It looks like pip is being installed as part of Cygwin, so you shouldn't have any 32/64 bit problems, but I kinda have to believe you are.

@gregersn

This comment has been minimized.

Show comment
Hide comment
@gregersn

gregersn Aug 27, 2013

I did install pip with easy_install , so the problem might lay there. Still a bit weird, since pip has no problem installing lots of other packages.

gregersn commented Aug 27, 2013

I did install pip with easy_install , so the problem might lay there. Still a bit weird, since pip has no problem installing lots of other packages.

@sigmavirus24

This comment has been minimized.

Show comment
Hide comment
@sigmavirus24

sigmavirus24 Aug 29, 2013

Member

I was under the impression pip was broken on Windows but I'm probably stuck in the past.

Member

sigmavirus24 commented Aug 29, 2013

I was under the impression pip was broken on Windows but I'm probably stuck in the past.

@Lukasa

This comment has been minimized.

Show comment
Hide comment
@Lukasa

Lukasa Aug 29, 2013

Member

Pip works fine on Windows, and in my experience is fine in Cygwin too. Except for this.

Member

Lukasa commented Aug 29, 2013

Pip works fine on Windows, and in my experience is fine in Cygwin too. Except for this.

@zodman

This comment has been minimized.

Show comment
Hide comment
@zodman

zodman Aug 29, 2013

same here the requests not install behind cygwin

zodman commented Aug 29, 2013

same here the requests not install behind cygwin

@meeotch

This comment has been minimized.

Show comment
Hide comment
@meeotch

meeotch Sep 9, 2013

I'm getting the same behavior as the OP: "pip install requests" works on 32 bit cygwin, but fails on 64 bit cygwin with messages about "no files/directories".

Possibly unrelated, but installation-relevant & equally mysterious: 64 bit cygwin wget fails to verify certs for https URLs, while 32 bit cygwin wget succeeds. (Certs are identical on each install, and running the 32 bit wget inside a 64 bit installation [including /usr/ssl/certs directory] works, too.) Anyway, what I'm saying is that installing python stuff under 64 bit cygwin has been a pain in the butt, and I'm keen to hear solutions.

meeotch commented Sep 9, 2013

I'm getting the same behavior as the OP: "pip install requests" works on 32 bit cygwin, but fails on 64 bit cygwin with messages about "no files/directories".

Possibly unrelated, but installation-relevant & equally mysterious: 64 bit cygwin wget fails to verify certs for https URLs, while 32 bit cygwin wget succeeds. (Certs are identical on each install, and running the 32 bit wget inside a 64 bit installation [including /usr/ssl/certs directory] works, too.) Anyway, what I'm saying is that installing python stuff under 64 bit cygwin has been a pain in the butt, and I'm keen to hear solutions.

@kennethreitz

This comment has been minimized.

Show comment
Hide comment
@kennethreitz

kennethreitz Sep 9, 2013

Member

This sounds like an issue with pip itself.

Member

kennethreitz commented Sep 9, 2013

This sounds like an issue with pip itself.

@sigmavirus24

This comment has been minimized.

Show comment
Hide comment
@sigmavirus24

sigmavirus24 Sep 9, 2013

Member

I like the way you think @kennethreitz. :P

It sounds more to me, however, that 64 bit cygwin is broken (maybe).

Member

sigmavirus24 commented Sep 9, 2013

I like the way you think @kennethreitz. :P

It sounds more to me, however, that 64 bit cygwin is broken (maybe).

@tanzaho

This comment has been minimized.

Show comment
Hide comment
@tanzaho

tanzaho Sep 22, 2013

Same thing here. Very interested if someone finds a solution.

tanzaho commented Sep 22, 2013

Same thing here. Very interested if someone finds a solution.

@sigmavirus24

This comment has been minimized.

Show comment
Hide comment
@sigmavirus24

sigmavirus24 Sep 22, 2013

Member

Have any details you can add @tanzaho ?

Member

sigmavirus24 commented Sep 22, 2013

Have any details you can add @tanzaho ?

@tanzaho

This comment has been minimized.

Show comment
Hide comment
@tanzaho

tanzaho Sep 22, 2013

I get the following when run on cygwin64 and trying to pip install the module :
Downloading requests-1.2.3.tar.gz (348kB): 348kB downloaded
Running setup.py egg_info for package requests
Cleaning up...
No files/directories in /home/tanzaho/.virtualenvs/django_wordiz/build/requests/pip-egg-info (from PKG-INFO)

tanzaho commented Sep 22, 2013

I get the following when run on cygwin64 and trying to pip install the module :
Downloading requests-1.2.3.tar.gz (348kB): 348kB downloaded
Running setup.py egg_info for package requests
Cleaning up...
No files/directories in /home/tanzaho/.virtualenvs/django_wordiz/build/requests/pip-egg-info (from PKG-INFO)

@mikewn

This comment has been minimized.

Show comment
Hide comment
@mikewn

mikewn Sep 24, 2013

I'm a bit new to running python setup scripts, especially under 64 bit cygwin in Windows 8. But firewall issues forced me to download the requests-1.2.3.tar.gz file by hand (I couldn't use pip or easy_install without timeouts). I gunzipped and untarred the file and then went in to the directory and ran:

python setup.py install

by hand, and it doesn't output anything and returns to the prompt. Nothing is created in any of the cygwin /usr/lib/python*/site-packages directories. So I think that the process for running the above command is broken rather than it being a connectivity problem or a problem with pip itself. So, you probably don't need to reproduce any network problems to reproduce this problem. Just try to do the above and see if you get the same thing when installing from the command line in a cygwin terminal.

Would like to see this fixed too, as pip and easy_install aren't options for me to install libraries.

Also, I get core dumps if I try to execute the command above with "python3" or "python3.2" instead of "python".

mikewn commented Sep 24, 2013

I'm a bit new to running python setup scripts, especially under 64 bit cygwin in Windows 8. But firewall issues forced me to download the requests-1.2.3.tar.gz file by hand (I couldn't use pip or easy_install without timeouts). I gunzipped and untarred the file and then went in to the directory and ran:

python setup.py install

by hand, and it doesn't output anything and returns to the prompt. Nothing is created in any of the cygwin /usr/lib/python*/site-packages directories. So I think that the process for running the above command is broken rather than it being a connectivity problem or a problem with pip itself. So, you probably don't need to reproduce any network problems to reproduce this problem. Just try to do the above and see if you get the same thing when installing from the command line in a cygwin terminal.

Would like to see this fixed too, as pip and easy_install aren't options for me to install libraries.

Also, I get core dumps if I try to execute the command above with "python3" or "python3.2" instead of "python".

@sigmavirus24

This comment has been minimized.

Show comment
Hide comment
@sigmavirus24

sigmavirus24 Sep 24, 2013

Member

Also, I get core dumps if I try to execute the command above

@mikewn could you provide those? I'm wondering if this isn't an issue in the way Python is built for Windows now. I don't have a windows box to test with though. (Also I didn't know anyone was willing to subject themselves to the horror of Windows 8 ;))

Member

sigmavirus24 commented Sep 24, 2013

Also, I get core dumps if I try to execute the command above

@mikewn could you provide those? I'm wondering if this isn't an issue in the way Python is built for Windows now. I don't have a windows box to test with though. (Also I didn't know anyone was willing to subject themselves to the horror of Windows 8 ;))

@Lukasa

This comment has been minimized.

Show comment
Hide comment
@Lukasa

Lukasa Sep 24, 2013

Member

@sigmavirus24 I have Windows 8. =) Strongly considering moving to Linux on that machine though. Anyway, off-topic.

It's worth noting that this is absolutely not reproducible on 32-bit Cygwin on Windows 7, which strongly points to a problem with Cygwin.

Member

Lukasa commented Sep 24, 2013

@sigmavirus24 I have Windows 8. =) Strongly considering moving to Linux on that machine though. Anyway, off-topic.

It's worth noting that this is absolutely not reproducible on 32-bit Cygwin on Windows 7, which strongly points to a problem with Cygwin.

@mikewn

This comment has been minimized.

Show comment
Hide comment
@mikewn

mikewn Sep 24, 2013

@sigmavirus24 - here's the dump. Basically running python3 or python3.2 both yield stack dumps for python 3.2. And no, this isn't my personal box that I'm running Windows 8 on. :)

https://gist.github.com/mikewn/6688148

mikewn commented Sep 24, 2013

@sigmavirus24 - here's the dump. Basically running python3 or python3.2 both yield stack dumps for python 3.2. And no, this isn't my personal box that I'm running Windows 8 on. :)

https://gist.github.com/mikewn/6688148

@ahmadia

This comment has been minimized.

Show comment
Hide comment
@ahmadia

ahmadia Oct 1, 2013

Does the script, perchance, call a subprocess that uses the UUID module in Python? It turns out that the UUID module in Cygwin64 Python is currently segfaulting, which might be causing mysterious failures.

ahmadia commented Oct 1, 2013

Does the script, perchance, call a subprocess that uses the UUID module in Python? It turns out that the UUID module in Cygwin64 Python is currently segfaulting, which might be causing mysterious failures.

@sigmavirus24

This comment has been minimized.

Show comment
Hide comment
@sigmavirus24

sigmavirus24 Oct 1, 2013

Member

The install script doesn't but urllib3 does use the uuid module for
multipart/form-data. Since everything is "compiled" upon install if it tries
to even import it then this could certainly be the issue. We could go back to
the environment flag to prevent compilation of bytecode but we had our heads
handed to us for that.

Member

sigmavirus24 commented Oct 1, 2013

The install script doesn't but urllib3 does use the uuid module for
multipart/form-data. Since everything is "compiled" upon install if it tries
to even import it then this could certainly be the issue. We could go back to
the environment flag to prevent compilation of bytecode but we had our heads
handed to us for that.

@ahmadia

This comment has been minimized.

Show comment
Hide comment
@ahmadia

ahmadia Oct 1, 2013

This is really an upstream problem for CPython and the Cygwin folks in my mind, so I'm not sure this is on you. I'll post a clean uuid patch against the current Python release on Cygwin on here for the self-starters.

ahmadia commented Oct 1, 2013

This is really an upstream problem for CPython and the Cygwin folks in my mind, so I'm not sure this is on you. I'll post a clean uuid patch against the current Python release on Cygwin on here for the self-starters.

@ahmadia

This comment has been minimized.

Show comment
Hide comment
@ahmadia

ahmadia Oct 2, 2013

Just a confirmation that I can install requests correctly with my UUID-patch Python. I just successfully completed a python setup.py install and the test suite passes. I'm putting together a patch for the Cygwin folks, I'll post back here when it's ready.

ahmadia commented Oct 2, 2013

Just a confirmation that I can install requests correctly with my UUID-patch Python. I just successfully completed a python setup.py install and the test suite passes. I'm putting together a patch for the Cygwin folks, I'll post back here when it's ready.

@sigmavirus24

This comment has been minimized.

Show comment
Hide comment
@sigmavirus24

sigmavirus24 Oct 2, 2013

Member

Thanks @ahmadia! I'll close this when you let us know the outcome. Otherwise I want it to be easy to find

Member

sigmavirus24 commented Oct 2, 2013

Thanks @ahmadia! I'll close this when you let us know the outcome. Otherwise I want it to be easy to find

@ahmadia

This comment has been minimized.

Show comment
Hide comment
@ahmadia

ahmadia Oct 2, 2013

Until the patch lands in Cygwin's official repository, you will need to hotfix your existing Python. You can do with the following simple patch from Evgeny Sologubov which short-circuits the uuid module from attempting to load further libraries after the uuid routines have been located.

Apply the following patch to the uuid.py module in /usr/lib/python2.7

diff -r 4a318a45c4c3 Lib/uuid.py
--- a/Lib/uuid.py   Mon Aug 19 13:07:18 2013 -0400
+++ b/Lib/uuid.py   Mon Aug 19 21:41:08 2013 +0400
@@ -429,6 +429,8 @@
             _uuid_generate_random = lib.uuid_generate_random
         if hasattr(lib, 'uuid_generate_time'):
             _uuid_generate_time = lib.uuid_generate_time
+            if _uuid_generate_random is not None:
+                break  # found everything we were looking for

     # The uuid_generate_* functions are broken on MacOS X 10.5, as noted
     # in issue #8621 the function generates the same sequence of values

ahmadia commented Oct 2, 2013

Until the patch lands in Cygwin's official repository, you will need to hotfix your existing Python. You can do with the following simple patch from Evgeny Sologubov which short-circuits the uuid module from attempting to load further libraries after the uuid routines have been located.

Apply the following patch to the uuid.py module in /usr/lib/python2.7

diff -r 4a318a45c4c3 Lib/uuid.py
--- a/Lib/uuid.py   Mon Aug 19 13:07:18 2013 -0400
+++ b/Lib/uuid.py   Mon Aug 19 21:41:08 2013 +0400
@@ -429,6 +429,8 @@
             _uuid_generate_random = lib.uuid_generate_random
         if hasattr(lib, 'uuid_generate_time'):
             _uuid_generate_time = lib.uuid_generate_time
+            if _uuid_generate_random is not None:
+                break  # found everything we were looking for

     # The uuid_generate_* functions are broken on MacOS X 10.5, as noted
     # in issue #8621 the function generates the same sequence of values
@sigmavirus24

This comment has been minimized.

Show comment
Hide comment
@sigmavirus24

sigmavirus24 Oct 3, 2013

Member

Since there's a solution here, I'm actually tempted to close this. How do you feel @Lukasa ?

Member

sigmavirus24 commented Oct 3, 2013

Since there's a solution here, I'm actually tempted to close this. How do you feel @Lukasa ?

@ahmadia

This comment has been minimized.

Show comment
Hide comment
@ahmadia

ahmadia Oct 3, 2013

Here's a two-liner to hotfix CPython from your Cygwin shell until the patch lands in Cygwin-apps (assuming you've got wget installed):

wget http://bugs.python.org/file31377/uuid.patch 
patch -d /usr/lib/python2.7 -p2 < uuid.patch 

+1 on closing this issue.

ahmadia commented Oct 3, 2013

Here's a two-liner to hotfix CPython from your Cygwin shell until the patch lands in Cygwin-apps (assuming you've got wget installed):

wget http://bugs.python.org/file31377/uuid.patch 
patch -d /usr/lib/python2.7 -p2 < uuid.patch 

+1 on closing this issue.

@sigmavirus24

This comment has been minimized.

Show comment
Hide comment
@sigmavirus24
Member

sigmavirus24 commented Oct 4, 2013

@Lukasa

This comment has been minimized.

Show comment
Hide comment
@Lukasa

Lukasa Oct 5, 2013

Member

Oh man I meant to write on this and I didn't. I think we should keep it open until cygwin fixes itself. Based on the pain we had with header keys after that issue got closed, I suspect people don't look at closed issues.

Member

Lukasa commented Oct 5, 2013

Oh man I meant to write on this and I didn't. I think we should keep it open until cygwin fixes itself. Based on the pain we had with header keys after that issue got closed, I suspect people don't look at closed issues.

@sigmavirus24

This comment has been minimized.

Show comment
Hide comment
@sigmavirus24

sigmavirus24 Oct 5, 2013

Member

@Lukasa sometimes people don't even look at open ones =P. Either way you're correct. I wonder if we might want a special tag for this kind of issue (where there's a problem we cannot resolve but has some kind of fix available.

Member

sigmavirus24 commented Oct 5, 2013

@Lukasa sometimes people don't even look at open ones =P. Either way you're correct. I wonder if we might want a special tag for this kind of issue (where there's a problem we cannot resolve but has some kind of fix available.

@gregersn

This comment has been minimized.

Show comment
Hide comment
@gregersn

gregersn Oct 7, 2013

I'm amazed with this process/progress! Awesome work!

gregersn commented Oct 7, 2013

I'm amazed with this process/progress! Awesome work!

@sigmavirus24

This comment has been minimized.

Show comment
Hide comment
@sigmavirus24

sigmavirus24 Oct 7, 2013

Member

It's all @ahmadia's fault. ;) Thank him since he found the relevant issues and possible solutions.

Member

sigmavirus24 commented Oct 7, 2013

It's all @ahmadia's fault. ;) Thank him since he found the relevant issues and possible solutions.

@joshfriend

This comment has been minimized.

Show comment
Hide comment
@joshfriend

joshfriend Oct 7, 2013

I just tried @ahmadia's fix and got the following messages from the patch tool:

$ patch -d /usr/lib/python2.7 -p2 < uuid.patch
patching file uuid.py
Reversed (or previously applied) patch detected!  Assume -R? [n]
Apply anyway? [n]
Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file uuid.py.rej

When I looked at /usr/lib/python2.7/uuid.py I noticed that the patch was already applied even though I had only just minutes before done a clean install of cygwin64. Importing uuid in python still segfaults even with the fix applied.

joshfriend commented Oct 7, 2013

I just tried @ahmadia's fix and got the following messages from the patch tool:

$ patch -d /usr/lib/python2.7 -p2 < uuid.patch
patching file uuid.py
Reversed (or previously applied) patch detected!  Assume -R? [n]
Apply anyway? [n]
Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file uuid.py.rej

When I looked at /usr/lib/python2.7/uuid.py I noticed that the patch was already applied even though I had only just minutes before done a clean install of cygwin64. Importing uuid in python still segfaults even with the fix applied.

@ahmadia

This comment has been minimized.

Show comment
Hide comment
@ahmadia

ahmadia Oct 7, 2013

Looks like the uuid patch landed on October 3rd

Not sure what's causing your problem, it sounds like something subtly wrong with the stack that we may not have caught upstream :(

Can you provide the backtrace?

ahmadia commented Oct 7, 2013

Looks like the uuid patch landed on October 3rd

Not sure what's causing your problem, it sounds like something subtly wrong with the stack that we may not have caught upstream :(

Can you provide the backtrace?

@joshfriend

This comment has been minimized.

Show comment
Hide comment
@joshfriend

joshfriend Oct 7, 2013

I was unable to get a Python stacktrace since the interpreter segfaults and exits immediately. I ran an import uuid statement with PDB and found where it died though:

> /usr/lib/python2.7/uuid.py(407)<module>()
-> if hasattr(lib, 'uuid_generate_random'):
(Pdb) s
--Call--
> /usr/lib/python2.7/ctypes/__init__.py(375)__getattr__()
-> def __getattr__(self, name):
(Pdb) s
> /usr/lib/python2.7/ctypes/__init__.py(376)__getattr__()
-> if name.startswith('__') and name.endswith('__'):
(Pdb) s
> /usr/lib/python2.7/ctypes/__init__.py(378)__getattr__()
-> func = self.__getitem__(name)
(Pdb) s
--Call--
> /usr/lib/python2.7/ctypes/__init__.py(382)__getitem__()
-> def __getitem__(self, name_or_ordinal):
(Pdb) s
> /usr/lib/python2.7/ctypes/__init__.py(383)__getitem__()
-> func = self._FuncPtr((name_or_ordinal, self))
(Pdb) s

And then it died.

Does that help?

joshfriend commented Oct 7, 2013

I was unable to get a Python stacktrace since the interpreter segfaults and exits immediately. I ran an import uuid statement with PDB and found where it died though:

> /usr/lib/python2.7/uuid.py(407)<module>()
-> if hasattr(lib, 'uuid_generate_random'):
(Pdb) s
--Call--
> /usr/lib/python2.7/ctypes/__init__.py(375)__getattr__()
-> def __getattr__(self, name):
(Pdb) s
> /usr/lib/python2.7/ctypes/__init__.py(376)__getattr__()
-> if name.startswith('__') and name.endswith('__'):
(Pdb) s
> /usr/lib/python2.7/ctypes/__init__.py(378)__getattr__()
-> func = self.__getitem__(name)
(Pdb) s
--Call--
> /usr/lib/python2.7/ctypes/__init__.py(382)__getitem__()
-> def __getitem__(self, name_or_ordinal):
(Pdb) s
> /usr/lib/python2.7/ctypes/__init__.py(383)__getitem__()
-> func = self._FuncPtr((name_or_ordinal, self))
(Pdb) s

And then it died.

Does that help?

@ahmadia

This comment has been minimized.

Show comment
Hide comment
@ahmadia

ahmadia Oct 7, 2013

Sort of :) Can you check the hashes on uuid.py and delete the corresponding uuid.pyc and uuid.pyo files? Just want to make sure we're looking at the same code.

$ md5sum.exe /usr/lib/python2.7/uuid.py
3aa63cb0e74a2ce74d915616688233fd */usr/lib/python2.7/uuid.py

ahmadia commented Oct 7, 2013

Sort of :) Can you check the hashes on uuid.py and delete the corresponding uuid.pyc and uuid.pyo files? Just want to make sure we're looking at the same code.

$ md5sum.exe /usr/lib/python2.7/uuid.py
3aa63cb0e74a2ce74d915616688233fd */usr/lib/python2.7/uuid.py
@ahmadia

This comment has been minimized.

Show comment
Hide comment
@ahmadia

ahmadia Oct 7, 2013

A "clean install", you say? What's the output of:

cygcheck.exe -c libuuid1

If this is the problem, installing libuuid1 should fix this. Please report back so I can notify upstream.

ahmadia commented Oct 7, 2013

A "clean install", you say? What's the output of:

cygcheck.exe -c libuuid1

If this is the problem, installing libuuid1 should fix this. Please report back so I can notify upstream.

@joshfriend

This comment has been minimized.

Show comment
Hide comment
@joshfriend

joshfriend Oct 7, 2013

Looks like hashes match and libuuid1 is fine:

$ md5sum.exe /usr/lib/python2.7/uuid.py
3aa63cb0e74a2ce74d915616688233fd */usr/lib/python2.7/uuid.py
$ cygcheck.exe -c libuuid1
Cygwin Package Information
Package              Version        Status
libuuid1             2.21.2-1       OK

I've tried removing uuid.pyc/uuid.pyo before too and that didn't help either. Anything else you want me to try?

joshfriend commented Oct 7, 2013

Looks like hashes match and libuuid1 is fine:

$ md5sum.exe /usr/lib/python2.7/uuid.py
3aa63cb0e74a2ce74d915616688233fd */usr/lib/python2.7/uuid.py
$ cygcheck.exe -c libuuid1
Cygwin Package Information
Package              Version        Status
libuuid1             2.21.2-1       OK

I've tried removing uuid.pyc/uuid.pyo before too and that didn't help either. Anything else you want me to try?

@sigmavirus24

This comment has been minimized.

Show comment
Hide comment
@sigmavirus24

sigmavirus24 Oct 7, 2013

Member

@Creepr can you import uuid from the interactive prompt?

Member

sigmavirus24 commented Oct 7, 2013

@Creepr can you import uuid from the interactive prompt?

@joshfriend

This comment has been minimized.

Show comment
Hide comment
@joshfriend

joshfriend Oct 7, 2013

@sigmavirus24 Nope. It just quits with no traceback.

joshfriend commented Oct 7, 2013

@sigmavirus24 Nope. It just quits with no traceback.

@ahmadia

This comment has been minimized.

Show comment
Hide comment
@ahmadia

ahmadia Oct 7, 2013

@Creepr It looks like something is wrong with your cyguuid-1.dll.

The following bit of code runs in uuid correctly for me. You can just execute directly in Python to test.

>>> import ctypes.util
>>> ctypes.util.find_library('uuid')
'cyguuid-1.dll'
>>> lib = ctypes.CDLL('cyguuid-1.dll')
>>> hasattr(lib, 'uuid_generate_random')
True

ahmadia commented Oct 7, 2013

@Creepr It looks like something is wrong with your cyguuid-1.dll.

The following bit of code runs in uuid correctly for me. You can just execute directly in Python to test.

>>> import ctypes.util
>>> ctypes.util.find_library('uuid')
'cyguuid-1.dll'
>>> lib = ctypes.CDLL('cyguuid-1.dll')
>>> hasattr(lib, 'uuid_generate_random')
True
@ahmadia

This comment has been minimized.

Show comment
Hide comment
@ahmadia

ahmadia Oct 7, 2013

Also, I hope this goes without saying, but you should have /usr/bin on your PATH and correct permissions or Python won't be able to load the DLL. If you get OSError: Permission denied, you need to make sure those are both right.

ahmadia commented Oct 7, 2013

Also, I hope this goes without saying, but you should have /usr/bin on your PATH and correct permissions or Python won't be able to load the DLL. If you get OSError: Permission denied, you need to make sure those are both right.

@joshfriend

This comment has been minimized.

Show comment
Hide comment
@joshfriend

joshfriend Oct 7, 2013

My output looks slightly different when running find_library() (returns None):

>>> import ctypes.util
>>> ctypes.util.find_library('uuid')
>>> lib = ctypes.CDLL('cyguuid-1.dll')
>>> hasattr(lib, 'uuid_generate_random')
True

I tried re-installing the uuid package with the cygwin installer and also a rebaseall. Neither worked.

/usr/bin is in my PATH with 755 permissions.

Looks like I can hack the uuid module for now to have the DLL names hardcoded:

for libname in ['cyguuid-1.dll', 'cygwin1.dll']:
    try:
        lib = ctypes.CDLL(libname)
    except:
        continue
    if hasattr(lib, 'uuid_generate_random'):
        _uuid_generate_random = lib.uuid_generate_random

Doing that seems to have fixed my issues, though it is probably not an ideal solution.

joshfriend commented Oct 7, 2013

My output looks slightly different when running find_library() (returns None):

>>> import ctypes.util
>>> ctypes.util.find_library('uuid')
>>> lib = ctypes.CDLL('cyguuid-1.dll')
>>> hasattr(lib, 'uuid_generate_random')
True

I tried re-installing the uuid package with the cygwin installer and also a rebaseall. Neither worked.

/usr/bin is in my PATH with 755 permissions.

Looks like I can hack the uuid module for now to have the DLL names hardcoded:

for libname in ['cyguuid-1.dll', 'cygwin1.dll']:
    try:
        lib = ctypes.CDLL(libname)
    except:
        continue
    if hasattr(lib, 'uuid_generate_random'):
        _uuid_generate_random = lib.uuid_generate_random

Doing that seems to have fixed my issues, though it is probably not an ideal solution.

@ahmadia

This comment has been minimized.

Show comment
Hide comment
@ahmadia

ahmadia Oct 7, 2013

Weird. Is anybody else able to test with a Cygwin64 box?

ahmadia commented Oct 7, 2013

Weird. Is anybody else able to test with a Cygwin64 box?

@joshfriend

This comment has been minimized.

Show comment
Hide comment
@joshfriend

joshfriend Oct 7, 2013

I just had a friend at work try it out. She had the same results as I did (find_library('uuid') returns None).

joshfriend commented Oct 7, 2013

I just had a friend at work try it out. She had the same results as I did (find_library('uuid') returns None).

@ahmadia

This comment has been minimized.

Show comment
Hide comment
@ahmadia

ahmadia Oct 7, 2013

I'm sorry, this isn't obvious at all, but I suspect you need libuuid-devel. Looking at the source code for find_library:

elif sys.platform == "cygwin":
    def find_library(name):
        for libdir in ['/usr/lib', '/usr/local/lib']:
            for libext in ['lib%s.dll.a' % name, 'lib%s.a' % name]:
                implib = os.path.join(libdir, libext)
                if not os.path.exists(implib):
                    continue
                cmd = "dlltool -I " + implib + " 2>/dev/null"
                res = os.popen(cmd).read().replace("\n","")
                if not res:
                    continue
                return res
        return None

It's looking for the stub DLL files needed for generating the actual libraries (that are populated by the -devel versions of the Cygwin packages).

ahmadia commented Oct 7, 2013

I'm sorry, this isn't obvious at all, but I suspect you need libuuid-devel. Looking at the source code for find_library:

elif sys.platform == "cygwin":
    def find_library(name):
        for libdir in ['/usr/lib', '/usr/local/lib']:
            for libext in ['lib%s.dll.a' % name, 'lib%s.a' % name]:
                implib = os.path.join(libdir, libext)
                if not os.path.exists(implib):
                    continue
                cmd = "dlltool -I " + implib + " 2>/dev/null"
                res = os.popen(cmd).read().replace("\n","")
                if not res:
                    continue
                return res
        return None

It's looking for the stub DLL files needed for generating the actual libraries (that are populated by the -devel versions of the Cygwin packages).

@sigmavirus24

This comment has been minimized.

Show comment
Hide comment
@sigmavirus24

sigmavirus24 Oct 7, 2013

Member

Hey guys,

I appreciate you trying to work this out but if you could work somewhere else and then post the steps for debugging and resolution in one concise comment. I'd rather not mute this thread but I'd also rather not have a ton of email to read when I get home tonight.

Member

sigmavirus24 commented Oct 7, 2013

Hey guys,

I appreciate you trying to work this out but if you could work somewhere else and then post the steps for debugging and resolution in one concise comment. I'd rather not mute this thread but I'd also rather not have a ton of email to read when I get home tonight.

@joshfriend

This comment has been minimized.

Show comment
Hide comment
@joshfriend

joshfriend Oct 7, 2013

Sorry @sigmavirus24, I promise to stop after this post. @ahmadia: you can email me at the address listed on my profile.

joshfriend commented Oct 7, 2013

Sorry @sigmavirus24, I promise to stop after this post. @ahmadia: you can email me at the address listed on my profile.

@sigmavirus24

This comment has been minimized.

Show comment
Hide comment
@sigmavirus24

sigmavirus24 Oct 8, 2013

Member

No need to apologize @Creepr. I'm really interested in what the final result is that you two find. I just get a ton of email and this is more of a Cygwin issue than a requests issue.

Member

sigmavirus24 commented Oct 8, 2013

No need to apologize @Creepr. I'm really interested in what the final result is that you two find. I just get a ton of email and this is more of a Cygwin issue than a requests issue.

@jvstein

This comment has been minimized.

Show comment
Hide comment
@jvstein

jvstein Oct 8, 2013

I had the same problem and tracked it back to the usage of dlltool in find_library.

cmd = "dlltool -I " + implib + " 2>/dev/null"
res = os.popen(cmd).read().replace("\n","")
if not res:
    continue

Installing the 'binutils' cygwin package solved it for me.

jvstein commented Oct 8, 2013

I had the same problem and tracked it back to the usage of dlltool in find_library.

cmd = "dlltool -I " + implib + " 2>/dev/null"
res = os.popen(cmd).read().replace("\n","")
if not res:
    continue

Installing the 'binutils' cygwin package solved it for me.

@joshfriend

This comment has been minimized.

Show comment
Hide comment
@joshfriend

joshfriend Oct 8, 2013

Like @jvstein I traced my issue back to dlltool:

$ dlltool -I /usr/lib/libuuid.dll.a
dlltool: Unable to open object file: cyguuid-1.dll: No such file or directory

However, I checked and did have binutils installed. I had several co-workers with the same issue as me run the latest cygwin setup.exe again and reinstall binutils. Unfortunately, this did not fix the issue for them (or me). I did also verify that we had the same versions of installed packages using cygcheck -s. I then removed cygwin and reinstalled from scratch, which fixed the issue for me. It may be worth noting that my previous cygwin installation was performed before the Oct 3rd patch to uuid.py.

I suppose the resolution steps are:

  1. Verify that python has the patch that was previously mentioned
  2. Update/Reinstall/Install binutils as needed
  3. As a last resort, reinstall cygwin64 from scratch

joshfriend commented Oct 8, 2013

Like @jvstein I traced my issue back to dlltool:

$ dlltool -I /usr/lib/libuuid.dll.a
dlltool: Unable to open object file: cyguuid-1.dll: No such file or directory

However, I checked and did have binutils installed. I had several co-workers with the same issue as me run the latest cygwin setup.exe again and reinstall binutils. Unfortunately, this did not fix the issue for them (or me). I did also verify that we had the same versions of installed packages using cygcheck -s. I then removed cygwin and reinstalled from scratch, which fixed the issue for me. It may be worth noting that my previous cygwin installation was performed before the Oct 3rd patch to uuid.py.

I suppose the resolution steps are:

  1. Verify that python has the patch that was previously mentioned
  2. Update/Reinstall/Install binutils as needed
  3. As a last resort, reinstall cygwin64 from scratch
@mstave

This comment has been minimized.

Show comment
Hide comment
@mstave

mstave Nov 26, 2013

Installing libuuid-devel on Win7 Cygwin64 fixed it for me.

mstave commented Nov 26, 2013

Installing libuuid-devel on Win7 Cygwin64 fixed it for me.

@tinkerer

This comment has been minimized.

Show comment
Hide comment
@tinkerer

tinkerer Dec 30, 2013

I encountered this error while trying to pip install django-allauth on Windows 7 in Cygwin64. libuuid-devel on Cygwin64 also fixed it for me

tinkerer commented Dec 30, 2013

I encountered this error while trying to pip install django-allauth on Windows 7 in Cygwin64. libuuid-devel on Cygwin64 also fixed it for me

@Lukasa

This comment has been minimized.

Show comment
Hide comment
@Lukasa

Lukasa Feb 3, 2014

Member

Right, in that case I proclaim this issue "Not our fault" and am closing it. =)

Thank you all for your contributions and insight!

Member

Lukasa commented Feb 3, 2014

Right, in that case I proclaim this issue "Not our fault" and am closing it. =)

Thank you all for your contributions and insight!

@jdunn-big

This comment has been minimized.

Show comment
Hide comment
@jdunn-big

jdunn-big Mar 21, 2014

Wow! Thank you.

jdunn-big commented Mar 21, 2014

Wow! Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment