New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PyCrypto ImportError with 32-bit Linux build #124

Closed
lanterndev opened this Issue Mar 28, 2014 · 6 comments

Comments

Projects
None yet
2 participants
@lanterndev
Contributor

lanterndev commented Mar 28, 2014

Hey Kevin, I'm trying to get an fteproxy build working in a 32-bit Ubuntu 13.10 VM. Running ./bin/fteproxy -h from a clone of latest master works fine, but after running make dist, fteproxy -h from the built version produces the following:

ubuntu@ubuntu-VirtualBox ~/C/fteproxy> ./dist/fteproxy-0.2.9-linux-i686/fteproxy -h
Traceback (most recent call last):
  File "<string>", line 34, in <module>
  File "/usr/local/lib/python2.7/dist-packages/PyInstaller/loader/pyi_importers.py", line 270, in load_module
    exec(bytecode, module.__dict__)
  File "/home/ubuntu/Code/fteproxy/build/fteproxy/out00-PYZ.pyz/fte", line 24, in <module>

  File "/usr/local/lib/python2.7/dist-packages/PyInstaller/loader/pyi_importers.py", line 270, in load_module
    exec(bytecode, module.__dict__)
  File "/home/ubuntu/Code/fteproxy/build/fteproxy/out00-PYZ.pyz/fte.encoder", line 23, in <module>
  File "/usr/local/lib/python2.7/dist-packages/PyInstaller/loader/pyi_importers.py", line 270, in load_module
    exec(bytecode, module.__dict__)
  File "/home/ubuntu/Code/fteproxy/build/fteproxy/out00-PYZ.pyz/fte.bit_ops", line 21, in <module>
  File "/usr/local/lib/python2.7/dist-packages/PyInstaller/loader/pyi_importers.py", line 270, in load_module
    exec(bytecode, module.__dict__)
  File "/home/ubuntu/Code/fteproxy/build/fteproxy/out00-PYZ.pyz/Crypto.Random", line 29, in <module>
  File "/usr/local/lib/python2.7/dist-packages/PyInstaller/loader/pyi_importers.py", line 270, in load_module
    exec(bytecode, module.__dict__)
  File "/home/ubuntu/Code/fteproxy/build/fteproxy/out00-PYZ.pyz/Crypto.Random._UserFriendlyRNG", line 38, in <module>
  File "/usr/local/lib/python2.7/dist-packages/PyInstaller/loader/pyi_importers.py", line 270, in load_module
    exec(bytecode, module.__dict__)
  File "/home/ubuntu/Code/fteproxy/build/fteproxy/out00-PYZ.pyz/Crypto.Random.Fortuna.FortunaAccumulator", line 39, in <module>
  File "/usr/local/lib/python2.7/dist-packages/PyInstaller/loader/pyi_importers.py", line 270, in load_module
    exec(bytecode, module.__dict__)
  File "/home/ubuntu/Code/fteproxy/build/fteproxy/out00-PYZ.pyz/Crypto.Random.Fortuna.FortunaGenerator", line 35, in <module>
  File "/usr/local/lib/python2.7/dist-packages/PyInstaller/loader/pyi_importers.py", line 270, in load_module
    exec(bytecode, module.__dict__)
  File "/home/ubuntu/Code/fteproxy/build/fteproxy/out00-PYZ.pyz/Crypto.Util.Counter", line 59, in <module>
ImportError: cannot import name _counter

Is it possible _counter.so didn't successfully make it from /usr/local/lib/python2.7/dist-packages/Crypto/Util/_counter.so into the out00-PYZ.pyz bundle, or did I just mess something up on my system?

Thanks!

@kpdyer kpdyer added this to the 0.2.10 milestone Mar 30, 2014

@kpdyer kpdyer self-assigned this Mar 30, 2014

@kpdyer

This comment has been minimized.

Show comment
Hide comment
@kpdyer

kpdyer Mar 30, 2014

Owner

Noted, and investigating.

Owner

kpdyer commented Mar 30, 2014

Noted, and investigating.

@kpdyer

This comment has been minimized.

Show comment
Hide comment
@kpdyer

kpdyer Mar 31, 2014

Owner

I was able to easily recreate this on Ubuntu 13.10 32-bit (Desktop) using the standard fteproxy build instructions [1] on a fresh VM.

I tried to figure out how to resolve this problem. As an example, I tried replacing the pycypto install that is default on Ubuntu with the one from pip, but this breaks Ubuntu. I also tried using PyCrypto's hiddenimport [2] functionality to explicitly include _counter and Crypto.Util.counter, but to no avail.

In addition, it appears that this problem is only on Ubuntu 13.10. This is not an issue on 10.x/12.x or 13.04.

Ultimately, the workaround for this issue is installing the latest development version of pyinstaller [3]. For sure not ideal, but I think this is good enough for now.

Can you test if this workaround resolves the problem for you? I've updated [1] to reflect this issue.

-Kevin

[1] https://github.com/kpdyer/fteproxy/blob/master/BUILDING.md
[2] http://www.pyinstaller.org/static/docs/Manual_v1.1.html#hooks
[3] http://www.pyinstaller.org/

Owner

kpdyer commented Mar 31, 2014

I was able to easily recreate this on Ubuntu 13.10 32-bit (Desktop) using the standard fteproxy build instructions [1] on a fresh VM.

I tried to figure out how to resolve this problem. As an example, I tried replacing the pycypto install that is default on Ubuntu with the one from pip, but this breaks Ubuntu. I also tried using PyCrypto's hiddenimport [2] functionality to explicitly include _counter and Crypto.Util.counter, but to no avail.

In addition, it appears that this problem is only on Ubuntu 13.10. This is not an issue on 10.x/12.x or 13.04.

Ultimately, the workaround for this issue is installing the latest development version of pyinstaller [3]. For sure not ideal, but I think this is good enough for now.

Can you test if this workaround resolves the problem for you? I've updated [1] to reflect this issue.

-Kevin

[1] https://github.com/kpdyer/fteproxy/blob/master/BUILDING.md
[2] http://www.pyinstaller.org/static/docs/Manual_v1.1.html#hooks
[3] http://www.pyinstaller.org/

@kpdyer kpdyer closed this Mar 31, 2014

@lanterndev

This comment has been minimized.

Show comment
Hide comment
@lanterndev

lanterndev Apr 1, 2014

Contributor

Thanks for looking into this. Just had a chance to give this a shot and here's what happened:

ubuntu@ubuntu-VirtualBox ~>  sudo pip install https://github.com/pyinstaller/pyinstaller/tarball/develop
[sudo] password for ubuntu: 
Downloading/unpacking https://github.com/pyinstaller/pyinstaller/tarball/develop
  Downloading develop (unknown size): 4.9MB downloaded
  Running setup.py (path:/tmp/pip-8UoCj0-build/setup.py) egg_info for package from https://github.com/pyinstaller/pyinstaller/tarball/develop
    Traceback (most recent call last):
      File "<string>", line 17, in <module>
      File "/tmp/pip-8UoCj0-build/setup.py", line 125, in <module>
        version=get_version(),
      File "PyInstaller/__init__.py", line 123, in get_version
        if VERSION[3] == 'dev' and VERSION[4] > 0:
    IndexError: tuple index out of range
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):

  File "<string>", line 17, in <module>

  File "/tmp/pip-8UoCj0-build/setup.py", line 125, in <module>

    version=get_version(),

  File "PyInstaller/__init__.py", line 123, in get_version

    if VERSION[3] == 'dev' and VERSION[4] > 0:

IndexError: tuple index out of range

----------------------------------------
Cleaning up...
Command python setup.py egg_info failed with error code 1 in /tmp/pip-8UoCj0-build
Storing debug log for failure in /home/ubuntu/.pip/pip.log
ubuntu@ubuntu-VirtualBox ~> 

Inspecting VERSION, it turns out it's getting the value (2, 1, 1, 'dev'), hence VERSION[4] causing the IndexError. So I patched it as follows:

diff --git a/PyInstaller/__init__.py b/PyInstaller/__init__.py
index fb6bb57..13601eb 100644
--- a/PyInstaller/__init__.py
+++ b/PyInstaller/__init__.py
@@ -120,6 +120,6 @@ def get_version():
     if len(VERSION) >= 4 and VERSION[3]:
         version = '%s%s' % (version, VERSION[3])
         # include git revision in version string
-        if VERSION[3] == 'dev' and VERSION[4] > 0:
+        if VERSION[3] == 'dev' and len(VERSION) >= 5 and VERSION[4] > 0:
             version = '%s-%s' % (version, VERSION[4])
     return version

and then was able to pip install from it and then build fteproxy successfully.

What revision of pyinstaller did you pull when you tried this? I'm on pyinstaller/pyinstaller@
3c0046a383. If you ended up with an earlier revision, maybe they just introduced this bug? Worth updating the build instructions to suggest pegging to a specific revision of pyinstaller?

Thanks again for the help with this!

Contributor

lanterndev commented Apr 1, 2014

Thanks for looking into this. Just had a chance to give this a shot and here's what happened:

ubuntu@ubuntu-VirtualBox ~>  sudo pip install https://github.com/pyinstaller/pyinstaller/tarball/develop
[sudo] password for ubuntu: 
Downloading/unpacking https://github.com/pyinstaller/pyinstaller/tarball/develop
  Downloading develop (unknown size): 4.9MB downloaded
  Running setup.py (path:/tmp/pip-8UoCj0-build/setup.py) egg_info for package from https://github.com/pyinstaller/pyinstaller/tarball/develop
    Traceback (most recent call last):
      File "<string>", line 17, in <module>
      File "/tmp/pip-8UoCj0-build/setup.py", line 125, in <module>
        version=get_version(),
      File "PyInstaller/__init__.py", line 123, in get_version
        if VERSION[3] == 'dev' and VERSION[4] > 0:
    IndexError: tuple index out of range
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):

  File "<string>", line 17, in <module>

  File "/tmp/pip-8UoCj0-build/setup.py", line 125, in <module>

    version=get_version(),

  File "PyInstaller/__init__.py", line 123, in get_version

    if VERSION[3] == 'dev' and VERSION[4] > 0:

IndexError: tuple index out of range

----------------------------------------
Cleaning up...
Command python setup.py egg_info failed with error code 1 in /tmp/pip-8UoCj0-build
Storing debug log for failure in /home/ubuntu/.pip/pip.log
ubuntu@ubuntu-VirtualBox ~> 

Inspecting VERSION, it turns out it's getting the value (2, 1, 1, 'dev'), hence VERSION[4] causing the IndexError. So I patched it as follows:

diff --git a/PyInstaller/__init__.py b/PyInstaller/__init__.py
index fb6bb57..13601eb 100644
--- a/PyInstaller/__init__.py
+++ b/PyInstaller/__init__.py
@@ -120,6 +120,6 @@ def get_version():
     if len(VERSION) >= 4 and VERSION[3]:
         version = '%s%s' % (version, VERSION[3])
         # include git revision in version string
-        if VERSION[3] == 'dev' and VERSION[4] > 0:
+        if VERSION[3] == 'dev' and len(VERSION) >= 5 and VERSION[4] > 0:
             version = '%s-%s' % (version, VERSION[4])
     return version

and then was able to pip install from it and then build fteproxy successfully.

What revision of pyinstaller did you pull when you tried this? I'm on pyinstaller/pyinstaller@
3c0046a383. If you ended up with an earlier revision, maybe they just introduced this bug? Worth updating the build instructions to suggest pegging to a specific revision of pyinstaller?

Thanks again for the help with this!

@lanterndev

This comment has been minimized.

Show comment
Hide comment
@lanterndev

lanterndev Apr 1, 2014

Contributor

submitted a PR to pyinstaller: pyinstaller/pyinstaller#118

Contributor

lanterndev commented Apr 1, 2014

submitted a PR to pyinstaller: pyinstaller/pyinstaller#118

@kpdyer

This comment has been minimized.

Show comment
Hide comment
@kpdyer

kpdyer Apr 1, 2014

Owner

Thanks for that! Looks like your PR has already been merged.

I'll add a specific revision of pyinstaller to the notes.

Owner

kpdyer commented Apr 1, 2014

Thanks for that! Looks like your PR has already been merged.

I'll add a specific revision of pyinstaller to the notes.

@lanterndev

This comment has been minimized.

Show comment
Hide comment
@lanterndev

lanterndev Apr 1, 2014

Contributor

My pleasure, and thank you!

On Tue, Apr 1, 2014 at 1:11 PM, Kevin P. Dyer notifications@github.comwrote:

Thanks for that! Looks like your PR has already been merged.

I'll add a specific revision of pyinstaller to the notes.

Reply to this email directly or view it on GitHubhttps://github.com/kpdyer/fteproxy/issues/124#issuecomment-39231755
.

Contributor

lanterndev commented Apr 1, 2014

My pleasure, and thank you!

On Tue, Apr 1, 2014 at 1:11 PM, Kevin P. Dyer notifications@github.comwrote:

Thanks for that! Looks like your PR has already been merged.

I'll add a specific revision of pyinstaller to the notes.

Reply to this email directly or view it on GitHubhttps://github.com/kpdyer/fteproxy/issues/124#issuecomment-39231755
.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment