PyCrypto ImportError with 32-bit Linux build

[lanterndev]

Hey Kevin, I'm trying to get an fteproxy build working in a 32-bit Ubuntu 13.10 VM. Running ./bin/fteproxy -h from a clone of latest master works fine, but after running make dist, fteproxy -h from the built version produces the following:

ubuntu@ubuntu-VirtualBox ~/C/fteproxy> ./dist/fteproxy-0.2.9-linux-i686/fteproxy -h
Traceback (most recent call last):
  File "<string>", line 34, in <module>
  File "/usr/local/lib/python2.7/dist-packages/PyInstaller/loader/pyi_importers.py", line 270, in load_module
    exec(bytecode, module.__dict__)
  File "/home/ubuntu/Code/fteproxy/build/fteproxy/out00-PYZ.pyz/fte", line 24, in <module>

  File "/usr/local/lib/python2.7/dist-packages/PyInstaller/loader/pyi_importers.py", line 270, in load_module
    exec(bytecode, module.__dict__)
  File "/home/ubuntu/Code/fteproxy/build/fteproxy/out00-PYZ.pyz/fte.encoder", line 23, in <module>
  File "/usr/local/lib/python2.7/dist-packages/PyInstaller/loader/pyi_importers.py", line 270, in load_module
    exec(bytecode, module.__dict__)
  File "/home/ubuntu/Code/fteproxy/build/fteproxy/out00-PYZ.pyz/fte.bit_ops", line 21, in <module>
  File "/usr/local/lib/python2.7/dist-packages/PyInstaller/loader/pyi_importers.py", line 270, in load_module
    exec(bytecode, module.__dict__)
  File "/home/ubuntu/Code/fteproxy/build/fteproxy/out00-PYZ.pyz/Crypto.Random", line 29, in <module>
  File "/usr/local/lib/python2.7/dist-packages/PyInstaller/loader/pyi_importers.py", line 270, in load_module
    exec(bytecode, module.__dict__)
  File "/home/ubuntu/Code/fteproxy/build/fteproxy/out00-PYZ.pyz/Crypto.Random._UserFriendlyRNG", line 38, in <module>
  File "/usr/local/lib/python2.7/dist-packages/PyInstaller/loader/pyi_importers.py", line 270, in load_module
    exec(bytecode, module.__dict__)
  File "/home/ubuntu/Code/fteproxy/build/fteproxy/out00-PYZ.pyz/Crypto.Random.Fortuna.FortunaAccumulator", line 39, in <module>
  File "/usr/local/lib/python2.7/dist-packages/PyInstaller/loader/pyi_importers.py", line 270, in load_module
    exec(bytecode, module.__dict__)
  File "/home/ubuntu/Code/fteproxy/build/fteproxy/out00-PYZ.pyz/Crypto.Random.Fortuna.FortunaGenerator", line 35, in <module>
  File "/usr/local/lib/python2.7/dist-packages/PyInstaller/loader/pyi_importers.py", line 270, in load_module
    exec(bytecode, module.__dict__)
  File "/home/ubuntu/Code/fteproxy/build/fteproxy/out00-PYZ.pyz/Crypto.Util.Counter", line 59, in <module>
ImportError: cannot import name _counter

Is it possible _counter.so didn't successfully make it from /usr/local/lib/python2.7/dist-packages/Crypto/Util/_counter.so into the out00-PYZ.pyz bundle, or did I just mess something up on my system?

Thanks!

[kpdyer]

Noted, and investigating.

[kpdyer]

I was able to easily recreate this on Ubuntu 13.10 32-bit (Desktop) using the standard fteproxy build instructions [1] on a fresh VM.

I tried to figure out how to resolve this problem. As an example, I tried replacing the pycypto install that is default on Ubuntu with the one from pip, but this breaks Ubuntu. I also tried using PyCrypto's hiddenimport [2] functionality to explicitly include _counter and Crypto.Util.counter, but to no avail.

In addition, it appears that this problem is only on Ubuntu 13.10. This is not an issue on 10.x/12.x or 13.04.

Ultimately, the workaround for this issue is installing the latest development version of pyinstaller [3]. For sure not ideal, but I think this is good enough for now.

Can you test if this workaround resolves the problem for you? I've updated [1] to reflect this issue.

-Kevin

[1] https://github.com/kpdyer/fteproxy/blob/master/BUILDING.md
[2] http://www.pyinstaller.org/static/docs/Manual_v1.1.html#hooks
[3] http://www.pyinstaller.org/

[lanterndev]

Thanks for looking into this. Just had a chance to give this a shot and here's what happened:

ubuntu@ubuntu-VirtualBox ~>  sudo pip install https://github.com/pyinstaller/pyinstaller/tarball/develop
[sudo] password for ubuntu: 
Downloading/unpacking https://github.com/pyinstaller/pyinstaller/tarball/develop
  Downloading develop (unknown size): 4.9MB downloaded
  Running setup.py (path:/tmp/pip-8UoCj0-build/setup.py) egg_info for package from https://github.com/pyinstaller/pyinstaller/tarball/develop
    Traceback (most recent call last):
      File "<string>", line 17, in <module>
      File "/tmp/pip-8UoCj0-build/setup.py", line 125, in <module>
        version=get_version(),
      File "PyInstaller/__init__.py", line 123, in get_version
        if VERSION[3] == 'dev' and VERSION[4] > 0:
    IndexError: tuple index out of range
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):

  File "<string>", line 17, in <module>

  File "/tmp/pip-8UoCj0-build/setup.py", line 125, in <module>

    version=get_version(),

  File "PyInstaller/__init__.py", line 123, in get_version

    if VERSION[3] == 'dev' and VERSION[4] > 0:

IndexError: tuple index out of range

----------------------------------------
Cleaning up...
Command python setup.py egg_info failed with error code 1 in /tmp/pip-8UoCj0-build
Storing debug log for failure in /home/ubuntu/.pip/pip.log
ubuntu@ubuntu-VirtualBox ~> 

Inspecting VERSION, it turns out it's getting the value (2, 1, 1, 'dev'), hence VERSION[4] causing the IndexError. So I patched it as follows:

diff --git a/PyInstaller/__init__.py b/PyInstaller/__init__.py
index fb6bb57..13601eb 100644
--- a/PyInstaller/__init__.py
+++ b/PyInstaller/__init__.py
@@ -120,6 +120,6 @@ def get_version():
     if len(VERSION) >= 4 and VERSION[3]:
         version = '%s%s' % (version, VERSION[3])
         # include git revision in version string
-        if VERSION[3] == 'dev' and VERSION[4] > 0:
+        if VERSION[3] == 'dev' and len(VERSION) >= 5 and VERSION[4] > 0:
             version = '%s-%s' % (version, VERSION[4])
     return version

and then was able to pip install from it and then build fteproxy successfully.

What revision of pyinstaller did you pull when you tried this? I'm on pyinstaller/pyinstaller@
3c0046a383. If you ended up with an earlier revision, maybe they just introduced this bug? Worth updating the build instructions to suggest pegging to a specific revision of pyinstaller?

Thanks again for the help with this!

[lanterndev]

submitted a PR to pyinstaller: pyinstaller/pyinstaller#118

[kpdyer]

Thanks for that! Looks like your PR has already been merged.

I'll add a specific revision of pyinstaller to the notes.

[lanterndev]

My pleasure, and thank you!

On Tue, Apr 1, 2014 at 1:11 PM, Kevin P. Dyer notifications@github.comwrote:

Thanks for that! Looks like your PR has already been merged.

I'll add a specific revision of pyinstaller to the notes.

Reply to this email directly or view it on GitHubhttps://github.com//issues/124#issuecomment-39231755
.