AttributeError: 'module' object has no attribute '_lib'

[Kr0nZ]

Not sure why this is happening, mitmproxy was working fine previously but now it just fails with this error msg. I haven't updated or changed anything, I even tried updating mitmproxy using (sudo pip install --upgrade mitmproxy) but I still get same msg

Traceback (most recent call last):
  File "/usr/local/bin/mitmproxy", line 43, in 
    config = process_proxy_options(parser, options)
  File "/usr/local/lib/python2.7/dist-packages/libmproxy/proxy/config.py", line 107, in process_proxy_options
    certforward = options.certforward,
  File "/usr/local/lib/python2.7/dist-packages/libmproxy/proxy/config.py", line 27, in __init__
    self.certstore = certutils.CertStore.from_store(self.confdir, CONF_BASENAME)
  File "/usr/local/lib/python2.7/dist-packages/netlib/certutils.py", line 153, in from_store
    dh = klass.load_dhparam(dhp)
  File "/usr/local/lib/python2.7/dist-packages/netlib/certutils.py", line 133, in load_dhparam
    bio = OpenSSL.SSL._lib.BIO_new_file(path, b"r")
AttributeError: 'module' object has no attribute '_lib'

[mhils]

Hi there,

which version of pyOpenSSL and cryptography are you using? (pip freeze)

Cheers,
Max

[Kr0nZ]

pyOpenSSL==0.14
cryptography==0.4

Tho I dont think I have updated any of these since the last time mitmproxy was working.

[mhils]

Hm, weird. What's the output of the following?

import OpenSSL
print OpenSSL.__version__
dir(OpenSSL.SSL)

Cheers,
Max

[Kr0nZ]

>>> import OpenSSL
>>> print OpenSSL.__version__
0.13
>>> dir(OpenSSL.SSL)
['Connection', 'ConnectionType', 'Context', 'ContextType', 'Error', 'FILETYPE_ASN1', 'FILETYPE_PEM', 'OPENSSL_VERSION_NUMBER', 'OP_ALL', 'OP_CIPHER_SERVER_PREFERENCE', 'OP_COOKIE_EXCHANGE', 'OP_DONT_INSERT_EMPTY_FRAGMENTS', 'OP_EPHEMERAL_RSA', 'OP_MICROSOFT_BIG_SSLV3_BUFFER', 'OP_MICROSOFT_SESS_ID_BUG', 'OP_MSIE_SSLV2_RSA_PADDING', 'OP_NETSCAPE_CA_DN_BUG', 'OP_NETSCAPE_CHALLENGE_BUG', 'OP_NETSCAPE_DEMO_CIPHER_CHANGE_BUG', 'OP_NETSCAPE_REUSE_CIPHER_CHANGE_BUG', 'OP_NO_QUERY_MTU', 'OP_NO_SSLv2', 'OP_NO_SSLv3', 'OP_NO_TICKET', 'OP_NO_TLSv1', 'OP_PKCS1_CHECK_1', 'OP_PKCS1_CHECK_2', 'OP_SINGLE_DH_USE', 'OP_SSLEAY_080_CLIENT_DH_BUG', 'OP_SSLREF2_REUSE_CERT_TYPE_BUG', 'OP_TLS_BLOCK_PADDING_BUG', 'OP_TLS_D5_BUG', 'OP_TLS_ROLLBACK_BUG', 'RECEIVED_SHUTDOWN', 'SENT_SHUTDOWN', 'SSLEAY_BUILT_ON', 'SSLEAY_CFLAGS', 'SSLEAY_DIR', 'SSLEAY_PLATFORM', 'SSLEAY_VERSION', 'SSL_CB_ACCEPT_EXIT', 'SSL_CB_ACCEPT_LOOP', 'SSL_CB_ALERT', 'SSL_CB_CONNECT_EXIT', 'SSL_CB_CONNECT_LOOP', 'SSL_CB_EXIT', 'SSL_CB_HANDSHAKE_DONE', 'SSL_CB_HANDSHAKE_START', 'SSL_CB_LOOP', 'SSL_CB_READ', 'SSL_CB_READ_ALERT', 'SSL_CB_WRITE', 'SSL_CB_WRITE_ALERT', 'SSL_ST_ACCEPT', 'SSL_ST_BEFORE', 'SSL_ST_CONNECT', 'SSL_ST_INIT', 'SSL_ST_MASK', 'SSL_ST_OK', 'SSL_ST_RENEGOTIATE', 'SSLeay_version', 'SSLv23_METHOD', 'SSLv2_METHOD', 'SSLv3_METHOD', 'SysCallError', 'TLSv1_METHOD', 'VERIFY_CLIENT_ONCE', 'VERIFY_FAIL_IF_NO_PEER_CERT', 'VERIFY_NONE', 'VERIFY_PEER', 'WantReadError', 'WantWriteError', 'WantX509LookupError', 'ZeroReturnError', '_C_API', '__doc__', '__file__', '__name__', '__package__']

Thanks, I think I see the problem now.

For some reason I have both 0.13 and 0.14 installed, not sure how that happened. 0.13 is located first in sys.path at '/usr/lib/python2.7/dist-packages/OpenSSL/version.py', and 0.14 is at '/usr/local/lib/python2.7/dist-packages/OpenSSL/version.py'.

I removed the 0.13 version, by moving '/usr/lib/python2.7/dist-packages/OpenSSL' to /tmp/, and mitmproxy is working now.

Thanks for the help.

[mhils]

Excellent. If there are any other issues, let me know! :-)

[bazitov]

This thread was really helpful, thanks.

Just for the other having the same issue my old OpenSSL 0.13 was here:
/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/OpenSSL

When you move it away, everything starts working again:
sudo mv /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/OpenSSL /tmp/

[stevencheung]

I've encountered a similar problem installing mitmproxy on mac osx 10.9.4
I got the source for netlib and mitmproxy from github.
Installing netlib and mitmproxy appeared to work (see below for output).
However, when I tried to run mitmproxy, I got the following error:

% mitmproxy
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/bin/mitmproxy", line 43, in <module>
    config = process_proxy_options(parser, options)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/libmproxy/proxy/config.py", line 127, in process_proxy_options
    certforward=options.certforward,
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/libmproxy/proxy/config.py", line 48, in __init__
    self.certstore = certutils.CertStore.from_store(self.confdir, CONF_BASENAME)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/netlib/certutils.py", line 156, in from_store
    dh = klass.load_dhparam(dhp)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/netlib/certutils.py", line 136, in load_dhparam
    bio = OpenSSL.SSL._lib.BIO_new_file(path, b"r")
AttributeError: 'module' object has no attribute '_lib'

I've confirmed that the openssl version is 0.14 using the method suggested by Maximilian

% python
Python 2.7.8 (default, Jul 13 2014, 17:11:32) 
[GCC 4.2.1 Compatible Apple LLVM 5.1 (clang-503.0.40)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import OpenSSL
>>> print OpenSSL.__version__
0.14
>>> dir(OpenSSL.SSL)
['Connection', 'ConnectionType', 'Context', 'ContextType', 'Error', 'FILETYPE_ASN1', 'FILETYPE_PEM', 'OPENSSL_VERSION_NUMBER', 'OP_ALL', 'OP_CIPHER_SERVER_PREFERENCE', 'OP_COOKIE_EXCHANGE', 'OP_DONT_INSERT_EMPTY_FRAGMENTS', 'OP_EPHEMERAL_RSA', 'OP_MICROSOFT_BIG_SSLV3_BUFFER', 'OP_MICROSOFT_SESS_ID_BUG', 'OP_MSIE_SSLV2_RSA_PADDING', 'OP_NETSCAPE_CA_DN_BUG', 'OP_NETSCAPE_CHALLENGE_BUG', 'OP_NETSCAPE_DEMO_CIPHER_CHANGE_BUG', 'OP_NETSCAPE_REUSE_CIPHER_CHANGE_BUG', 'OP_NO_QUERY_MTU', 'OP_NO_SSLv2', 'OP_NO_SSLv3', 'OP_NO_TICKET', 'OP_NO_TLSv1', 'OP_PKCS1_CHECK_1', 'OP_PKCS1_CHECK_2', 'OP_SINGLE_DH_USE', 'OP_SSLEAY_080_CLIENT_DH_BUG', 'OP_SSLREF2_REUSE_CERT_TYPE_BUG', 'OP_TLS_BLOCK_PADDING_BUG', 'OP_TLS_D5_BUG', 'OP_TLS_ROLLBACK_BUG', 'RECEIVED_SHUTDOWN', 'SENT_SHUTDOWN', 'SSLEAY_BUILT_ON', 'SSLEAY_CFLAGS', 'SSLEAY_DIR', 'SSLEAY_PLATFORM', 'SSLEAY_VERSION', 'SSL_CB_ACCEPT_EXIT', 'SSL_CB_ACCEPT_LOOP', 'SSL_CB_ALERT', 'SSL_CB_CONNECT_EXIT', 'SSL_CB_CONNECT_LOOP', 'SSL_CB_EXIT', 'SSL_CB_HANDSHAKE_DONE', 'SSL_CB_HANDSHAKE_START', 'SSL_CB_LOOP', 'SSL_CB_READ', 'SSL_CB_READ_ALERT', 'SSL_CB_WRITE', 'SSL_CB_WRITE_ALERT', 'SSL_ST_ACCEPT', 'SSL_ST_BEFORE', 'SSL_ST_CONNECT', 'SSL_ST_INIT', 'SSL_ST_MASK', 'SSL_ST_OK', 'SSL_ST_RENEGOTIATE', 'SSLeay_version', 'SSLv23_METHOD', 'SSLv2_METHOD', 'SSLv3_METHOD', 'SysCallError', 'TLSv1_METHOD', 'VERIFY_CLIENT_ONCE', 'VERIFY_FAIL_IF_NO_PEER_CERT', 'VERIFY_NONE', 'VERIFY_PEER', 'WantReadError', 'WantWriteError', 'WantX509LookupError', 'ZeroReturnError', '_C_API', '__doc__', '__file__', '__name__', '__package__']

Moreover, the mac doesn't appear to have openssl 0.13 there:

% ls -lt /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/OpenSSL
ls: /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/OpenSSL: No such file or directory

Here is the output for installing mitmproxy

% sudo pip install ~/git/mitmproxy
Unpacking ./mitmproxy
  Running setup.py (path:/tmp/pip-8mXnyu-build/setup.py) egg_info for package from file:///Users/local/git/mitmproxy

    warning: no previously-included files found matching 'README.mkd'
    warning: no files found matching '*' under directory 'doc'
    warning: no files found matching '*' under directory 'libmproxy/resources'
    warning: no previously-included files matching '*.swo' found under directory 'test'
    warning: no previously-included files matching '*.swp' found under directory 'test'
    warning: no previously-included files matching '*.pyc' found under directory 'test'
  Requirement already satisfied (use --upgrade to upgrade): mitmproxy==0.11 from file:///Users/local/git/mitmproxy in /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages
Requirement already satisfied (use --upgrade to upgrade): netlib>=0.11 in /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages (from mitmproxy==0.11)
Requirement already satisfied (use --upgrade to upgrade): urwid>=1.1 in /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages (from mitmproxy==0.11)
Requirement already satisfied (use --upgrade to upgrade): pyasn1>0.1.2 in /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages (from mitmproxy==0.11)
Requirement already satisfied (use --upgrade to upgrade): requests>=1.2.2 in /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages (from mitmproxy==0.11)
Requirement already satisfied (use --upgrade to upgrade): pyopenssl>=0.14 in /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages (from mitmproxy==0.11)
Requirement already satisfied (use --upgrade to upgrade): Pillow>=2.3.0 in /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages (from mitmproxy==0.11)
Requirement already satisfied (use --upgrade to upgrade): lxml in /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages (from mitmproxy==0.11)
Requirement already satisfied (use --upgrade to upgrade): flask in /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages (from mitmproxy==0.11)
Requirement already satisfied (use --upgrade to upgrade): cryptography>=0.2.1 in /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages (from pyopenssl>=0.14->mitmproxy==0.11)
Requirement already satisfied (use --upgrade to upgrade): six>=1.5.2 in /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages (from pyopenssl>=0.14->mitmproxy==0.11)
Requirement already satisfied (use --upgrade to upgrade): Werkzeug>=0.7 in /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages (from flask->mitmproxy==0.11)
Requirement already satisfied (use --upgrade to upgrade): Jinja2>=2.4 in /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages (from flask->mitmproxy==0.11)
Requirement already satisfied (use --upgrade to upgrade): itsdangerous>=0.21 in /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages (from flask->mitmproxy==0.11)
Requirement already satisfied (use --upgrade to upgrade): cffi>=0.8 in /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages (from cryptography>=0.2.1->pyopenssl>=0.14->mitmproxy==0.11)
Requirement already satisfied (use --upgrade to upgrade): markupsafe in /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages (from Jinja2>=2.4->flask->mitmproxy==0.11)
Requirement already satisfied (use --upgrade to upgrade): pycparser in /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages (from cffi>=0.8->cryptography>=0.2.1->pyopenssl>=0.14->mitmproxy==0.11)
Cleaning up...

Suggestions appreciated!
Thanks a lot,
Steven

[mhils]

HI @stevencheung,

sorry for the late reply. Your case looks fairly weird - here's what I would try:

• Most dumb approach: pip uninstall pyOpenSSL cryptography followed by pip install pyOpenSSL cryptography.
• Take a look at /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages - If you have 0.14 installed, your SSL.py should be similar to https://github.com/pyca/pyopenssl/blob/f73a3cb6eea1cb122a1621926994cc8124f2c872/OpenSSL/SSL.py#L12 (including the import call of _lib).
• Last resort: Search your machine for pyopenssl folders - does anything turn up?

Cheers,
Max

[stevencheung]

Thanks a lot for your suggestions, Max!

Unfortunately, I haven't got mitmproxy to work.

I've tried uninstalling and reinstalling pyOpenSSL and cryptography.
BTW, I forgot to mention that I couldn't install cryptography using "pip install cryptography".
Instead, I used the workaround suggested by Chris Kief:
http://chriskief.com/2014/03/25/installing-cryptography-via-pip-with-macports-or-homebrew/

i.e., sudo env ARCHFLAGS="-arch x86_64" LDFLAGS="-L/opt/local/lib" CFLAGS="-I/opt/local/include" pip install cryptography

pip reported that both pyOpenSSL 0.14 and cryptography 0.5.3 have been installed:
% pip freeze | grep -i openssl
pyOpenSSL==0.14
% pip freeze | grep -i cryptography
cryptography==0.5.3

Also, I've confirmed that OpenSSL is installed in
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/

% pwd
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages

% ls -tl |head
total 4832
drwxr-xr-x 35 root wheel 1190 Aug 18 10:37 libmproxy/
drwxr-xr-x 8 root wheel 272 Aug 18 10:37 mitmproxy-0.11-py2.7.egg-info/
drwxr-xr-x 8 root wheel 272 Aug 18 10:03 pyOpenSSL-0.14-py2.7.egg-info/
drwxr-xr-x 20 root wheel 680 Aug 18 10:03 OpenSSL/
drwxr-xr-x 17 root wheel 578 Aug 18 10:02 cryptography/
drwxr-xr-x 9 root wheel 306 Aug 18 10:02 cryptography-0.5.3-py2.7.egg-info/

Moreover,
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/OpenSSL/SSL.py is the same as that shown at
https://github.com/pyca/pyopenssl/blob/f73a3cb6eea1cb122a1621926994cc8124f2c872/OpenSSL/SSL.py#L12

I've got a different error this time:
% which mitmproxy
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/bin/mitmproxy

% mitmproxy
Traceback (most recent call last):
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/bin/mitmproxy", line 3, in
from libmproxy import proxy, console, cmdline
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/libmproxy/console/init.py", line 5, in
from .. import controller, utils, flow, script, proxy
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/libmproxy/flow.py", line 9, in
import flask
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/flask/init.py", line 17, in
from werkzeug.exceptions import abort
ImportError: No module named werkzeug.exceptions

I'm not sure what "werkzeug" is, but the pip module is already installed.
% pip freeze | grep -i werkzeug
Werkzeug==0.9.6

% pip freeze |grep -i flask
Flask==0.10.1

More suggestions would be appreciated!
Thanks,
Steven

[mhils]

I have no idea how you messed up your Python environment like that 😉
It looks like reinstalling pyOpenSSL solved your problem with this one, so I would try reinstalling werkzeug as well. python -c "import werkzeug.exceptions" should work without flaws.

Cheers,
Max

[stevencheung]

Thanks for your help, Max!

After reinstalling werkzeug, the import error for werkzeug.exceptions has gone.
However, it looks like I'm back to the pyopenssl problem.
I've uninstalled and reinstalled pyopenssl, but still have issues with mitmproxy.

% pip freeze | grep -i mitmproxy
mitmproxy==0.11

% pip freeze | grep -i netlib
netlib==0.11

% pip freeze | grep -i pyopenssl
pyOpenSSL==0.14

% mitmproxy
Traceback (most recent call last):
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/bin/mitmproxy", line 43, in
config = process_proxy_options(parser, options)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/libmproxy/proxy/config.py", line 127, in process_proxy_options
certforward=options.certforward,
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/libmproxy/proxy/config.py", line 48, in init
self.certstore = certutils.CertStore.from_store(self.confdir, CONF_BASENAME)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/netlib/certutils.py", line 155, in from_store
dh = klass.load_dhparam(dhp)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/netlib/certutils.py", line 135, in load_dhparam
bio = OpenSSL.SSL._lib.BIO_new_file(path, b"r")
AttributeError: 'module' object has no attribute '_lib'

At this point, I'm giving up installing mitmproxy on my mac,
and will use a Linux machine for running mitmproxy ...

Thanks again,
Steven

[mhils]

Sorry that it didn't work out - haven't seen something like this before. If
you're just interested in running mitmproxy, you should check out our
binary distribution ;-)

Let me know if you have any other issues or feedback, I'm happy to help.

Cheers,
Max

Am Dienstag, 19. August 2014 schrieb stevencheung :

Thanks for your help, Max!

After reinstalling werkzeug, the import error for werkzeug.exceptions has
gone.
However, it looks like I'm back to the pyopenssl problem.
I've uninstalled and reinstalled pyopenssl, but still have issues with
mitmproxy.

% pip freeze | grep -i mitmproxy
mitmproxy==0.11

% pip freeze | grep -i netlib
netlib==0.11

% pip freeze | grep -i pyopenssl
pyOpenSSL==0.14

% mitmproxy
Traceback (most recent call last):
File
"/opt/local/Library/Frameworks/Python.framework/Versions/2.7/bin/mitmproxy",
line 43, in
config = process_proxy_options(parser, options)
File
"/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/libmproxy/proxy/config.py",
line 127, in process_proxy_options
certforward=options.certforward,
File
"/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/libmproxy/proxy/config.py",
line 48, in init
self.certstore = certutils.CertStore.from_store(self.confdir,
CONF_BASENAME)
File
"/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/netlib/certutils.py",
line 155, in from_store
dh = klass.load_dhparam(dhp)
File
"/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/netlib/certutils.py",
line 135, in load_dhparam
bio = OpenSSL.SSL._lib.BIO_new_file(path, b"r")
AttributeError: 'module' object has no attribute '_lib'

At this point, I'm giving up installing mitmproxy on my mac,
and will use a Linux machine for running mitmproxy ...

Thanks again,
Steven

—
Reply to this email directly or view it on GitHub
#281 (comment).