Issues with pyzmq and ipython on EPD update

[yogeshc]

I had installed EPD python and upgraded it using enpkg --update-all.

It made the following updates: http://pastebin.com/2NmvWHHS

After the update, ipython notebook complains about the pyzmq installation. So based on zeromq/pyzmq#162 , I tried to install pyzmq using the command:
pip install --no-install pyzmq --upgrade and
python setup.py configure --zmq=/usr/local build_ext --inplace

However I still get an error as shown in http://pastebin.com/WKTtALdN.

I have also shown that on running just ipython, it is possible to import zmq, and IPython.zmq. So I am not completely sure if it is IPython/pyzmq issue. If anyone has faced similar issues, I would like to know about it before I go diving in to find some solutions.

[minrk]

PyZMQ issues should only be brought to pyzmq, not IPython. I will close here, and we can continue to dig around in pyzmq.

[yogeshc]

Sounds fair. I reported it here as well only because I was not completely sure, it was entirely in pyzmq domain. Especially because the import statements work without error.

[minrk]

Arg, spoke a bit too soon, before I understood what the issue is. So it looks like it might be a version check bug in IPython. That's weird. Can you try updating IPython (current is 0.13.2, not 0.13.1).

[yogeshc]

I see the same output with 13.2. I have attached the log here:
http://pastebin.com/wCrvx01s

[minrk]

That's interesting, because you would only get that error message if import zmq fails, but you clearly have pyzmq somewhere. So, most likely: either your PATH is mixed up, you have multiple Pythons getting in the way of each other, or you have a zmq directory or zmq.py file in your working directory.

[yogeshc]

But on Ipython console I get this:

# ipython
Python 2.7.3 |EPD 7.3-2 (32-bit)| (default, Apr 11 2012, 18:02:54) 
Type "copyright", "credits" or "license" for more information.

IPython 0.13.2 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: import zmq

In [2]: import IPython.zmq

In [3]: 

[minrk]

Yes, that's why there's something wrong with your path. When you start the notebook, you are not using the same Python or IPython install.

[yogeshc]

# which ipython
/opt/epd-7.3-2-rh5-x86/bin/ipython
# which python
/opt/epd-7.3-2-rh5-x86/bin/python

[yogeshc]

I am sorry, but I am not able to understand how I can use a different ipython when running

# ipython 

and yet another ipython when just adding the argument

# ipython notebook

[minrk]

I'm not sure how it's happening. One thing you might do is edit /opt/epd-7.3-2-rh5-x86/lib/python2.7/site-packages/IPython/zmq/__init__.py, at line 53, changing

raise ImportError("%s requires pyzmq >= %s"%(module, minimum_version))

to just

raise

This will show you the original error that prevents import zmq from succeeding.

[yogeshc]

This is the log after changing it to raise. It is now similar to pyzmq import error before installing pyzmq.

http://pastebin.com/4vh0GQgN

[minrk]

And in a regular Python session: import zmq; print zmq?

[yogeshc]

# python
Enthought Python Distribution -- www.enthought.com
Version: 7.3-2 (32-bit)

Python 2.7.3 |EPD 7.3-2 (32-bit)| (default, Apr 11 2012, 18:02:54) 
[GCC 4.1.2 20080704 (Red Hat 4.1.2-44)] on linux2
Type "credits", "demo" or "enthought" for more information.
>>> import zmq
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/epd-7.3-2-rh5-x86/lib/python2.7/site-packages/zmq/__init__.py", line 26, in <module>
    from zmq.utils import initthreads # initialize threads
ImportError: libzmq.so.1: cannot open shared object file: No such file or directory
>>> print zmq
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'zmq' is not defined
>>> 

[yogeshc]

But now even IPython, shows the same

# ipython
Python 2.7.3 |EPD 7.3-2 (32-bit)| (default, Apr 11 2012, 18:02:54) 
Type "copyright", "credits" or "license" for more information.

IPython 0.13.2 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: import zmq
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-1d4d77cc188d> in <module>()
----> 1 import zmq

/opt/epd-7.3-2-rh5-x86/lib/python2.7/site-packages/zmq/__init__.py in <module>()
     24     del here, libzmq, ctypes, os
     25 
---> 26 from zmq.utils import initthreads # initialize threads
     27 initthreads.init_threads()
     28 

ImportError: libzmq.so.1: cannot open shared object file: No such file or directory

[yogeshc]

I don't understand, I only changed the raise statement

[minrk]

If I recall, you said something about building pyzmq inplace. Were you perhaps running IPython from inside the pyzmq build directory before? It sounds like you don't actually have a working pyzmq install.

[yogeshc]

Yes, that's my bad. I did build it in-place and was running from the directory before.

But if I don't do that, I was getting the error similar to zeromq/pyzmq#162

[yogeshc]

I can run

pip install pyzmq

But that will produce similar results as zeromq/pyzmq#162

[minrk]

After you built it in place, did you install it?

[yogeshc]

No, I just used

# python setup.py configure --zmq=/usr/local build_ext --inplace

[yogeshc]

I used the above statement followed by install and it appears to be working att start, however, when I open ipython notebook, zmq, keeps throwing Bad address errors.
http://pastebin.com/t0e1r6tM

[minrk]

That's extremely weird, your zeromq install seems broken in some way. Let's try to get back to a state that makes sense:

1. completely remove IPython and pyzmq
2. download pyzmq
3. install pyzmq:
   • python setup.py configure --zmq=/usr/local
   • python setup.py build
   • python setup.py install
4. test pyzmq (after leaving the pyzmq build directory)
   • nosetests -v zmq.tests
5. reinstall IPython
6. test plain IPython
   • ipython
   • import zmq; print zmq
7. test the notebook
   • ipython notebook

[yogeshc]

Thanks for those steps. It did resolve the issue, however, I am still unclear as to what exactly was wrong in the first place. I am particularly concerned that just doing and enpkg --update breaks up things and would like to know if such steps have to be taken for every pyzmq/ipython related update in future. If that is the case then I can hack-up a simple bash script that could be used for future updates.

For the record, here is the output log for all steps:

1. completely remove IPython and pyzmq
   http://pastebin.com/QCPJeHTD
2. download pyzmq
   http://pastebin.com/Ak4aJ4z6
3. install pyzmq:
   python setup.py configure --zmq=/usr/local
   python setup.py build
   python setup.py install
   http://pastebin.com/pHiQArct
4. test pyzmq (after leaving the pyzmq build directory)
   nosetests -v zmq.tests
   http://pastebin.com/TQCw7zkG
5. reinstall IPython
   http://pastebin.com/Bczrwzzr
6. test plain IPython
   ipython
   import zmq; print zmq
   http://pastebin.com/B4iDNBj1
7. test the notebook
   ipython notebook
   http://pastebin.com/2SqPS09C

[takluyver]

You'll probably have to ask Enthought about their update systems. I don't think anything we do should cause that problem.

[minrk]

Yes, there's definitely nothing to do with IPython here at all. What happened was a strange, incomplete install of pyzmq, either linked against the wrong libzmq or something else. But @takluyver is right, Enthought is the place to ask.