SciPy.weave broken in IPython notebook/ qtconsole #3017

Closed
groakat opened this Issue Mar 13, 2013 · 11 comments

Comments

Projects
None yet
6 participants

groakat commented Mar 13, 2013

Problem

Compiling and running of compiled scipy.weave code is not possible with IPython notebook and IPython console. However, it works with standalone, plain IPython.

Minimal Example

http://projects.scipy.org/scipy/browser/trunk/Lib/weave/examples/array3d.py?rev=1558

Error

compiling code

find the traceback here:
http://pastebin.com/FdSadJtV
final error:
AttributeError: 'OutStream' object has no attribute 'fileno'

executing code

For both notebook and qtconsole, the output is printed in the terminal that was used to start them.

tested version

failure in

ipython 0.13.1
scipy '0.11.0'
numpy '1.7.0'

working in

compilation

Apparently it is working with numpy 1.6.0. I did not test it by myself, but that was a response on IRC. It was there also reproduced using numpy 1.7.0. I could compile it in an virtualenv using numpy 1.6 using the notebook. (Still trying to get the qtconsole to run so that I can test that)

output

Even with numpy 1.6 the output is directed to the terminal that was used to start the IPython kernel

@juliantaylor juliantaylor referenced this issue in numpy/numpy Mar 13, 2013

@g2p g2p Assume we can use sys.stdout.fileno() and friends.
The workaround shouldn't be necessary in a recent python shell.
Tested in python, ipython.

In fact, distutils is almost never imported and run manually from an
interactive shell.

This removes the use of `sys.__stdout__`, which broke with distribute
0.6.30 on Python 3.
649c908
Contributor

g2p commented Mar 14, 2013

That awful exec_command should really be ditched, subprocess does the job fine.

Something like:

-        if _with_python and sys.stdout.fileno() == -1:
+        if _with_python and (not hasattr(sys.stdout, 'fileno') or sys.stdout.fileno() == -1):

(untested) should make things work in the notebook.

groakat commented Mar 14, 2013

Sorry, I basically started last week with IPython and dont know much about the scripts it involves. Where about do I have to look for that line? Or is it in numpy?

Contributor

g2p commented Mar 14, 2013

It's in numpy, look at the "juliantaylor referenced this issue" link. Here is the current exec_command.py.

Owner

minrk commented Mar 14, 2013

The overridden iostreams don't have fileno attribute defined, because there is no fd associated with them.
So the question is what is the best way to indicate this:

  1. have no fileno method (current)
  2. fileno() returns None
  3. fileno() returns special 'no FD' integer value (e.g. -1)

Is there an official / common behavior for file-like objects that don't have an FD associated with them? It looks like weave expects returning -1 to be this indicator - is that a general behavior documented anywhere?

Owner

minrk commented Mar 14, 2013

Based on IOBase, it looks like we should be raising an IOError when stream.fileno() is called.

Contributor

g2p commented Mar 14, 2013

I was going to link that. Also you could create a file descriptor (by making a pipe, or maybe using zeromq).

Owner

minrk commented Mar 14, 2013

I don't think we want to create an fd, just express the exceptional state appropriately.

Owner

takluyver commented Mar 15, 2013

io.BytesIO and io.StringIO raise io.UnsupportedOperation when you try to call fileno(). According to the docs, that subclasses IOError and ValueError.

Contributor

g2p commented Mar 15, 2013

As far as the breakage is concerned, there is a fix: numpy/numpy#3128 and an 1.7.x backport: numpy/numpy#3136 . It handles IOError as it should.

certik commented Mar 24, 2013

Both fixes merged into numpy.

minrk closed this in 27e3083 Apr 13, 2013

Similar error appears using the mdp package. Script works in python and ipython, but not ipython notebook:

https://www.google.com/search?q=mdp+package+python&oq=mdp+package+python&ie=UTF-8#q=modular+toolkit+for+data+processing+python

ipython: 0.13.2
python: 2.7.3.final.0
mdp: 3.3
parallel python: 1.6.2-monkey-patched
libsvm: libsvm.so.3
joblib: 0.6.4
sklearn: 0.13.1
numx: scipy 0.12.0
symeig: scipy.linalg.eigh

script

import mdp
mdp.test()
import bimdp
bimdp.test()

traceback

AttributeError Traceback (most recent call last)
in ()
1 import mdp
----> 2 mdp.test()
3 import bimdp
4 bimdp.test()

...

/usr/lib/python2.7/subprocess.pyc in _get_handles(self, stdin, stdout, stderr)
1061 else:
1062 # Assuming file-like object
-> 1063 c2pwrite = stdout.fileno()
1064
1065 if stderr is None:

AttributeError: 'OutStream' object has no attribute 'fileno'

@mattvonrocketstein mattvonrocketstein pushed a commit to mattvonrocketstein/ipython that referenced this issue Nov 3, 2014

@minrk minrk raise UnsupportedOperation on iostream.fileno()
closes #3017
6f07e71
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment