Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

ignored numpy import error breaks notebook, hides problem #2323

Closed
stsci-sienkiew opened this Issue · 3 comments

3 participants

@stsci-sienkiew

I have IPython 0.13 installed. On most of my Linux machines, it works. They all run a python environment from the same file server.

One machine is missing a FORTRAN library, which results in an ImportError exception when importing numpy. On this machine, the command "ipython notebook" raises this exception:

Traceback (most recent call last):
  File "/usr/stsci/pyssgdev/Python-2.7.3/bin/ipython", line 7, in <module>
    launch_new_instance()
  File "/home/sienkiew/x/IPython/frontend/terminal/ipapp.py", line 388, in launch_new_instance
    app.initialize()
  File "<string>", line 2, in initialize
  File "/home/sienkiew/x/IPython/config/application.py", line 84, in catch_config_error
    return method(app, *args, **kwargs)
  File "/home/sienkiew/x/IPython/frontend/terminal/ipapp.py", line 313, in initialize
    super(TerminalIPythonApp, self).initialize(argv)
  File "<string>", line 2, in initialize
  File "/home/sienkiew/x/IPython/config/application.py", line 84, in catch_config_error
    return method(app, *args, **kwargs)
  File "/home/sienkiew/x/IPython/core/application.py", line 325, in initialize
    self.parse_command_line(argv)
  File "/home/sienkiew/x/IPython/frontend/terminal/ipapp.py", line 308, in parse_command_line
    return super(TerminalIPythonApp, self).parse_command_line(argv)
  File "<string>", line 2, in parse_command_line
  File "/home/sienkiew/x/IPython/config/application.py", line 84, in catch_config_error
    return method(app, *args, **kwargs)
  File "/home/sienkiew/x/IPython/config/application.py", line 420, in parse_command_line
    return self.initialize_subcommand(subc, subargv)
  File "<string>", line 2, in initialize_subcommand
  File "/home/sienkiew/x/IPython/config/application.py", line 84, in catch_config_error
    return method(app, *args, **kwargs)
  File "/home/sienkiew/x/IPython/config/application.py", line 352, in initialize_subcommand
    subapp = import_item(subapp)
  File "/home/sienkiew/x/IPython/utils/importstring.py", line 40, in import_item
    module = __import__(package,fromlist=[obj])
  File "/home/sienkiew/x/IPython/frontend/html/notebook/notebookapp.py", line 54, in <module>
    from .clustermanager import ClusterManager
  File "/home/sienkiew/x/IPython/frontend/html/notebook/clustermanager.py", line 27, in <module>
    from IPython.parallel.apps.ipclusterapp import IPClusterStart
  File "/home/sienkiew/x/IPython/parallel/__init__.py", line 36, in <module>
    from .client.client import Client
  File "/home/sienkiew/x/IPython/parallel/client/client.py", line 48, in <module>
    from IPython.parallel import error
ImportError: cannot import name error

If I place a diagnostic print at the beginning of IPython/parallel/init.py , I find that it is being imported twice, but in neither case does it execute all the way to the end of init.py.

Once I fix the problem with the numpy import, ipython notebook works correctly.

I've tried to diagnose this further, but I don't have sufficient knowledge of IPython to get any more details.

@minrk
Owner

Can you try again with the 0.13.1 branch?

I cannot reproduce the error with any deliberately introduced ImportErrors in numpy so far, even with 0.13, but we should check to make sure we haven't already fixed whatever it is.

@stsci-sienkiew

I loaded https://github.com/ipython/ipython/tree/0.13.1 in my browser. On that page, I found a URL, so I used "git clone https://github.com/ipython/ipython.git" and then ran "setup.py install" in the ipython directory that resulted.

Since you could not reproduce the problem, I created numpy.py containing only "raise ImportError"; with this, ipython works as expected. So, I removed it and started over.

In my system, "import numpy" fails with ImportError when it can't find the fortran library, but only after importing > 70 packages that are nested within numpy. IPython ignores this error, but that results in a second attempt to import numpy, which fails in a different way - it finds an AttributeError deep within numpy rather than ImportError. The AttributeError escapes the try/except , so I can see it. This is an improvement, but it is an accident of the numpy implementation.

At this point, I think this is a built-in deficiency in the de-facto usage of python. The convention is: To detect if a package is present, try to import it; ImportError means it is not present. As we have seen, that is not a reliable way to detect the presence of a package.

I'm not sure what to suggest for IPython. The command "ipython notebook" failed in a way that hid the fact that the problem is with numpy, but I don't see an obviously clear and correct solution. Maybe imp.find_module() is a better way to test if a package is present, but I have not fully investigated the implications.

At the moment, I think my solution is to provide a "test everything" function to all my users. When they report a problem with something (ipython in this example), I can have them run "test everything" to see if anything else is broken at the same time. (Of course, in this case the user only told me that ipython did not work, not that several other things also did not work.)

b.t.w. If you want to reproduce the problem in your system, I think all you need to do is remove numpy/linalg/lapack_lite.so from the installed numpy.

@ivanov
Owner

This has been inactive for nearly two years, and sounds like part of the problem was caused by importing a broken numpy when running the notebook server, which isn't something IPython does out of the box, and now that the --pylab flag has been deprecated (and removed in master) the issue no longer applies.

@ivanov ivanov closed this
@ivanov ivanov added this to the no action milestone
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.