Prolonged ipython closing prevents other ipython processes from starting. #10117

Open
Skuzzzy opened this Issue Jan 5, 2017 · 6 comments

Projects

None yet

3 participants

@Skuzzzy
Skuzzzy commented Jan 5, 2017

When I use ipython and allocate some large python objects ipython takes longer to shutdown, which makes sense, but I ran into the issue today, where while ipython is shutting down (taking a long time due to my large objects I have allocated), it stops any other instance of ipython coming up.

Typically this isn't an issue but depending on what objects I have allocated, ipython can take 30-60 seconds to shutdown I've found (just off the top of my head, I can get exact numbers if you all would like)

below is a trace of an ipython session that I tried to start as the other was coming down.

-bash-4.1$ ipython
Traceback (most recent call last):
  File "/usr/bin/ipython", line 7, in <module>
    launch_new_instance()
  File "/usr/lib/python2.6/site-packages/IPython/frontend/terminal/ipapp.py", line 388, in launch_new_instance
    app.initialize()
  File "<string>", line 2, in initialize
  File "/usr/lib/python2.6/site-packages/IPython/config/application.py", line 84, in catch_config_error
    return method(app, *args, **kwargs)
  File "/usr/lib/python2.6/site-packages/IPython/frontend/terminal/ipapp.py", line 324, in initialize
    self.init_shell()
  File "/usr/lib/python2.6/site-packages/IPython/frontend/terminal/ipapp.py", line 340, in init_shell
    ipython_dir=self.ipython_dir)
  File "/usr/lib/python2.6/site-packages/IPython/config/configurable.py", line 318, in instance
    inst = cls(*args, **kwargs)
  File "/usr/lib/python2.6/site-packages/IPython/frontend/terminal/interactiveshell.py", line 360, in __init__
    user_module=user_module, custom_exceptions=custom_exceptions
  File "/usr/lib/python2.6/site-packages/IPython/core/interactiveshell.py", line 436, in __init__
    self.init_history()
  File "/usr/lib/python2.6/site-packages/IPython/core/interactiveshell.py", line 1489, in init_history
    self.history_manager = HistoryManager(shell=self, config=self.config)
  File "/usr/lib/python2.6/site-packages/IPython/core/history.py", line 409, in __init__
    self.new_session()
  File "<string>", line 2, in new_session
  File "/usr/lib/python2.6/site-packages/IPython/core/history.py", line 60, in needs_sqlite
    return f(*a,**kw)
  File "/usr/lib/python2.6/site-packages/IPython/core/history.py", line 427, in new_session
    NULL, "") """, (datetime.datetime.now(),))
OperationalError: database is locked

If you suspect this is an IPython bug, please report it at:
    https://github.com/ipython/ipython/issues
or send an email to the mailing list at ipython-dev@scipy.org

You can print a more detailed traceback right now with "%tb", or use "%debug"
to interactively debug it.

Extra-detailed tracebacks for bug-reporting purposes can be enabled via:
    c.Application.verbose_crash=True
@Skuzzzy
Skuzzzy commented Jan 5, 2017

Note to reproduce this, just load large objects (mine were around 12 gig scipy matrices). Exit the session with those loaded, it should hang for some time after you confirm that you would like to exit the session. Then while it is hanging, open another ipython session.

@minrk
Member
minrk commented Jan 5, 2017

@Skuzzzy thanks for the report! It's a bit odd that we're holding on to the database lock for a long time during shutdown, but it seems like this should be fixable. I would guess that the thing to look at is the shutdown process of the HistoryManager, to see if there's a way to ensure that it closes promptly.

@takluyver
Member

Is it possible that you've turned on HistoryManager.db_log_output? It's an unusual setting, but could be relevant.

@Skuzzzy
Skuzzzy commented Jan 10, 2017

@takluyver To my knowledge I am using ipython with the default configurations

@takluyver
Member

I can't get it to hang for long enough to try to reproduce this - even filling up half my memory with random numbers from numpy, it exits almost immediately.

Just to double check that the setting I mentioned hasn't somehow got turned on, can you check:

get_ipython().history_manager.db_log_output

(The default is False)

@takluyver
Member

I just spotted that the traceback shows Python 2.6, and thus you must be on an old version of IPython as well. Is this reproducible with a recent version of IPython (which will need a newer Python version)?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment