sqlite exceptions on exit if I use ipdb in a multi-threaded app

[mgedmin]

1. Start a multi-threaded app (ZTK-based in my case, but I don't think it matters)
2. Insert an ipdb breakpoint in some view
3. Try to render that view
4. Press either ^C or ^D (assuming Linux) on the console at the ipdb prompt

What happens: BdbQuit is raised which terminates the debugging session but then I also get the following spew:

BdbQuit
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/usr/lib/python2.7/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/path/to/my/virtualenv/lib/python2.7/site-packages/IPython/core/interactiveshell.py", line 3115, in atexit_operations
    self.history_manager.end_session()
  File "/path/to/my/virtualenv/lib/python2.7/site-packages/IPython/core/history.py", line 504, in end_session
    self.writeout_cache()
  File "<string>", line 2, in writeout_cache
  File "/path/to/my/virtualenv/lib/python2.7/site-packages/IPython/core/history.py", line 65, in needs_sqlite
    return f(self, *a, **kw)
  File "/path/to/my/virtualenv/lib/python2.7/site-packages/IPython/core/history.py", line 680, in writeout_cache
    self._writeout_input_cache(conn)
  File "/path/to/my/virtualenv/lib/python2.7/site-packages/IPython/core/history.py", line 664, in _writeout_input_cache
    (self.session_number,)+line)
ProgrammingError: SQLite objects created in a thread can only be used in that same thread.The object was created in thread id 47876379322112 and this is thread id 47876201294400
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/usr/lib/python2.7/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/path/to/my/virtualenv/lib/python2.7/site-packages/IPython/core/interactiveshell.py", line 3115, in atexit_operations
    self.history_manager.end_session()
  File "/path/to/my/virtualenv/lib/python2.7/site-packages/IPython/core/history.py", line 504, in end_session
    self.writeout_cache()
  File "<string>", line 2, in writeout_cache
  File "/path/to/my/virtualenv/lib/python2.7/site-packages/IPython/core/history.py", line 65, in needs_sqlite
    return f(self, *a, **kw)
  File "/path/to/my/virtualenv/lib/python2.7/site-packages/IPython/core/history.py", line 680, in writeout_cache
    self._writeout_input_cache(conn)
  File "/path/to/my/virtualenv/lib/python2.7/site-packages/IPython/core/history.py", line 664, in _writeout_input_cache
    (self.session_number,)+line)
ProgrammingError: SQLite objects created in a thread can only be used in that same thread.The object was created in thread id 47876379322112 and this is thread id 47876201294400
Error in sys.exitfunc:
---------------------------------------------------------------------------
ProgrammingError                          Traceback (most recent call last)
/usr/lib/python2.7/atexit.pyc in _run_exitfuncs()
     22         func, targs, kargs = _exithandlers.pop()
     23         try:
---> 24             func(*targs, **kargs)
     25         except SystemExit:
     26             exc_info = sys.exc_info()

/path/to/my/virtualenv/lib/python2.7/site-packages/IPython/core/interactiveshell.pyc in atexit_operations(self)
   3113         # this must be *before* the tempfile cleanup, in case of temporary
   3114         # history db
-> 3115         self.history_manager.end_session()
   3116 
   3117         # Cleanup all tempfiles left around

/path/to/my/virtualenv/lib/python2.7/site-packages/IPython/core/history.pyc in end_session(self)
    502     def end_session(self):
    503         """Close the database session, filling in the end time and line count."""
--> 504         self.writeout_cache()
    505         with self.db:
    506             self.db.execute("""UPDATE sessions SET end=?, num_cmds=? WHERE

/path/to/my/virtualenv/lib/python2.7/site-packages/IPython/core/history.pyc in writeout_cache(self, conn)

/path/to/my/virtualenv/lib/python2.7/site-packages/IPython/core/history.pyc in needs_sqlite(f, self, *a, **kw)
     63         return []
     64     else:
---> 65         return f(self, *a, **kw)
     66 
     67 

/path/to/my/virtualenv/lib/python2.7/site-packages/IPython/core/history.pyc in writeout_cache(self, conn)
    678         with self.db_input_cache_lock:
    679             try:
--> 680                 self._writeout_input_cache(conn)
    681             except sqlite3.IntegrityError:
    682                 self.new_session(conn)

/path/to/my/virtualenv/lib/python2.7/site-packages/IPython/core/history.pyc in _writeout_input_cache(self, conn)
    662             for line in self.db_input_cache:
    663                 conn.execute("INSERT INTO history VALUES (?, ?, ?, ?)",
--> 664                                 (self.session_number,)+line)
    665 
    666     def _writeout_output_cache(self, conn):

ProgrammingError: SQLite objects created in a thread can only be used in that same thread.The object was created in thread id 47876379322112 and this is thread id 47876201294400

Maybe it's an IPython bug; I don't know which piece of code is responsible for registering those atexit handlers.

[gbastien]

@gotcha, this should be fixed :-p

I think some users would jump to ipdb if this was not failing... We wil try to fix this at our next sprint with you!

Gauthier

[gotcha]

@mgedmin thanks for reporting this known :-( bug.

I'll try to get to it... Anyone else is welcome !

[gotcha]

Can you try the fix at 19deda0 from disable-ipython-history branch ?

[mgedmin]

Uh, I don't remember what project I was debugging when I discovered this. ZTK with Sqlite? Doesn't ring a bell.

[mafrosis]

This has been happening to me for a couple of years now. I only just bothered to google a solution.

[gotcha]

Since version 0.8.2, ipdb does load IPython configuration, it is now possible to disable history manager through configuration.

[cirosantilli]

Would be good to say in this thread how to disable the history manager.

[gotcha]

@cirosantilli I would point to IPython config and HistoryAccessor.enabled
Is that enough ?