config option for disabling history store

[shabble]

I am using ipython within the django runserver app, which has a feature to auto-reload when it detects changes to project files. Every time this happens, however, it throws a number of exceptions seemingly stemming from ipython trying to save history to an sqlite database from multiple threads.

An excerpt of the error is as follows:

Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/Users/shabble/work/caproj/env/lib/python2.7/site-packages/IPython/core/interactiveshell.py", line 2986, in atexit_operations
    self.history_manager.end_session()
  File "/Users/shabble/work/caproj/env/lib/python2.7/site-packages/IPython/core/history.py", line 432, in end_session
    self.writeout_cache()
  File "<string>", line 2, in writeout_cache
  File "/Users/shabble/work/caproj/env/lib/python2.7/site-packages/IPython/core/history.py", line 60, in needs_sqlite
    return f(*a,**kw)
  File "/Users/shabble/work/caproj/env/lib/python2.7/site-packages/IPython/core/history.py", line 607, in writeout_cache
    self._writeout_input_cache(conn)
  File "/Users/shabble/work/caproj/env/lib/python2.7/site-packages/IPython/core/history.py", line 591, in _writeout_input_cache
    (self.session_number,)+line)
ProgrammingError: SQLite objects created in a thread can only be used in that same thread.The object was created in thread id 4365488128 and this is thread id 140735200143712
Error in sys.exitfunc:
Exception AttributeError: AttributeError("'_DummyThread' object has no attribute '_Thread__block'",) in <module 'threading' from '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.pyc'> ignored
Exception AttributeError: AttributeError("'_DummyThread' object has no attribute '_Thread__block'",) in <module 'threading' from '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.pyc'> ignored
Exception AttributeError: AttributeError("'_DummyThread' object has no attribute '_Thread__block'",) in <module 'threading' from '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.pyc'> ignored
Exception AttributeError: AttributeError("'_DummyThread' object has no attribute '_Thread__block'",) in <module 'threading' from '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.pyc'> ignored
Exception AttributeError: AttributeError("'_DummyThread' object has no attribute '_Thread__block'",) in <module 'threading' from '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.pyc'> ignored
Exception AttributeError: AttributeError("'_DummyThread' object has no attribute '_Thread__block'",) in <module 'threading' from '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.pyc'> ignored
---------------------------------------------------------------------------
ProgrammingError                          Traceback (most recent call last)
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/atexit.pyc in _run_exitfuncs()
     22         func, targs, kargs = _exithandlers.pop()
     23         try:
---> 24             func(*targs, **kargs)
     25         except SystemExit:
     26             exc_info = sys.exc_info()

/Users/shabble/work/caproj/env/lib/python2.7/site-packages/IPython/core/interactiveshell.pyc in atexit_operations(self)
   2984         # this must be *before* the tempfile cleanup, in case of temporary
   2985         # history db
-> 2986         self.history_manager.end_session()
   2987 
   2988         # Cleanup all tempfiles left around

/Users/shabble/work/caproj/env/lib/python2.7/site-packages/IPython/core/history.pyc in end_session(self)
    430     def end_session(self):
    431         """Close the database session, filling in the end time and line count."""
--> 432         self.writeout_cache()
    433         with self.db:
    434             self.db.execute("""UPDATE sessions SET end=?, num_cmds=? WHERE

/Users/shabble/work/caproj/env/lib/python2.7/site-packages/IPython/core/history.pyc in writeout_cache(self, conn)

/Users/shabble/work/caproj/env/lib/python2.7/site-packages/IPython/core/history.pyc in needs_sqlite(f, *a, **kw)
     58         return []
     59     else:
---> 60         return f(*a,**kw)
     61 
     62 

/Users/shabble/work/caproj/env/lib/python2.7/site-packages/IPython/core/history.pyc in writeout_cache(self, conn)
    605         with self.db_input_cache_lock:
    606             try:
--> 607                 self._writeout_input_cache(conn)
    608             except sqlite3.IntegrityError:
    609                 self.new_session(conn)

/Users/shabble/work/caproj/env/lib/python2.7/site-packages/IPython/core/history.pyc in _writeout_input_cache(self, conn)
    589             for line in self.db_input_cache:
    590                 conn.execute("INSERT INTO history VALUES (?, ?, ?, ?)",
--> 591                                 (self.session_number,)+line)
    592 
    593     def _writeout_output_cache(self, conn):

ProgrammingError: SQLite objects created in a thread can only be used in that same thread.The object was created in thread id 4365488128 and this is thread id 140735200143712

My understanding is that this is caused by the sqlite history store used by ipython is getting upset due to the threading model of the django server code. Oddly, the same error occurs even when using a single threaded runserver (./manage.py runserver --nothreading)

On the suggestion of @minrk in #ipython freenode irc channel, I tried:

c.HistoryManager.hist_file = ':memory:', but this does not fix the problem. They suggested I write this up as a ticket with the probable solution being to have an option of disabling the history store altogether. This is probably similar to #909 which handles the case of sqlite being entirely unavailable.

Thanks.

[ustun]

@shabble Oddly, the same error occurs even when using a single threaded runserver Regarding the reason of why this happens: I believe this is because Django actually executes two instances in order to be able to reload: one for running the server, the other for checking the files. When you execute with --noreload, a single instance will run, and no error will be observed on SIGINT.

[AMHZR]

Check this out......

http://bugs.python.org/issue14308

[Altynai]

My problem is just as same as yours. I finally figured it out that it's the "ipdb" package used in my django project that caused this. Try to remove any package that using ipython, hope it can help : )