Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disable IPython History in executing preprocessor #1017

merged 7 commits into from May 23, 2019

Disable IPython history when preprocessing

IPython saves history in an SQLite DB. This can break in unfortunate ways when
running multiple IPython kernels simultaneously.

NBConvert will now default to disabling history saving in IPython kernels.
  • Loading branch information
alexrudy committed May 6, 2019
commit cb2ef94840731f34a748860d984d4dca342d88be
@@ -220,6 +220,20 @@ class ExecutePreprocessor(Preprocessor):

ipython_hist_file = Unicode(

This comment has been minimized.

Copy link

MSeal May 8, 2019


👍 We should have probably had this default a long time ago regardless of parallelism concerns.

help="""Path to file to use for SQLite history database for an IPython kernel.
The specific value `:memory:` (including the colon
at both end but not the back ticks), avoids creating a history file. Otherwise, IPython
will create a history file for each kernel.
When running kernels simultaneously (e.g. via multiprocessing) saving history a single
SQLite file can result in database errors, so using `:memory:` is recommended in non-interactive

kernel_manager_class = Type(
help='The kernel manager class to use.'
@@ -268,6 +282,8 @@ def start_new_kernel(self, **kwargs):
'kernelspec', {}).get('name', 'python')
km = self.kernel_manager_class(kernel_name=self.kernel_name,
if km.ipykernel and self.ipython_hist_file:

This comment has been minimized.

Copy link

mpacer May 13, 2019


Is ipykernel an attribute of a KernelManager? It doesn't seem to be

If not, I'd prefer to know where this is being assigned.

This comment has been minimized.

Copy link

alexrudy May 13, 2019

Author Contributor

It is a property of the Kernel Manager but it isn't documented. I used it to encapsulate the logic about using an ipython specific argument, rather than doing my own check against the kernel name before passing this argument in.

self.extra_arguments += ['--HistoryManager.hist_file={}'.format(self.ipython_hist_file)]
km.start_kernel(extra_arguments=self.extra_arguments, **kwargs)

kc = km.client()
@@ -0,0 +1,46 @@
"cells": [
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"from IPython import get_ipython"
"cell_type": "code",
"execution_count": 2,
"metadata": {
"scrolled": true
"outputs": [],
"source": [
"ip = get_ipython()\n",
"assert ip.history_manager.hist_file == ':memory:'"
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.0"
"nbformat": 4,
"nbformat_minor": 2
@@ -238,6 +238,7 @@ def assert_notebooks_equal(expected, actual):
("Unicode.ipynb", dict(kernel_name="python")),
("UnicodePy3.ipynb", dict(kernel_name="python")),
("update-display-id.ipynb", dict(kernel_name="python")),
("Check History in Memory.ipynb", dict(kernel_name="python")),
def test_run_all_notebooks(input_name, opts):
ProTip! Use n and p to navigate between commits in a pull request.
You can’t perform that action at this time.