An interface to communicate with Jupyter kernels in Emacs.
Table of Contents
- What does this package do?
- How do I install this package?
- How does this package compare to other similar packages?
- How do I use the built-in frontends?
- Naming conventions
- Callbacks and hooks
- Waiting for messages
- Message property lists
- Modify behavior depending on kernel language
What does this package do?
API for creating Jupyter frontends in Emacs
This package provides an API for communicating with a Jupyter kernel via
sockets (http://github.com/dzop/emacs-zmq). It utilizes Emacs’ object
eieio, to define Jupyter client and kernel manager classes
that can be sub-classed to provide support for any kind of Jupyter frontend in
Emacs. Currently, there is a built-in REPL frontend and
org-mode source block
The base Jupyter client provides default implementations for handling common message replies from a kernel that integrate well with Emacs’ built-in features. Below is a partial list of how kernel messages are integrated with Emacs:
- Sending an inspect request will display the inspect reply in the
*Help*buffer and previous inspect requests can be revisited by calling
help-go-forwardwhile in the
- Making completion requests to a kernel is done through the
- If the kernel asks for input from the user, a prompt is displayed in the minibuffer.
- Search through REPL history using
Support differences between kernel languages
There are many methods that can be extended to take into account differences
between kernel languages. This is achieved by providing a method specializer,
jupyter-lang, that can be added to the
&context section of a method
For example, the Python kernel sends
text/plain data in its inspect replies,
but most of the time the documentation requested is written using
reStructuredText (rST) markup, the officially supported markup language for
documentation strings in Python. In
emacs-jupyter all insertion of messages
into a buffer is handled by the
jupyter-insert method. This method is
extended to properly highlight rST when an inspect reply message is being
inserted and the message is from a Python kernel.
See the files
jupyter-julia.el for how these
languages are integrated into the
emacs-jupyter framework. Note, you don’t
need to require these support files directly since they will be loaded
automatically when needed.
How do I install this package?
NOTE: Your Emacs needs to have been built with module support for this
package to work since it relies on the
emacs-zmq package. See the README of
that package for more information.
The recommended way to install this package is through the built-in package manager in Emacs.
Ensure MELPA is in your
(add-to-list 'package-archives '("melpa" . "https://melpa.org/packages/"))
Ensure the latest versions of MELPA packages are available
M-x package-refresh-contents RET
M-x package-install RET jupyter RET
Building a package archive using cask
One way to install this package is to build a package archive using
(https://github.com/cask/cask) to build a local Emacs package file. To do this,
clone the repository, enter its directory, and run the following at the command
This creates a file
dist/jupyter-0.6.0.tar containing the package archive. To
- Start your Emacs normally
- Ensure MELPA is in your
M-x package-install-file ~/path/to/jupyter/dist/jupyter-0.6.0.tar
For a manual installation you can add the repository directory to your
load-path and ensure the following dependencies are installed:
- markdown-mode (optional)
- company-mode (optional)
(add-to-list 'load-path "~/path/to/jupyter") (require 'jupyter)
Building the widget support (EXPERIMENTAL)
There is also support for interacting with Jupyter widgets through an external browser. If a widget is to be displayed, an external browser is opened first to display the widget. In this case, Emacs acts as a relay for passing messages between the kernel and the external browser.
If you would like to try out this limited support, you will need to have
to run the following commands from the root project directory:
How does this package compare to other similar packages?
There are two popular packages that implement similar functionality to this one
- Interacts with a Jupyter kernel via
- Interacts with a Jupyter kernel via
- ob-ipython (scimax)
- Scimax is an Emacs starterkit for scientists and engineers that extends
- Scimax is an Emacs starterkit for scientists and engineers that extends
- emacs-ipython-notebook (ein)
- A Jupyter notebook interface in Emacs.
emacs-jupyter extends the features of
ob-ipython by integrating more with
org-mode and providing a better REPL interface to the kernel. For example,
ob-ipython currently does not provide a function for
ob-ipython also starts a new process for every
request since it relies on calling a Python script to send and receive messages
emacs-jupyter directly uses
zmq sockets via
communication and only starts a process on every new client connection. This
difference in how messages are passed between Emacs and a kernel is notable
when making completion requests.
ob-ipython will incur the overhead of
starting up a new process and new sockets on every completion request which
can potentially be every keystroke if you type slow enough.
scimax version of
ob-ipython offers some useful extensions to the
ob-ipython experience such as custom keybindings when inside an
org-mode source block, selective display of mimetypes, and jumping to source
block error locations. Many of these features have also been implemented in
ein is more of a full featured solution for a Jupyter notebook interface in
Emacs. The goals of
ein are different.
ein aims to be a
frontend to the Jupyter notebook server API
(https://github.com/jupyter/jupyter/wiki/Jupyter-Notebook-Server-API) which is
an extra layer between the user and a kernel
In addition to being notebook client,
ein offers many more powerful features
for Python kernels.
emacs-jupyter, on the other hand, offers an API that
implements the Jupyter messaging protocol for communication with a kernel via
zmq sockets. The API tries to integrate the interaction between the user and
a kernel with built-in Emacs features. The REPL support and
integration are examples of how the API can be used. In the future, it would be
nice to add some kind of notebook interface in
emacs-jupyter or at least an
efficient conversion process between notebook files and
How do I use the built-in frontends?
To start a new kernel on the
localhost and connect a REPL client to it
M-x jupyter-run-repl. Alternatively you can connect to an existing
kernel by supplying the kernel’s connection file using
The REPL supports most of the rich output that a kernel may send to a client.
If the kernel requests a widget to be displayed, a browser is opened that
displays the widget. If the kernel sends image data, the image will be
displayed in the REPL buffer. If LaTeX is sent, it will be compiled (using
org-mode) and displayed.
Rich kernel output
A Jupyter kernel provides many representations of results that may be used by the frontend, in this case Emacs. Luckily, Emacs provides good support for most of the available representations.
The supported mimetypes along with their dependencies are shown below in order of priority if multiple representations are returned. Note, if a dependency is not available in your Emacs, a mimetype with a lower priority will be used to display output.
|Emacs built with libxml2|
|Emacs built with librsvg2|
To send an inspect request to the kernel, press
M-i when the cursor is at the
location of the code you would like to inspect.
Completion is implemented through the
completion-at-point interface. In
addition to completing symbols in the REPL buffer, completion also works in
buffers associated with a REPL. For
org-mode users, there is even completion
org-mode buffer when editing the contents of a Jupyter source code
You can navigate through the REPL history using
You can also search through the history using
isearch. To search through
history, use the standard
C-s to search forward
through history and
C-s C-r to search backward.
Associating other buffers with a REPL
After starting a REPL, it is possible to associate the REPL with other buffers
if they pass certain criteria. Currently, the buffer must have the
that corresponds to the REPL’s kernel language. To associate a buffer with a
REPL you can run the command
jupyter-repl-associate-buffer will ask you for the REPL you would like to
associate with the
current-buffer and enable the minor mode
jupyter-repl-interaction-mode. This minor mode populates the following
keybindings for interacting with the REPL:
If code sent for evaluation causes a file to be opened via
opened file is associated with the corresponding REPL client if possible. This
behavior is most useful, for example, when using the
edit function in IJulia.
server-mode in Emacs you should have something like the following
in your Emacs configuration before starting any kernels.
(server-mode 1) (setenv "EDITOR" "emacsclient")
Note this probably wont work properly when there are multiple competing clients
sending requests to their underlying kernels that want to open files. Or if the
underlying kernel takes longer than
jupyter-long-timeout seconds to open a
jupyter-server-mode-set-client for more details.
A global minor mode that will persist a kernel connection to a buffer about to
be displayed if the current buffer is in
the buffer being switched to has the same
major-mode. This mode is
automatically enabled whenever
Set the maximum number of lines before the REPL buffer is truncated.
If non-nil, allow inserting a newline in a REPL cell whenever the kernel is
busy. Normally this isn’t allowed since the REPL relies on the kernel
responding to messages when
RET is pressed, but a kernel does not respond to
messages when it is busy.
If non-nil, when evaluating code using the
M-x jupyter-eval-line-or-region, copy the evaluated code as a REPL input
cell and display any output generated in the REPL. When this variable is nil,
copying to the REPL does not occur and output/results are inserted in pop-up
buffers or added to the
*Messages* buffer according to
There is also support for Jupyter widgets integrated into the REPL. If any of the results returned by a kernel have a widget representation, a browser is opened and the widget is displayed in the browser. There is only one browser per client.
This feature is currently considered experimental and has only been tested for simple uses of widgets. See =jupyter-widget-client=.
org-mode source blocks
For users of
org-mode, integration with
org-babel is provided through the
ob-jupyter library. To enable Jupyter support for source code blocks, add
(org-babel-do-load-languages 'org-babel-load-languages '((emacs-lisp . t) (julia . t) (python . t) (jupyter . t)))
jupyter should be added as the last element when loading languages
since it depends on the values of variables such as
ob-jupyter has been loaded, new source
code blocks with names of the form
jupyter-LANG will be available.
LANG can be
any one of the kernel languages found on your system. See
Every Jupyter source code block requires that the
:session parameter be
specified since all interaction with a kernel is through a REPL. For example,
to interact with a
python kernel you would create a new source block like so
#+BEGIN_SRC jupyter-python :session py x = 'foo' y = 'bar' x + ' ' + y #+END_SRC
By default, source blocks are executed synchronously. To execute a source block
asynchronously set the
:async parameter to
#+BEGIN_SRC jupyter-python :session py :async yes x = 'foo' y = 'bar' x + ' ' + y #+END_SRC
Since a particular language may have multiple kernels available, the default
kernel used will be the first one found by
the language. To change the kernel, set the
#+BEGIN_SRC jupyter-python :session py :async yes :kernel python2 x = 'foo' y = 'bar' x + ' ' + y #+END_SRC
Note, the same session name can be used for different values of
the underlying REPL buffer’s name is based on both
Any of the defaults for a language can be changed by setting
org-babel-default-header-args:jupyter-LANG to an appropriate value. For example
to change the defaults for the
julia kernel, you can set
org-babel-default-header-args:jupyter-julia to something like
(setq org-babel-default-header-args:jupyter-julia '((:async . "yes") (:session . "jl") (:kernel . "julia-1.0")))
If you use the
ob-async package, make sure you add the Jupyter source block
languages to ob-async-no-async-languages-alist so that
emacs-jupyter when the
:async header argument is specified. For
example you can put the following in your configuration:
(setq ob-async-no-async-languages-alist '("jupyter-python" "jupyter-julia"))
Overriding built-in src-block languages
You may find having to specify the names of Jupyter source blocks using
jupyter-LANG a bit verbose and want to have the built-in support for
source blocks overridden to use the machinery of
jupyter-LANG source blocks.
This can be done by calling the function
For example, to override the behavior of
python source blocks so that they
jupyter-python source blocks, you can add the following in your
initialization (after calling
After calling the above function, all
python source blocks are effectively
jupyter-python source blocks and the variable
org-babel-default-header-args:python will be set to the value of
org-babel-default-header-args:python will not be an alias of
org-babel-default-header-args:jupyter-python, the value of the former is
merely set to the value of the latter after calling
If you decide you want to go back to the original behavior or
blocks, you can restore the overridden functions by calling
Rich kernel output
org-mode a code block returns scalar data (plain text, numbers, lists,
tables, …), an image file name, or code from another language. All of this
information must be specified in the code block’s header arguments, but all of
this information is already provided in the messages passed between a Jupyter
kernel and its frontends.
When a kernel provides representations of results other than plain text, those richer representations have priority. For example if the kernel returns LaTeX code, the results are wrapped in a LaTeX source block. Similarly for HTML and markdown. If an image is returned, the image is automatically saved to file and a link to the file will be the result of the code block.
Below are the supported mimetypes ordered by priority
- image/svg+xml, image/jpeg, image/png
Since it is possible to determine how a result should be represented in
org-mode via its MIME type, only a few header arguments are supported.
A note on using the
:results header argument
Results are inserted in the
org-mode buffer in such a way that most header
arguments that control how results should be inserted don’t need to specified.
There are some cases where this behavior is not wanted and which can be
controlled by setting the
:results header argument.
- Insert unwrapped LaTeX
- Normally LaTeX results are wrapped in a
BEGIN_EXPORTblock, in order to insert LaTeX unwrapped, specify
- Suppress table creation
- Whenever a result can be converted into an
org-modetable, e.g. when it look like
[1, 2 , 3], it is automatically converted into a table. To suppress this behavior you can specify
Fixing the file name of images with the
Whenever an image result is returned, a random image file name is generated and
the image is written into
org-babel-jupyter-resourse-directory. In order to
specify your own file name for the image, you can give an appropriate value to
:file header argument.
Changing the mime-type priority with the
The priority of mimetypes used to display results can be overwritten using the
:display option. If instead of displaying HTML results we’d wish to display
plain text, the argument
:display text/plain text/html would prioritize plain
text results over html ones. The following example displays plain text instead
#+BEGIN_SRC jupyter-python :session py :display plain import pandas as pd data = [[1, 2], [3, 4]] pd.DataFrame(data, columns=["Foo", "Bar"]) #+END_SRC
Image output without the
:file header argument
For images sent by the kernel, if no
:file parameter is provided to the code
block, a file name is automatically generated based on the image data and the
image is written to file in
org-babel-jupyter-resource-directory. This is
great for quickly generating throw-away plots while you are working on your
code. Once you are happy with your results you can specify the
parameter to fix the file name.
This variable is similar to
org-preview-latex-image-directory but solely for
any files created when Jupyter code blocks are run, e.g. automatically
generated image file names.
Deletion of generated image files
Whenever you run a code block multiple times and replace its results, before
the results are replaced, any generated files will be deleted to reduce the
Convert rich kernel output with the
:pandoc header argument
By default html, markdown, and latex results are wrapped in a
block. If the header argument
:pandoc t is set, they are instead
converted to org-mode format with pandoc. You can control which outputs get
converted with the custom variable
Editing the contents of a code block
When editing a Jupyter code block’s contents, i.e. by pressing
C-c '= when at
a code block, =jupyter-repl-interaction-mode is automatically enabled in the
edit buffer and the buffer will be associated with the REPL session of the code
You may also bind the command
org-babel-jupyter-scratch-buffer to an
appropriate key in
org-mode to display a scratch buffer in the code block’s
major-mode and connected to the code block’s session.
Connecting to an existing kernel
To connect to an existing kernel, pass the kernel’s connection file as the
value of the
:session parameter. The name of the file must have a
suffix for this to work.
If the connection file is a remote file name, i.e. has a prefix like
/method:host:, the kernel’s ports are assumed to live on
attempting to connect to the kernel,
ssh tunnels for the connection are
created. So if you had a remote kernel on a host named
ec2 whose connection
/run/user/1000/jupyter/kernel-julia-0.6.json on that host, you could
#+BEGIN_SRC jupyter-julia :session /ssh:ec2:/run/user/1000/jupyter/kernel-julia-0.6.json ... #+END_SRC
Note, the kernel on the remote host needs to have the ZMQ socket ports exposed. This means that starting a kernel using
jupyter notebook --no-browser
currently doesn’t work since the notebook server does not allow communication with a kernel using ZMQ sockets. You will have to use the connection file created from using something like
jupyter kernel --kernel=python
Password handling for remote connections
Currently there is no password handling, so if your
ssh connection requires a
password I suggest you instead use key-based authentication. Or if you are
connecting to a server using a
pem file add something like
Host ec2 User <user> HostName <host> IdentityFile <identity>.pem
Starting a remote kernel
:session is a remote file name that doesn’t end in
/ssh:ec2:jl, then a kernel on the remote host
/ssh:ec2: is started using
jupyter kernel command on the host. The local part of the session name
serves to distinguish different remote sessions on the same host.
Standard output, displayed data, and code block results
One significant difference between Jupyter code blocks and regular
code blocks is that the underlying Jupyter kernel can request that the client
display extra data in addition to output or the result of a code block. See
To account for this, Jupyter code blocks do not go through the normal
org-mode result insertion mechanism (see
downside of this is that, compared to normal code blocks, only a small subset
of the header arguments common to all code blocks are supported. The upside is
that all forms of results produced by a kernel can be inserted into the buffer
similar to a Jupyter notebook.
The implementation of
org-mode code blocks is really meant to handle either
capturing the standard output or the result of a code block. When using
Jupyter code blocks, if the kernel produces output or asks to display extra
information, the results are appended to a
A minor mode that enables completion and custom keybindings when
inside a Jupyter code block. This mode is enabled by default in
buffers, but only has an effect when
point is inside a Jupyter code block.
Custom keybindings inside Jupyter code blocks
You can define new keybindings that are enabled when
point is inside a
Jupyter code block by using the function
bindings are added to
jupyter-org-interaction-mode-map and are only active
jupyter-org-interaction-mode is enabled.
By default the following keybindings from
jupyter-org-interaction-mode is enabled
Customizable variables available for all frontends
If the number of lines of an evaluation result is smaller than this variable,
the function stored in
jupyter-eval-short-result-display-function is used to
display the result. Otherwise the result is displayed in a pop-up buffer.
This variable is mainly used by the
jupyter-eval-* commands such as
Methods that send messages to a kernel are named
<msg-type> is any message type. The message types are identical to
those defined in the Jupyter spec with
_ characters replaced by
characters. So to send an
execute-request you would call
Similarly, methods that are responsible for handling messages received from a
kernel are named
Methods that require a message type as an argument such as
jupyter-add-callback should do so by passing a message type keyword such as
- The base class for Jupyter frontends. Handles all message sending and receiving to/from a Jupyter kernel.
- The base class for starting local kernel processes.
- (EXPERIMENTAL) A subclass of
jupyter-kernel-clientthat adds support for displaying Jupyter widgets in an external browser.
- A subclass of
jupyter-kernel-clientthat implements a REPL. Note, a
jupyter-repl-clientalso has a
jupyter-widget-clientas a parent class.
- A subclass of
jupyter-repl-clientthat adds support for evaluating
org-modesource code blocks and inserting the results in the
Lower level classes
- A general class for asynchronous communication with a
subprocess. The subprocess polls its standard input for “events” from the
parent process. To add a new event to be handled by the subprocess you use
jupyter-ioloop-add-event. The resulting subprocess event handler created using
jupyter-ioloop-add-eventcan potentially send an event back to the parent process. In the parent, events are handled by extending the
- A subclass of
jupyter-ioloopconfigured to start a subprocess that handles messages being passed on Jupyter channels between a kernel and the parent Emacs process. This is what
jupyter-kernel-clientuses to communicate with a kernel.
Communicating with a kernel
Initializing a connection
jupyter-kernel-client to start communicating with a kernel, the
following steps are taken:
- Initialize the connection using
- Start listening on the client’s channels with
When starting a local kernel process, both steps are taken care of in
For remote kernels, you will have to manually supply the connection JSON file
jupyter-initialize-connection and start the kernel channels.
Once a connection is initialized, messages can be sent to the kernel using the
jupyter-send-<msg-type> family of methods, where
<msg-type> is any valid
request message type (see
jupyter-message-types). These methods
asynchronously send a message to the kernel using a subprocess associated with
each client, see help:jupyter-zmq-channel-ioloop, and they each return a
jupyter-request object which encapsulates the information necessary for
handling reply messages associated with the request in the future.
There are two ways to handle the reply messages sent by the kernel: (1)
jupyter-kernel-client and override the
jupyter-handle-<msg-type> family of methods or (2) attach callbacks to the
jupyter-request objects returned by the
Both ways can occur in parallel.
When a message is received,
jupyter-handle-message is called on the client to
kick off the message handling process. Any callbacks associated with the
jupyter-request of the message are evaluated and the appropriate
jupyter-handle-<msg-type> method called.
Note, the default handler methods of
jupyter-kernel-client are no-ops with
the exception of
jupyter-handle-input-request which requests input from the
user and sends it to the kernel.
Represents a client connected to a Jupyter kernel.
Initializing a connection
jupyter-initialize-connection takes a client and a connection file as
arguments and configures the client to communicate with the kernel whose
connection information is contained in the connection file.
After initializing a connection, to begin communicating with a kernel call
(let ((client (jupyter-kernel-client))) (jupyter-initialize-connection client "kernel1234.json") (jupyter-start-channels client))
jupyter-initialize-connection is mainly useful when initializing a remote
connection or connecting to an existing kernel. In order to start a new kernel
(cl-destructuring-bind (manager client) (jupyter-start-new-kernel "python") BODY)
The above code starts a new
python kernel and returns the
jupyter-kernel-manager object used to manage the lifetime of the local kernel
process and the
jupyter-kernel-client connected to the manager’s kernel.
jupyter-start-channels will already have been called on the returned client
To create multiple client’s connected to the kernel of a
To start a client’s channels, use
jupyter-start-channels. To stop a client’s
jupyter-stop-channels. To determine if at least one channel is
You can also start individual channels with
(jupyter-start-channel client :shell)
and stop a channel with
(jupyter-stop-channel client :shell)
Making requests to a kernel
To free up Emacs from having to process messages sent to and received from a kernel, an Emacs subprocess is created for every client. This subprocess is responsible for polling the client’s channels for messages and taking care of message signing, encoding, and decoding. The parent Emacs process is only responsible for supplying the message property lists (the representation used for Jupyter messages in Emacs) when sending a message and will receive the decoded message property list when receiving a message. The exception to this is the heartbeat channel which is implemented using timers in the parent Emacs process.
Note, the message property lists should not be accessed directly. There are helper functions which should be used to access the message fields. See Message property lists.
The lifetime of a request
Sending a request to a kernel is done through one of the
jupyter-send-<msg-type> methods of a
jupyter-kernel-client. The arguments
of the Jupyter message that each method represents are passed as keyword
arguments, the keywords all have names according to the Jupyter messaging spec
_ replaced by
-. These methods construct the message property
lists based on their arguments and pass the constructed message to the
jupyter-send method of a client. The
jupyter-send method then returns a new
jupyter-request representing the sent message.
(jupyter-send-execute-request client :code "1 + 2") ; Returns a `jupyter-request'
When a request is sent, the message ID of the request is added to the client’s
request table which maps message IDs to their corresponding
When a message is received from the kernel the request that generated it is
found in the request table by using the
jupyter-message-parent-id of the
message. The slots of the
jupyter-request are updated, any callbacks
associated with the
jupyter-request are run for the message, and the message
is dispatched to the appropriate channel handler method of the client (one of
A request is considered complete and is dropped from the request table once a
status: idle message has been received for the request and it is not the most
recently made request.
When one of the send methods are called, a
jupyter-request object is
instantiated by a call to
jupyter-generate-request and the instantiated
request is returned by the send method so that the caller can attach their
callbacks as described above.
Most likely, subclasses would want to attach extra information to a request.
For example, an
org-mode client that sends an
:execute-request based on the
contents of a source code block might want to keep track of the code block’s
buffer position so that it can insert the results at the right location when
they are ready.
This is the purpose of the
jupyter-generate-request method. If a
jupyter-request object is not general enough for some purpose, a subclass of
jupyter-kernel-client can define a new request object, ensuring that the slots
jupyter-request are included, and return the new type of request when
jupyter-generate-request is called for a message.
For example, below is the definition of the
jupyter-org-request type for
handling requests made in an
(cl-defstruct (jupyter-org-request (:include jupyter-request)) result-type block-params results silent id-cleared-p marker async)
And the context specializers used are
(cl-defmethod jupyter-generate-request ((client jupyter-org-client) msg &context (major-mode org-mode)) ...) ; Return a `jupyter-org-request'
Notice that the
major-mode context allows for
to be used by
jupyter-generate-request when the request is generated in
org-mode buffers and to use the less specialized
jupyter-request in other
When a request is completed, i.e. when the kernel sends an idle message for a
request, you may want to do some final cleanup of the request. This is the
purpose of the
jupyter-drop-request method, it gets called when an idle
message has been received for a kernel but only when the request is not the
most recently sent request.
Handling received messages
The handler methods of a
jupyter-kernel-client are called whenever the
corresponding message is received from the kernel. They are intended to be
overwritten by subclasses and most of the default implementations do nothing
with the exception of the
:input-reply handler asks for input from the user through the
minibuffer and sends it to the kernel whereas the
default message handlers store the state of open
comms in the client’s
The handler methods have the following signature
(cl-defmethod jupyter-handle-<msg-type> ((client jupyter-kernel-client) req arg1 arg2 ...) BODY)
req will be the
jupyter-request object that generated the message.
arg2, … will be the unwrapped message contents passed to the handler, their
number of arguments and their order are dependent on the message type.
Alternatively you may work with the full message property list by accessing the
jupyter-request-last-message slot of the
See message callbacks for another way of handling received messages.
A note on boolean arguments
For message types that have boolean message fields, the symbol in the variable
jupyter--false represents a false value so when checking the contents of
these arguments it is best to explicitly check for
(if (eq arg1 t) ...)
This is because there are some ambiguities between translating JSON values to
their Emacs Lisp equivalents, since
nil in Emacs is used both as signifying
false or nothing whereas JSON has
null for nothing.
Client local variables
Some variables which are used internally by
jupyter-kernel-client have client
local values. For example the variable
jupyter-include-other-output tells a
jupyter-kernel-client to pass IOPub messages originating from a different
client to their corresponding handlers and defaults to
nil, i.e. do not
handle IOPub messages from other clients. To modify a client local variable you
(jupyter-set client 'jupyter-include-other-output t)
and to retrieve the client local value, use
(jupyter-get client 'jupyter-include-other-output)
These functions just set/get the value of a buffer local variable in a private
buffer of the client. You may work with these buffer local variables directly
by using the
jupyter-with-client-buffer macro, just be sure to use
setq-local if you are setting a new client local variable otherwise you may
change the global value of the variable. Alternatively you can define a
variable as automatically buffer local when set with
(jupyter-with-client-buffer client (message "jupyter-include-other-output: %s" jupyter-include-other-output) (setq-local jupyter-include-other-output (not jupyter-include-other-output)))
The channel hook variables
jupyter-stdin-message-hook are all client
local variables and functions can be added to or removed from them using
jupyter-remove-hook. See Channel hooks.
Manage the lifetime of a kernel on the
To get a list of kernelspecs on your system, as represented in Emacs, use
jupyter-available-kernelspecs which processes the output of the shell command
jupyter kernelspec list
to construct the list of kernelspecs.
supports remote hosts. If the
default-directory points to a remote system,
the returned kernelspecs are those on the remote system.
To find all kernelspecs whose kernels match some regular expression use
jupyter-find-kernelspecs. In case you would like to get the kernelspec for a
specific kernel, use
You may also use
jupyter-completing-read-kernelspec in an
interactive spec to ask the user to select a kernel from
the list of available kernelspecs.
Managing the lifetime of a kernel
Starting a kernel
As was mentioned previously, to start a new kernel on the
create a connected client, use
jupyter-start-new-kernel which takes a kernel
name and returns a
jupyter-kernel-manager which manages the lifetime of the
kernel, and a connected
(cl-destructuring-bind (manager client) (jupyter-start-new-kernel "python") BODY)
Instead of supplying an exact kernel name, you may also supply the prefix of
one. Then the first available kernel that has the same prefix will be started.
Stopping a kernel
To shutdown a kernel, use
jupyter-shutdown-kernel. To check if a kernel is
Interrupting a kernel
To interrupt a kernel, use
Making clients connected to a kernel
Once you have a kernel manager you can make new
jupyter-kernel-client (or a
subclass of one) instances using
This class adds support for interacting with Jupyter widgets using an external
browser for the widget display. In order for this to work properly you will
need to have
simple-httpd and the
websocket packages installed, in
The default implementation of
jupyter-widget-client overrides the following
methods of a
(jupyter-handle-comm-close) (jupyter-handle-comm-open) (jupyter-handle-comm-msg)
Comm messages in Jupyter are a way to allow for custom messages between the kernel and a client. In the case of Jupyter widgets they are used to sync widget state between the kernel and client.
It would be amazing to add custom Jupyter widgets to Emacs using the built
widget library which would work for widgets such as text boxes, buttons, and
other simple widgets, but there doesn’t seem to be a way to support more
The default implementation of
jupyter-kernel-client only keeps track of open
comms through a client’s
comms slot. The
adds the functionality to display and interact with widgets through an external
browser. This works by relaying the comm messages between the browser and the
kernel through a websocket. For this to work, you will also need to have the
websocket Emacs packages available.
This feature is currently experimental, but seems to work well. I was able to interact with an ipyleaflet map without any noticeable delay.
Callbacks and hooks
There are mainly two ways of evaluating code when receiving a message from the
kernel. Either sub-classing
jupyter-kernel-client and overriding the handler
methods or adding message callbacks to the
jupyter-request objects returned
by the send methods. If both methods are used in parallel, the message
callbacks will run before the handler methods.
When working with a subclass of
jupyter-kernel-client, to prevent a subset of
handler methods from firing when a message is received for a request, see
Also provided are message hook variables which are local to each client object
and look like
<channel> can be one of
stdin. These hooks also provide an alternative method of
suppressing client handlers from running based on the received message.
To add callbacks to a request, use
jupyter-add-callback which accepts a
jupyter-request as its first argument and alternating (message type,
callback) pairs as the remaining arguments. The callbacks are registered with
the request object to run whenever a message of the appropriate type is
received. For example, to do something when a client receives a
:kernel-info-reply you would do the following:
(jupyter-add-callback (jupyter-send-kernel-info-request client) :kernel-info-reply (lambda (msg) (let ((info (jupyter-message-content msg))) BODY)))
To print out the results of an execute request:
(jupyter-add-callback (jupyter-send-execute-request client :code "1 + 2") :execute-result (lambda (msg) (message (jupyter-message-data msg :text/plain))))
To add multiple callbacks to a request:
(jupyter-add-callback (jupyter-send-execute-request client :code "1 + 2") :execute-result (lambda (msg) (message (jupyter-message-data msg :text/plain))) :status (lambda (msg) (when (jupyter-message-status-idle-p msg) (message "DONE!"))))
There is also the possibility of running the same handler for different message types:
(jupyter-add-callback (jupyter-send-execute-request client :code "1 + 2") '(:status :execute-result :execute-reply) (lambda (msg) (pcase (jupyter-message-type msg) (:status ...) (:execute-reply ...) (:execute-result ...))))
Hook variables are available for each channel:
jupyter-shell-message-hook. Unless you want
to run a channel hook for every client, use
jupyter-add-hook to add a
function to one of the channel hooks.
jupyter-add-hook only adds to the
client local value of the hook variables.
(jupyter-add-hook client 'jupyter-iopub-message-hook (lambda (msg) (when (jupyter-message-status-idle-p msg) (message "Kernel idle."))))
To remove a client local hook, use
Channel hooks also provide a way of suppressing the handler methods. If any of the channel hooks return a non-nil value, the handler method for that message will be suppressed.
In addition to suppressing handler methods using channel hooks, to prevent a
client from running its handler methods for a particular request you can
jupyter-inhibit-handlers to an appropriate value before the request is
made. For example, to prevent a client from running its stream handler for a
request you would do the following:
(let ((jupyter-inhibit-handlers '(:stream))) (jupyter-send-execute-request client :code "print(\"foo\")\n1 + 2"))
jupyter-inhibit-handlers can be either a list of message types or
latter meaning inhibit handlers for all message types. Alternatively you can
jupyter-request-inhibited-handlers slot of a
object. This slot can take the same values as
Waiting for messages
All message passing between the kernel and Emacs happens asynchronously. So if a code path in Emacs Lisp is dependent on some message already having been received, e.g. an idle message, there needs to be primitives that will block so that there is a guarantee that a particular message has been received before proceeding.
The following functions all wait for different conditions to be met on the
received messages of a request and return the message that caused the function
to stop waiting or
nil if no message was received within a timeout period.
The default timeout is
For example, to wait until an idle message has been received for a request:
(let ((timeout 4)) (jupyter-wait-until-idle (jupyter-send-execute-request client :code "import time\ntime.sleep(3)") timeout))
To wait until a message of a specific type is received for a request:
(jupyter-wait-until-received :execute-reply (jupyter-send-execute-request client :code "[i*10 for i in range(100000)]"))
The most general form of the blocking functions is
takes a message type and a predicate function of a single argument. Whenever a
message is received that matches the message type, the message is passed to the
function to determine if
jupyter-wait-until should return from waiting.
(defun stream-prints-50-p (msg) (let ((text (jupyter-message-get msg :text))) (cl-loop for line in (split-string text "\n") thereis (equal line "50")))) (let ((timeout 2)) (jupyter-wait-until (jupyter-send-execute-request client :code "[print(i) for i in range(100)]") :stream #'stream-prints-50-p timeout))
The above code runs
stream-prints-50-p for every
stream message received
from a kernel (here assumed to be a python kernel) for an execute request that
prints the numbers 0 to 99 and waits until the kernel has printed the number 50
before returning from the
jupyter-wait-until call. If the number 50 is not
printed before the two second timeout,
Otherwise it returns the stream message whose content contains the number 50.
Message property lists
There is really no need to construct or access message property lists directly.
jupyter-send-<msg-type> client methods already handle creating them by
jupyter-message-<msg-type> family of functions. Similarly, when a
message is received from a kernel the message properties are unwrapped and
passed as arguments to the
jupyter-handle-<msg-type> client methods. If
required, the message property list is available in the
jupyter-request-last-message slot of the
jupyter-request passed to the
jupyter-handle-<msg-type> client methods.
On the other hand, message callbacks pass the message property list directly to the callback. In this case, the following functions can be used to access the fields of the property list:
;; Get the `:content' propery of MSG (jupyter-message-content msg) ;; Get the message type (one of the keys in `jupyter-message-types') (jupyter-message-type msg) ;; Get the value of KEY in the MSG contents (jupyter-message-get msg key) ;; Get the value of the MIMETYPE in MSG's :data property ;; MIMETYPE should be one of `:image/png', `:text/plain', ... (jupyter-message-data msg mimetype)
Note that access of the message property lists should only occur through the
jupyter-message-* functions since the main parts of a message such as the
content and header are lazily decoded.
jupyter-with-message-content gives a way to extract and
bind the keys of a
(jupyter-with-message-content msg (status ename) ...) ; status and ename keys of (jupyter-message-content msg) are bound
There is also
jupyter-with-message-data which extracts
and binds the mimetypes of
(jupyter-with-message-data msg ((res text/plain)) ...) ; res is bound to (jupyter-message-data msg :text/plain)
Modify behavior depending on kernel language
Since Jupyter supports many different programming language kernels, each with varying degrees of support in Emacs there needs to be a general way of modifying the behavior of the client to take this into account.
This is achieved using the
&context specializer of
cl-defmethod. There are
currently two specializers in use,
jupyter-lang is a context specializer that matches when the kernel language
jupyter-current-client is equal to the specializer’s argument. For
example, below is the function that gets called in the REPL buffer when the
kernel language is
julia for indenting the current line:
(cl-defmethod jupyter-indent-line (&context (jupyter-lang julia)) (call-interactively #'julia-latexsub-or-indent))
There are many other entry points where methods may be overridden in such a way. Below is the full list of methods that can be overridden in this way
|Insert Jupyter results into the buffer|
|Return code and position for inspect and complete requests|
|Indent the current cell in the REPL buffer|
|Return the completion prefix for the current context|
|Evaluate code when a completion candidate has been selected|
|Evaluate code after a REPL buffer has been initialized|
|Evaluate code when the input cell code changes|
|Follow a markdown link at point|
|Handle a payload sent by the kernel|
|Transform result of execution into an |
|Transform code of a src-block before sending it to the kernel|
In addition to the
jupyter-lang context, there is also the
jupyter-repl-mode context which is identical to the
but does its check against
jupyter-repl-lang-mode if the
jupyter-current-client is a
jupyter-repl-client. This is useful to modify
behavior depending on the
major-mode that is used for a particular language.
For example for
js2-mode is used as the REPL languages
js2-mode does not
jupyter-org-client is a subclass of
jupyter-kernel-client meant to
display the results of a Jupyter code block in an
The main entry point for extending how results are inserted into the
buffer is the method help:jupyter-org-result, which dispatches on the MIME type
of a result returned from a kernel. The MIME type priority is given in
jupyter-org-result can return either an
org-element object or a string. In the former case, the
transformed into its string representation before insertion into the buffer. In
the later case, the string is inserted into the
org-mode buffer as is,
without any further processing.
There are helper functions for generating
org-element objects which have
For a kernel language to extend the behavior of how results are inserted, the
jupyter-lang method specializer can be used. For example, below is how
:text/plain results are modified for Python code blocks
(cl-defmethod jupyter-org-result ((_mime (eql :text/plain)) &context (jupyter-lang python) &rest _ignore) (let ((result (cl-call-next-method))) (cond ((stringp result) (org-babel-python-table-or-string result)) (t result))))
cl-call-next-method calls down to a less specialized method of
jupyter-org-result and if the returned result is still expected to be plain
org-babel-python-table-org-string to convert any results that
look like Python arrays into
org-mode tables before returning its result.
Bind a key that is only available when
point is inside a Jupyter code block.
When the command bound to the key is evaluated,
be bound to the client of the current code block, also the syntax table will be
the same as the underlying kernel language’s (see
These keys only have an effect when
jupyter-org-interaction-mode is enabled.