Add _repr_html_ for inline backend to eliminate need for %matplotlib inline

[tdpetrou]

When launching a jupyter notebook, the environment variable MPLBACKEND is set to inline, but figures will still not be displayed in the output unless %matplotlib inline or import matplotlib.pyplot is run. Both of these commands hook IPython to a function that gets triggered whenever a Figure is the output from a cell. This function is responsible for printing the figure to the screen.

Proposed solution:

A _repr_html_ method exists for figures, but is only implemented for webagg backends. Something like the following could be added to _repr_html_.

if rcParams['backend'] == 'module://ipykernel.pylab.backend_inline':
    from io import BytesIO
    from base64 import b64encode
    png_bytes = BytesIO()
    self.canvas.print_figure(png_bytes, format='png')
    s = png_bytes.getvalue()
    s1 = b64encode(s).decode()
    return f'<img src="data:image/png;base64, {s1}"/>'

This is @tacaswell's idea. I previously created an issue with IPython - ipython/ipython#12190 (comment)

[tacaswell]

Adding a _repr_html_ will not replace the figure life cycle management that inline currently does. Every other backend makes use of the global state in pyplot to allow you to build up figures over multiple cells (like you would in a desktop context), however inline destroys the Figure instance on show. This means you can not build up figures over multiple cells, but it does mean each cell is fully independent and you can get away with only using plt.* (because at the top of the cell there is never a "current figure").

[timhoffm]

What does this mean for this ticket? Would we change the behavior of inline wrt. life cycle management? If not is there a benefit of _repr_html_ compared to the current state?

[tacaswell]

I think that we need someone with good understanding of jupyter and its repr model to answer that.

I could see a path where having a _repr_html_ (or widget etc) that replaces the current show logic that grabs a display area inserts the figure into it. Some of this discussion may be best done in ipympl (as they own the widget version of the backend). Not sure how @SylvainCorlay feels about this, but it may make sense to move the static inline backend there as well (as the key thing really is the jupyter integration)?

[westurner]

Jupyter internals info from https://news.ycombinator.com/item?id=25923123 re: xeus-cling (c++ kernel) and widgets:

Jupyter kernels implement the Jupyter kernel message spec. Introspection, Completion:
https://jupyter-client.readthedocs.io/en/latest/messaging.html#introspection

Debugging (w/ DAP: Debug Adapter Protocol) https://jupyter-client.readthedocs.io/en/latest/messaging.html#debug-request

A display_data Jupyter kernel message includes a data key with a dict value: "The data dict contains key/value pairs, where the keys are MIME types and the values are the raw data of the representation in that format." https://jupyter-client.readthedocs.io/en/latest/messaging.html#display-data

ipython.display: https://github.com/ipython/ipython/blob/master/IPython/display.py

ipython.core.display: https://github.com/ipython/ipython/blob/master/IPython/core/display.py

ipython.lib.display: https://github.com/ipython/ipython/blob/master/IPython/lib/display.py

You can also run Jupyter kernels in a shell with jupyter/jupyter_console:

   pip install jupyter-console jupyter-client
   jupyter kernelspec list
   jupyter console --kernel python3

IPython and Jupyter Notebook and JupyterLab call IPython.display.display(last_expression_in_input_cell)) for every input cell. IPython.display.display(obj) tries a bunch of obj._repr_*_() methods as appropriate for the given output format (e.g. html, latex, markdown, pretty (str)) and tries obj.__repr__() or obj.__str__() last (which is what the CPython REPL always calls to display non-assignment expressions).

---

ipympl https://github.com/matplotlib/ipympl/blob/master/ipympl/backend_nbagg.py is built atop ipywidgets: https://ipywidgets.readthedocs.io/en/latest/examples/Widget%20List.html

[github-actions]

This issue has been marked "inactive" because it has been 365 days since the last comment. If this issue is still present in recent Matplotlib releases, or the feature request is still wanted, please leave a comment and this label will be removed. If there are no updates in another 30 days, this issue will be automatically closed, but you are free to re-open or create a new issue if needed. We value issue reports, and this procedure is meant to help us resurface and prioritize issues that have not been addressed yet, not make them disappear. Thanks for your help!

[timhoffm]

I believe _repr_html_ is not helpful for the inline context. To use _repr_html_, the last return value of a cell would need to be a Figure object. For any reasonable code, that is not automatically the case. Thus, one would have to append an explict fig statement:

fig = plt.figure
[...]  # plot code
fig

but that's basically equivalent to a show() call, and doesn't justify a new way:

fig = plt.figure
[...]  # plot code
plt.show()

Thus, closing as not planned.

[westurner]

This has always been confusing to me;
maybe because of the implicit matplotlib.pyplot.show().
More explicit would be to call display() when yielding output, which is what is implicitly called with the last expression of an input cell:

fig = plt.figure
[...]  # plot code
display(fig)

• This is more like the "print() to print output" style.
• This is more understandable when there are other print() and display() calls in the same cell:

import pandas as pd
import matplotlib.pyplot as plt
def test_example():
    data = list(range(10))
    df = pd.DataFrame(dict(x=data))
    print(1, data)
    display(2, 'two')

    df.plot(legend=True)
    #plt.show()
    ##display(plt)  # -> plt._repr_html_()

    display(4, data)

test_example()

• Where does display(fig) get run in this (pandas+matplotlib) code (when there is no explicit global plt.show())?
  • Why is the plot the last thing in the cell?
• This is probalby in the docs somewhere?
  • When you %matplotlib inline or import matplotlib.pyplot in a Jupyter notebook, it will:

[tacaswell]

@westurner https://github.com/matplotlib/mpl-gui may also be of interest to you as might matplotlib/ipympl#171

[westurner]

• matplotlib.pyplot.ion() (plt.ion()) mentions

  • matplotlib/lib/matplotlib/pyplot.py

    Lines 696 to 699 in 80b08bd

    	ioff : Disable interactive mode.
    	isinteractive : Whether interactive mode is enabled.
    	show : Show all figures (and maybe block).
    	pause : Show all figures, and block for a time.

  • If those can already occur anywhere in a notebook,
    couldn't we also have / why can't we also just,
    in matplotlib.pyplot?:

    def _repr_html_():
        show()

    So that this works, and is explicit about when it is display()'ing or printing output:

    import matplotlib.pyplot as plt
    display(plt)

[westurner]

• make %matplotlib widget behave like inline? ipympl#171

There could also or instead be a per-cell %%matplotlib cell magic.

That's how ipytest works. An example use case with %%ipytest and matplotlib:

%%ipytest
from matplotlib.pyplot import plt

def test_thing1():
    data = range(10)
    df = pd.DataFrame(dict(x=data))

    print(1, data)
    display(2, 'two')

    display(3)
    df.plot(legend=True)
    # plt.show()  
    # display(plt)
    # -> cell_output[n]['text/html'] = plt._repr_html_()

    display(4, data)

    assert bool(plt)

#test_thing1()  # ipytest automatically calls this (after preprocessing lines with assert keywords) because it starts with test_

But then there would need to be multiple IPython cell magics though;
multiple cell magics would be overly verbose and less usable
when display(plt) could call plt._repr_html_() which could just call plt.show()?:

---

What people typically do, FWIU:

%matplotlib inline
# import matplotlib.pyplot as plt

df = pd.DataFrame(dict(x=[1,2,3])
df.plot(legend=True)

• Because of the %matplotlib inline IPython line magic and/or import matplotlib.pyplot as plt python import statement,
  df.plot(legend=True) results in a non-interactive image of matplotlib cell output
  that is saved in the .ipynb file
  so that users can review the cached output without [re-]running the notebook themselves,
  and so that there is a static .png/.jpg for jupyter-nbconvert --to latex or --to pdf.

---

• TODO: run this code; is matplotlib output displayed when the tests are run (and stdout is captured and thus hidden by pytest if the test doesn't fail)

#%pip install -q ipytest
#%%ipytest
def test_thing1():
   df = pd.DataFrame(dict(x=list(range(10)))
   df.plot(legend=True)
   _ = df.plot(legend=True, kind='scatter')
   assert isinstance(_,   bool) 
test_thing1()