embed and close button

[gsaurabhr]

It would be really nice to have an 'embed and close' button in the toolbar, which closes the figure canvas, removes the widget and embeds the corresponding png into the notebook. That way, the figures will also persist when the notebook is closed and opened again etc.

[efiring]

If I understand correctly, something like this is not just nice to have, it is essential if the widget is going to be an acceptable replacement for the notebook backend. Without it, open figures pile up, holding memory, and eventually triggering the matplotlib "many open figures" warning.
It also seems to be essential if one is to be able to save a notebook as html with the figures embedded, without having to switch to the inline backend and rerun it.
It would be nice to know whether this is even on the radar, or whether I am misunderstanding something.

[thriveth]

I agree, I sometimes work in with figures of quite large datasets (images) and in the Notebook backend I can close and embed a static picture instead of the widget. It is not just nice for export and later viewing, it also helps freeing up memory.

[ianhi]

As discussed in the linked PR none of us have figured out a good way to do this yet. An alternative that would be helpful (for me at least) would be to have an equivalent button/method that created a new cell below the one in which the figure is displayed and then embedded the png of the figure as an attachment to the markdown cell. I think this should be possible through the INotebook interface provided by jupyterlab

I'm pretty sure this is possible in JupyterLab and not so sure about in the notebook. Though given that nbagg and the apparent sunsetting of the notebook maybe it is ok for ipympl to start adding Jlab specific features?

[ianhi]

I think for the jupyterlab the code would look something like this (based on https://github.com/QuantStack/jupyterlab-snippets/blob/master/src/index.ts):

        const current = notebookTracker.currentWidget;
        const notebook = current.content;
        NotebookActions.insertBelow(notebook);
        const activeCell = notebook.activeCell;
        activeCell.model.value.text = content;

except that the final line should make use of https://jupyterlab.readthedocs.io/en/stable/api/classes/cells.markdowncell-1.html#updatecellsourcewithattachment which is what jlab uses internally when you paste an image from the clipboard.

https://github.com/jupyterlab/jupyterlab/blob/7d0d2e45897ede23e5a1acbc095e6c5a05d493fe/packages/cells/src/widget.ts#L1336-L1338

[ianhi]

Wait, actually the fully proper solution of modifying the output area may be possible in jlab at least by using the https://jupyterlab.readthedocs.io/en/stable/api/interfaces/outputarea.ioutputareamodel-1.html api. I think that previous attempts at this have stalled out because python methods like update_display seem fail when used with widgets, so shifting the problem in js/ts realm may be the solution.

Potentially though this is something that could/should be implemented by ipywidgets.

[ianhi]

@martinRenou @tacaswell thoughts on starting to make use of jlab specific apis?

[martinRenou]

I find the solution to update the cell output better than the one adding a new cell.

We would still need to figure out a strategy for that, when should we update the display? Only when the figure is closed? When the Notebook is saved?

[ianhi]

I find the solution to update the cell output better than the one adding a new cell.

Same, the new cell was just the first thought I had as that's what I've been doing manually.

We would still need to figure out a strategy for that, when should we update the display? Only when the figure is closed? When the Notebook is saved?

I think as a first step we could have an embed button that triggers it per view like in your PR and also when the figure is closed all views should embed themselves. Although I'm not exactly sure how that would work if they're incorporated in a larger widget layout :/

Then separately: Whatever code in ipywidgets that renders the message when a view doesn't have a model (the Canvas(... text) should be modified to also check if the view provides a to_image representation and display that if it's availiable.

[martinRenou]

an embed button that triggers it per view like in your PR

I am not sure this would be possible on a per-view basis.

Although I'm not exactly sure how that would work if they're incorporated in a larger widget layout :/

This workflow would not work. But I suppose it's fine.

Then separately: Whatever code in ipywidgets that renders the message when a view doesn't have a model (the Canvas(... text) should be modified to also check if the view provides a to_image representation and display that if it's availiable.

IMO we need to not rely on ipywidgets anymore when the embed button has been pressed. Widget models are not saved in Notebooks by default, but raw images are so we need to replace the widget output entirely with a raw image. My PR was wrong in that sense.

[martinRenou]

What would you think of embedding raw images automatically when saving the Notebook? We would need to hook to the classic Notebook saving and the JupyterLab save event (if there is such an event).

[jasongrout]

There is a jlab save event. That is how we save widget state on save.

[martinRenou]

Thanks a lot Jason. That will be useful. I guess the cleanest thing to do would be to look for the checkbox that says "save widget state" state, if it's checked, we don't do anything and let ipywidgets save the widget state, otherwise we replace the widget with a raw image.

[jasongrout]

Thanks a lot Jason. That will be useful. I guess the cleanest thing to do would be to look for the checkbox that says "save widget state" state, if it's checked, we don't do anything and let ipywidgets save the widget state, otherwise we replace the widget with a raw image.

That sounds reasonable - you either get a live snapshot, or a "dead" image one. Would that require a kernel execution?

I'd like to think about how to make this a more general feature of widgets. We've talked for a long time about having a widget-level "snapshot" or "html" function that would return static html representing the current widget state we could embed as an output in the document.

[jasongrout]

It would also be great to think about the problem of communicating that the output does not necessarily represent the code just above it. Typically, the notebook encourages the belief that the code just above an output will reproduce that output (especially if the notebook is run in order). Dynamic outputs (a feature of Jupyter itself) violate that assumption, and ipywidgets does too (either the user could have interacted with it directly, or a later cell can change the output).

In the context of reproducibility and communication, it would be great to convey to the user somehow that the output may not come directly from the cell above it.

[martinRenou]

Just for heads up, I made an attempt to implement this for JupyterLab: #294

It does not work though, I am trying to overwrite the cells output.data on Notebook save but this seems to be quickly overwritten by ipywidgets.