Close a widget but the associate '<div class="output_area">' still exists

[lizequn]

Since 7.0 widgets in notebook are rendered in <div class="output_area">,
but when I close a widget, the associate <div class="output_area"> still exists.

from ipywidgets import Layout, Button, Box
from IPython.display import display
items = [Button(description='example') ]
box1 = Box(children=items)
box2 = Box(children=items)
box3= Box(children=items)
display(box1)
display(box2)
display(box3)
box2.close()

output:

How could I remove this part ?

[jasongrout]

Good point. The easiest way to get rid of it is to put all of your widgets inside of a VBox:

from ipywidgets import Layout, Button, Box, VBox
from IPython.display import display
items = [Button(description='example') ]
box1 = Box(children=items)
box2 = Box(children=items)
box3= Box(children=items)
display(VBox([box1, box2, box3]))
box2.close()

I don't think the notebook has the concept of removing one of the outputs once they've been displayed (without clearing all of the outputs), so it might be tricky to get the output container to actually be removed.

[jasongrout]

The widget is put into the output at

ipywidgets/widgetsnbextension/src/extension.js

Line 120 in 5fefc65

PhosphorWidget.Widget.attach(view.pWidget, node);

. Likely that's where a listener for the view's remove event would remove the output area that was created. The tricky thing is that the output region may have some bookkeeping to do if an output area is removed - that will take looking into the notebook code to see if there is anything that needs to be done to remove an output area.

[lizequn]

Thanks for help
The empty output area holds a small white space, if I continuing create and close widgets, that will result in large white space.
As you described, only display once can definitely help. But when I also want to dynamic add widgets, I create a function like display_or_update which need detect current notebook block has already displayed or not, so how should I get that ?
If I create a python var as flag, it will work for all notebook not just current code block.

from ipywidgets import Layout, Button, Box, VBox
from IPython.display import display
items = [Button(description='example') ]
box1 = Box(children=items)
box2 = Box(children=items)
box3= Box(children=items)
box4 = Box(children=items)
widgets = VBox([box1, box2])

#display widget or add new widget to display
def display_or_update(vbox,widgets):
    widgets.children = widgets.children+(vbox,)
    if not displayed:  <--- how could I get this from context ?
        display(widgets)
display_or_update(box3,widgets)
display_or_update(box4,widgets)
#in another code block
widgets = VBox([box3])
display_or_update(box4,widgets)

[jasongrout]

Why not display the empty widgets vbox at the very start, so you know it's displayed, rather than trying to guess if it is displayed? Or having your update function just update, and then you are responsible for displaying after updating? Or just always displaying the widgets box in your function, regardless?

Edit: to be clear, I'm still not sure what sort of interaction you're wanting, so I'm suggesting lots of alternatives

[jasongrout]

There is also an on_displayed callback that is called when a widget is displayed from python:

def cb(w, **kwargs):
    print(w)
    print(kwargs)
box1.on_displayed(cb)
box1

Does that help?

[Zulko]

Hi there,

Thanks for a great and useful library.

I just wanted to flag that this particular issue is causing important problems in the popular progress bar library tqdm. See tqdm issue here. In a nutshell, the only way to run progress bars with tqdm in the jupyter notebook is by running the tqdm_notebook routine, which relies on ipywidgets. Since the last ipywidget version, every progress bar appears a few pixels lower than the previous. For complex programs, which may display hundreds of progress bars, this is an issue.

I haven't had time to look properly but I am not sure if the tqdm issue could be solved using the VBox trick. As I understand it, this trick implies that you know in advance all widgets you are going to display, while in the case of tqdm_notebook, progress bars will added on the fly by the program.

Thanks in advance

[jasongrout]

@Zulko, thanks for pointing this out. This may take a change in the notebook to handle deleting an output from a cell.

I just checked, and there is the same problem in JupyterLab - closing a widget leaves behind the output container, which has a small height.

[lizequn]

Hi, @jasongrout , sorry for the late response.
As you suggested, I solved this by separating the display and update function.

[jasongrout]

I've also been thinking about this. I think we can safely just set that output area to have display: none, which effectively deletes it from the display, but doesn't mess up the bookkeeping. The only issue then is what to do about the data in that output - do we somehow delete the widget metadata in the output? So I'm still thinking about this.

I'm glad you found something that works in the interim!

[marscher]

I think it is safe to delete the widget data as well in most cases (eg. tqdm progress bars). One could introduce a flag to keep the data, to keep it accessible in case one really needs it.

[jasongrout]

I set this to 7.2, to investigate just hiding the div, without deleting the view (which requires changes in the notebook, it seems). The problem with just hiding the div is that it looks like the widget is gone, but if the notebook is saved and opened again, we'll probably see the widget text there again since the output data is still there.

[jasongrout]

Another workaround: Create a single Output widget that things like those scrollbars are redirected to. When the scrollbars are finished, clear the single Output widget.

[phausamann]

ipywidgets 7.2 seems to have fixed this issue for jupyter notebook, but I still see a similar issue in jupyterlab:

jupyterlab 0.33.4
ipywidgets 7.3.1
jupyterlab-manager 0.36
tqdm 4.24.0

[agoertz-fls]

notebook 5.6.0 breaks this again as it adds a run_this_cell div box to every output area with a padding of 5 so the space grows by 10 pixels for every close operation.

[nanoant]

FYI I identified the source of the problem in JupyterLab -- each removed output is nested in some two additional divs:

<div class="p-Widget p-Panel jp-OutputArea-child">
  <div class="p-Widget jp-OutputPrompt jp-OutputArea-prompt"></div>
  <div class="p-Widget p-Panel jp-OutputArea-output p-mod-hidden"></div>
</div>

While 2nd inner div with p-mod-hidden gets display: none CSS the first jp-OutputPrompt jp-OutputArea-prompt has non-zero padding and border, even when has no content at all, causing the space to grow.
I believe this is a problem at the junction of ipywidgets and JupyterLab. I don't know if there is a way to remove output completely (instead hiding it), so these divs do not remain at all.
My current workaround is an additional CSS:

.p-Widget.jp-OutputPrompt.jp-OutputArea-prompt:empty {
  padding: 0;
  border: 0;
}

This is far from perfect as still the internal DOM grows with empty divs, we just don't see that anymore.
And btw. please re-open this issue as the problem is NOT fixed.

[jasongrout]

Neither the notebook nor JupyterLab really support an output removing itself, so any 'solutions' here in the ipywidgets repo are only hacks to make the display appear gone (but the output isn't really gone). That's why the workaround is so brittle and fragile. In that sense, the issue is closed as "can't fix" because it's out of ipywidget's hands.

To approach getting rid of top-level outputs (including widgets), some options are to put the outputs in an output widget (which can get rid of outputs), or use the display_update mechanism to overwrite previous outputs. Both of these (real) solutions work from the standpoint of the thing displaying the widgets, rather than the internal close method of widgets.

I've opened another issue to document some of these workarounds: #2215.

[jasongrout]

@nanoant - if you have an idea of how to truly do this that I've missed, or want to help documenting some approaches for dealing with this issue (including your visibility workaround), please consider submitting a PR to here, or jupyterlab or notebook as appropriate.

I didn't realize there was a :empty selector - that's cool and looks useful here!

[flying-sheep]

we could also solve this via CSS variables

.p-Widget.jp-OutputPrompt.jp-OutputArea-prompt:empty {
    --jp-border-width: 0;
    --jp-code-padding: 0;
}

I think once @nanoant’s solution is added to the CSS, there’s no real need to get rid of the areas: I doubt they use up too much memory or have a performance impact

[jasongrout]

That css touches only JLab Dom elements, not ipywidgets elements. Would you mind opening this issue in jupyterlab?

[maresb]

Here is an additional annoyance caused by those lingering output areas from closed widgets. GitHub's HTML preview renders the outputs[n]['data']['text/plain'] property of the corresponding output element, even though it is hidden in the notebook view. For example, the output cell from

import ipywidgets as widgets
import time

print('0')
out = widgets.Output()
print('1')
display(out)
with out:
    print('2')
print('3')
time.sleep(1)
out.close()

renders in GitHub as

This happens despite Output() not being visible from the original notebook. I am running JupyterLab 0.35.4, but the same also happens when I open it under /tree/ (Jupyter Notebook).

Moreover, when I use tqdm_notebook, I see HBox(children=(IntProgress(value=1, bar_style='info',... in place of all my closed progress bars when I preview the notebook with GitHub.

[dovahcrow]

Are we gonna solve this issue in ipywidgets or jupyter? As of today (ipywidgets 7.5.0, jupyterlab 1.0.1, this empty div problem is still there.

[stas00]

Neither the notebook nor JupyterLab really support an output removing itself, so any 'solutions' here in the ipywidgets repo are only hacks to make the display appear gone (but the output isn't really gone). [...]

The problem actually isn't just to remove the no longer needed div block, but the problem happens because the output gets broken up into multiple divs, each with padding, so even if that temporary HBox is completely removed the padding problem is still there. So not only it needs to remove itself it also needs to potentially reconnect the split up into multiple divs "stdout" output.

So it needs to be able to reach out to the parent

and tell it to rewrite the whole tree, while (1) removing the unwanted div (2) re-merging the 2 divs that it split when it was created (or practically the next print() was started in a new div..

For example with tqdm that so many people complain about the vertical whitespace. The problem happens because tqdm breaks up the stdout/stderr streams and they each end up in a dedicated div, only one of them is the widget div:

<div class="output_area"><div class="run_this_cell"></div><div class="prompt"></div><div class="output_subarea jupyter-widgets-view" style="display: none;"></div></div>

and before and after divs with the split content:

<div class="output_area"><div class="run_this_cell"></div><div class="prompt"></div><div class="output_subarea output_text output_stream output_stdout"><pre>some text</pre></div></div>

It is because those divs have css padding defined, those add up when one div ends and a new one beings - thus appearing like an extraneous new line.

---

Here is the modified hack for jupyter nb, that removes the vertical whitespace by mucking with css.

So the hack is to override the css not to have the padding for those divs. And voila, there is no vertical space anymore. (This of course may make some IPython.display mixed output appear slightly closer to each other)

Note that the exact naming of the class may change in the future, so this is correct for the notebook server is: 6.0.3

1. Here is the global set and forget solution:

add to ~/.jupyter/custom/custom.css or wherever your jupyter config is the following:

div.output_subarea {
  padding: unset;
}

but it may not be what you want as it'll affect all your other nbs.

You will need to reload your notebook for the new css to be picked up. And if that file didn't exist and you have just created it, you may have to restart the jupyter server too.

2. Here is local to nb solution, where you will have to run the following code:

from IPython.core.display import HTML, display
def rm_out_padding(): display(HTML("<style>div.output_subarea { padding:unset;}</style>"))
rm_out_padding()

Now if you run a bunch of tqdm's as in the example below, you no longer have vertical whitespace injected when it gets removed:

from tqdm.notebook import tqdm_notebook
from time import sleep
for j in tqdm_notebook(range(2), leave=False):
    sleep(0.1)
    print("some lines")
print("done")
for j in tqdm_notebook(range(2), leave=False):
    sleep(0.1)
    print("some lines")
print("done")

If the vertical space shows up still you either did something wrong, or you run a different jupyter server, or perhaps you have something else overriding the css. To debug the crazy world of multiple css definitions overriding each other, use firefox's or chrome's DOM Inspector (Ctrl-Shift-I). Point to the part of the HTML you're interested in and it'll show you all the involved css and which one overrides which (you may have to click on the little pointer image first to activate the inspecting mode).

See the image with all the important bits covered in this post highlighted.