debounce/throttle utility decorator for observe functions

[tschopo]

(Original title: Slider flickering in firefox)

When useing the slider widget, the output flickers (see example video). This happens in firefox, chrome works fine. I also tried using different backends.

Also happens with the following code:
https://github.com/jupyter-widgets/ipywidgets/blob/master/docs/source/examples/Image%20Browser.ipynb

Also happens when not using matplotlib, but IPython.display to display the images.

Using IPython 6.1.0, Jupyter 4.3.22

Example: First 10 seconds is firefox, rest is Chrome: https://youtu.be/5osMRj22l8w

[jasongrout]

Can you try the latest beta 7.0.0b2? The printing of the error has a side effect of causing the flickering. See note (4) at #1522.

[tschopo]

Still flickering in Firefox with the beta (but still works in chrome). I also tried setting the widget height. When I put continuous_update=False it flickers once on mouse release.

Now it seems like its buffering the input from the slider, so it keeps playing even when finished adjusting the slider (only firefox, works correctly in chrome). Interrupting the kernel doesn't stop it from playing the buffered frames.

Is displaying the previous image until the next image is fully loaded possible (or displaying the next image on top of the previous one)?

Video of the observed behaviour: https://youtu.be/sSMR9j_B5rA

[jasongrout]

Still flickering in Firefox with the beta (but still works in chrome). I also tried setting the widget height. When I put continuous_update=False it flickers once on mouse release.

Now it seems like its buffering the input from the slider, so it keeps playing even when finished adjusting the slider (only firefox, works correctly in chrome).

This is on 7.0 beta 2, right? We took care of at least one flicker issue in beta 2.

I wonder if this is a speed issue. Widgets will typically wait for one state update to complete before sending the next, to try to prevent having a backlog of state updates to catch up to.

Is displaying the previous image until the next image is fully loaded possible (or displaying the next image on top of the previous one)?

from matplotlib import pyplot as plt
import time
import ipywidgets as widgets


@widgets.interact(x=(0,10))
def f(x):
    time.sleep(3)
    plt.plot([x,10],[3,x])
    plt.show()

illustrates that we leave the previous image up while we wait for the processing and the next image to come.

[tschopo]

Yes this is on the beta 2. My computer is relatively fast (core i7-3520M).

Using Interactive, the output flickers (only in firefox):

import ipywidgets as widgets
import IPython
from io import BytesIO
from PIL import Image
from sklearn import datasets

digits = datasets.load_digits()

frames = digits.images

def view_image(i):
    img = Image.fromarray(frames[i],mode='RGB')
    f = BytesIO()
    img.save(f,'png')
    IPython.display.display(IPython.display.Image(data=f.getvalue(), width=300))
    
interactive_widget = widgets.interactive(view_image, i=widgets.IntSlider(min=0, max=len(frames)-1, step=1)) 

display(interactive_widget)

But when I use an Image widget combined with slider, everything works fine:

import ipywidgets as widgets
import IPython
from io import BytesIO
from PIL import Image
from sklearn import datasets

slider = widgets.IntSlider(min=0, max=len(frames)-1, step=1)

#convert numpy array to bytes
img = Image.fromarray(frames[0],mode='RGB')
f = BytesIO()
img.save(f,'png')

image_widget = widgets.Image(
    value=f.getvalue(),
    format='png',
    height = '500px',
    width = '300px'
)

main_layout = widgets.VBox([slider,image_widget])
display(main_layout)

# slider handler
def on_value_change(change):
    i = change['new']
    img = Image.fromarray(frames[i],mode='RGB')
    f = BytesIO()
    img.save(f,'png')
    image_widget.value = f.getvalue()
    
slider.observe(on_value_change, names='value')

[jasongrout]

The difference between those two cases is that the interact is deleting and creating a new img tag in response to the display message, while the slider/image widget stay on the page and only the image src gets swapped out. I can see that causing a flicker. Perhaps Chrome does some different optimization with deleting and adding new nodes that avoids the flicker.

[tschopo]

That makes sense.

Is there a way to plot without creating new img tags? I tried avoiding Interactive, but the following code also has the flickering problem:

from ipywidgets import IntSlider, Output, VBox
from sklearn import datasets
import matplotlib.pyplot as plt

digits = datasets.load_digits()
frames = digits.images

slider = IntSlider(min=0, max=100, step=1)
output = Output()

def view_image(i):
    output.clear_output(wait=True)
    with output:
        plt.imshow(frames[i['new']], interpolation='nearest')
        plt.show()
slider.observe(view_image, 'value')
VBox([slider, output])

[jasongrout]

That output clearing (output.clear_output) is exactly what interact is doing, and that's where the img tag is getting deleted.

Perhaps you can generated a png directly from matplotlib and use the Image widget, and set its source directly?

[tschopo]

Yes, that works .

import ipywidgets as widgets
import IPython
from io import BytesIO
from PIL import Image
from sklearn import datasets
import matplotlib.pyplot as plt


digits = datasets.load_digits()
frames = digits.images

slider = widgets.IntSlider(min=0, max=len(frames)-1, step=1)

#plot the image to bytebuffer
buf = BytesIO()
plt.imshow(frames[0], interpolation='nearest')
plt.savefig(buf, format='png')

image_widget = widgets.Image(
    value=buf.getvalue(),
    format='png',
    height = '500px',
    width = '300px'
)

main_layout = widgets.VBox([slider,image_widget])
display(main_layout)

# slider handler
def on_value_change(change):
    i = change['new']
    
    #plot the image to bytebuffer
    buf = BytesIO()
    plt.imshow(frames[i], interpolation='nearest')
    plt.savefig(buf, format='png')
    image_widget.value = buf.getvalue()
    
slider.observe(on_value_change, names='value')

Now there is still the "buffering effect", that the images keep changeing even when the slider stopped. Maybe if I allow updateing only every 0.1 seconds or something with a timer...

Thank you so much for your help!

[maartenbreddels]

This should go upstream one day, but you can fix that using what I have in vaex:

from vaex.notebook.utils import debounced
.....
slider.observe(debounced(delay_seconds=0.1)(on_value_change), names='value')

The weird syntax is because it's a decorator, i.e. you can do:

@debounced(delay_seconds=0.1)
def on_value_change(change):
 ...

It should actually have a throttle, not a debounce, but it works better I think.

[jasongrout]

Perhaps we can provide some more support for throttling messages sent from the frontend too (beyond the existing msg_throttle variable).

It would be great as well to support different listeners depending on if you are in middle of a fast stream of state changes, vs at the end. For example, you might want to compute a fast approximation while a slider is being dragged, then a slower final image when the slider drag is completed.

[maartenbreddels]

(see also #663)
I think jslink should support this, and interact as well. But maybe we can have a per widget option, say every widget had a _debounce and/or _throttle property, and save_changes() will respect that. Maybe not for the 7 release though, a bit risky I guess.

[jasongrout]

Maybe not for the 7 release though, a bit risky I guess.

+1. I think we should finalize and release 7.0 at this point.