Draw synced data at canvas creation

[martinRenou]

This will fix #64

It also allows retrieving the canvas image when refreshing the page:

This is only possible when setting sync_image_data to True

[janfreyberg]

Hmm, this currently works for me, but only with Canvas, and not MultiCanvas. MultiCanvas only produces a blank canvas.

Is this fixable as well? I mainly use the MultiCanvas, as I am using interactive features and it seems great for that.

[martinRenou]

Right, it should be easily fixable.

A workaround for you would be to set sync_image_data to True for each subcanvas of the multi canvas.

Subcanvases are in private APIs, so it is not perfect. We need to expose sync_image_data to the multi canvas, it should be just a python fix.

[janfreyberg]

ah yes, nice - the refresh works when setting sync_image_data on sub-canvases to True! Very cool!

However - I still don't see the canvases when converted to HTML:

(left is converted with nbconvert --to html)

It seems a bit... flaky? Sometimes I see some of the drawn shapes, but not the background image, sometimes I see nothing in the converted HTML.

[martinRenou]

What is your workflow exactly with nbconvert? So I can reproduce

[martinRenou]

I wonder if nbconvert is not missing all the comm messages, like voilà does https://ipycanvas.readthedocs.io/en/latest/advanced.html

Unfortunately, using the on_client_ready does not work with nbconvert. I feel like this is something that I won't be able to fix easily 😞

The way I understand the problem is that nbconvert executes the Notebook without any client available (no front-end, so no Web canvas API available for drawing).
By executing the Notebook the comm messages are sent containing the draw commands (fill_rect, clear etc), but there is no client yet for receiving and executing those draw commands.

None of these draw commands are saved in the Python widget model, which is why when you actually display the Canvas when opening the html file, there is nothing to draw, and no kernel for sending new draw commands.

I guess all of I just said is only true if you provide --execute to nbconvert, and I don't know it this will ever work.

[martinRenou]

However, without the --execute, if you executed the Notebook by hand using JupyterLab or the classic Jupyter Notebook, I don't see any reason for nbconvert to not work. I'll try to find the issue.

Edit:
Even a simple IntSlider does not show up without the --execute option. I guess this is an nbconvert issue.

[janfreyberg]

Hi, my workflow right now is the following:

• run my notebook, using jupyter notebook, getting my widgets to the state I want them
• save the notebook using "Widgets > Save widget state"
• close jupyter notebook
• run nbconvert --to html (although in practice, this is actually done by nbsphinx)

nbsphinx does not execute if the notebook was saved with output.

This does produce "working" widgets for IntSlider (and others), but I could imagine the fact that there is no canvas being an issue.

I recently saw that nbconvert is thinking about including a web-browser "screenshot" preprocessing step for widgets, which would probably solve these problems:

jupyter/nbconvert#901

Until then, I think keeping screenshots is completely fine; I was just (as always!) looking for a way to automate things :P

thanks for looking into this, I really appreciate it!

[janfreyberg]

Just for reference, in case someone else would like to try it, I now use a little hack:

at the end of my documentation notebook (in which I manually execute all the cells, and do some canvas interactions), I get the image data, and convert it to a ipywidgets.Image widget. I then replace the canvases in my objects with these widgets, click "Widgets -> save widget state", and from then on nbconvert shows the "snapshot" of the canvas.

Unfortunately the annotation does still have to happen completely manual, presumably because of the issue with the lack of client that you mention.