Discuss or document how to include widget output

[choldgraf]

Over in executablebooks/jupyter-cache#31 @chrisjsewell and I ran into a bunch of confusing issues where executing notebooks results in widget state not making it into the output notebook. We've tried a bunch of stuff around "trusting" the notebook etc but can't seem to figure it out.

In the docs we allude to this:

If you can’t view widget results after execution, you may need to select Trust Notebook under the File menu.

But is there a place to programmatically "trust" a notebook so one can do it just before running nbclient?

I imagine that others that use nbclient might run into the same issues, so perhaps this issue is worth documenting a bit more.

[chrisjsewell]

Yes, this is the minimal example that I am having trouble with:

ipywidgets.ipynb:

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from ipywidgets import interact\n", 
    "def f(x):\n",
    "    return x\n",
    "\n",
    "interact(f, x=10)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}

from nbformat import read
from nbclient import execute
nb = read("ipywidgets.ipynb", 4)
execute(nb)

output:

{'cells': [{'cell_type': 'code',
   'execution_count': 1,
   'metadata': {},
   'outputs': [{'output_type': 'display_data',
     'metadata': {},
     'data': {'text/plain': '10'}},
    {'output_type': 'execute_result',
     'metadata': {},
     'data': {'text/plain': ''},
     'execution_count': 1}],
   'source': 'from ipywidgets import interact\ndef f(x):\n    return x\n\ninteract(f, x=10)'}],
 'metadata': {'kernelspec': {'display_name': 'Python 3',
   'language': 'python',
   'name': 'python3'},
  'language_info': {'name': 'python',
   'version': '3.7.6',
   'mimetype': 'text/x-python',
   'codemirror_mode': {'name': 'ipython', 'version': 3},
   'pygments_lexer': 'ipython3',
   'nbconvert_exporter': 'python',
   'file_extension': '.py'},
  'widgets': {'application/vnd.jupyter.widget-state+json': {'state': {'681322fe51bb4ffb9cde6343142607ff': {'model_name': 'LayoutModel',
      'model_module': '@jupyter-widgets/base',
      'model_module_version': '1.2.0',
      'state': {'_model_module': '@jupyter-widgets/base',
       '_model_module_version': '1.2.0',
       '_model_name': 'LayoutModel',
       '_view_count': None,
       '_view_module': '@jupyter-widgets/base',
       '_view_module_version': '1.2.0',
       '_view_name': 'LayoutView',
       'align_content': None,
       'align_items': None,
       'align_self': None,
       'border': None,
       'bottom': None,
       'display': None,
       'flex': None,
       'flex_flow': None,
       'grid_area': None,
       'grid_auto_columns': None,
       'grid_auto_flow': None,
       'grid_auto_rows': None,
       'grid_column': None,
       'grid_gap': None,
       'grid_row': None,
       'grid_template_areas': None,
       'grid_template_columns': None,
       'grid_template_rows': None,
       'height': None,
       'justify_content': None,
       'justify_items': None,
       'left': None,
       'margin': None,
       'max_height': None,
       'max_width': None,
       'min_height': None,
       'min_width': None,
       'object_fit': None,
       'object_position': None,
       'order': None,
       'overflow': None,
       'overflow_x': None,
       'overflow_y': None,
       'padding': None,
       'right': None,
       'top': None,
       'visibility': None,
       'width': None}},
     '41708465e7904d2b8ec584ef0755bffd': {'model_name': 'VBoxModel',
      'model_module': '@jupyter-widgets/controls',
      'model_module_version': '1.5.0',
      'state': {'_dom_classes': ['widget-interact'],
       '_model_module': '@jupyter-widgets/controls',
       '_model_module_version': '1.5.0',
       '_model_name': 'VBoxModel',
       '_view_count': None,
       '_view_module': '@jupyter-widgets/controls',
       '_view_module_version': '1.5.0',
       '_view_name': 'VBoxView',
       'box_style': '',
       'children': ['IPY_MODEL_4aacb2793a314676b0e2f7019098c470',
        'IPY_MODEL_1221e24520064dfeb77f760ba87f4793'],
       'layout': 'IPY_MODEL_681322fe51bb4ffb9cde6343142607ff'}},
     '07cc1dbe3c584ec2ab2e2e38293d6f86': {'model_name': 'LayoutModel',
      'model_module': '@jupyter-widgets/base',
      'model_module_version': '1.2.0',
      'state': {'_model_module': '@jupyter-widgets/base',
       '_model_module_version': '1.2.0',
       '_model_name': 'LayoutModel',
       '_view_count': None,
       '_view_module': '@jupyter-widgets/base',
       '_view_module_version': '1.2.0',
       '_view_name': 'LayoutView',
       'align_content': None,
       'align_items': None,
       'align_self': None,
       'border': None,
       'bottom': None,
       'display': None,
       'flex': None,
       'flex_flow': None,
       'grid_area': None,
       'grid_auto_columns': None,
       'grid_auto_flow': None,
       'grid_auto_rows': None,
       'grid_column': None,
       'grid_gap': None,
       'grid_row': None,
       'grid_template_areas': None,
       'grid_template_columns': None,
       'grid_template_rows': None,
       'height': None,
       'justify_content': None,
       'justify_items': None,
       'left': None,
       'margin': None,
       'max_height': None,
       'max_width': None,
       'min_height': None,
       'min_width': None,
       'object_fit': None,
       'object_position': None,
       'order': None,
       'overflow': None,
       'overflow_x': None,
       'overflow_y': None,
       'padding': None,
       'right': None,
       'top': None,
       'visibility': None,
       'width': None}},
     '468c3fb3ac864c41b384c8417b7c1ccd': {'model_name': 'SliderStyleModel',
      'model_module': '@jupyter-widgets/controls',
      'model_module_version': '1.5.0',
      'state': {'_model_module': '@jupyter-widgets/controls',
       '_model_module_version': '1.5.0',
       '_model_name': 'SliderStyleModel',
       '_view_count': None,
       '_view_module': '@jupyter-widgets/base',
       '_view_module_version': '1.2.0',
       '_view_name': 'StyleView',
       'description_width': '',
       'handle_color': None}},
     '4aacb2793a314676b0e2f7019098c470': {'model_name': 'IntSliderModel',
      'model_module': '@jupyter-widgets/controls',
      'model_module_version': '1.5.0',
      'state': {'_dom_classes': [],
       '_model_module': '@jupyter-widgets/controls',
       '_model_module_version': '1.5.0',
       '_model_name': 'IntSliderModel',
       '_view_count': None,
       '_view_module': '@jupyter-widgets/controls',
       '_view_module_version': '1.5.0',
       '_view_name': 'IntSliderView',
       'continuous_update': True,
       'description': 'x',
       'description_tooltip': None,
       'disabled': False,
       'layout': 'IPY_MODEL_07cc1dbe3c584ec2ab2e2e38293d6f86',
       'max': 30,
       'min': -10,
       'orientation': 'horizontal',
       'readout': True,
       'readout_format': 'd',
       'step': 1,
       'style': 'IPY_MODEL_468c3fb3ac864c41b384c8417b7c1ccd',
       'value': 10}},
     '855d7e689a6345f1b7571114e5eca6dd': {'model_name': 'LayoutModel',
      'model_module': '@jupyter-widgets/base',
      'model_module_version': '1.2.0',
      'state': {'_model_module': '@jupyter-widgets/base',
       '_model_module_version': '1.2.0',
       '_model_name': 'LayoutModel',
       '_view_count': None,
       '_view_module': '@jupyter-widgets/base',
       '_view_module_version': '1.2.0',
       '_view_name': 'LayoutView',
       'align_content': None,
       'align_items': None,
       'align_self': None,
       'border': None,
       'bottom': None,
       'display': None,
       'flex': None,
       'flex_flow': None,
       'grid_area': None,
       'grid_auto_columns': None,
       'grid_auto_flow': None,
       'grid_auto_rows': None,
       'grid_column': None,
       'grid_gap': None,
       'grid_row': None,
       'grid_template_areas': None,
       'grid_template_columns': None,
       'grid_template_rows': None,
       'height': None,
       'justify_content': None,
       'justify_items': None,
       'left': None,
       'margin': None,
       'max_height': None,
       'max_width': None,
       'min_height': None,
       'min_width': None,
       'object_fit': None,
       'object_position': None,
       'order': None,
       'overflow': None,
       'overflow_x': None,
       'overflow_y': None,
       'padding': None,
       'right': None,
       'top': None,
       'visibility': None,
       'width': None}},
     '1221e24520064dfeb77f760ba87f4793': {'model_name': 'OutputModel',
      'model_module': '@jupyter-widgets/output',
      'model_module_version': '1.0.0',
      'state': {'_dom_classes': [],
       '_model_module': '@jupyter-widgets/output',
       '_model_module_version': '1.0.0',
       '_model_name': 'OutputModel',
       '_view_count': None,
       '_view_module': '@jupyter-widgets/output',
       '_view_module_version': '1.0.0',
       '_view_name': 'OutputView',
       'layout': 'IPY_MODEL_855d7e689a6345f1b7571114e5eca6dd',
       'msg_id': '',
       'outputs': []}}},
    'version_major': 2,
    'version_minor': 0}}},
 'nbformat': 4,
 'nbformat_minor': 4}

As you can see, interact(f, x=10) is only outputting the text/plain mimetype and not the additional javascript type that is obtained from normally running this notebook (irrespective of whether the notebook is trusted or not).

[davidbrochart]

Same here, I could not get the widget in the notebook, either by using nbclient or nbconvert with:

jupyter nbconvert --execute ipywidgets.ipynb --ExecutePreprocessor.store_widget_state=True --to notebook

[davidbrochart]

It works fine if you replace ipywidgets.ipynb with just a slider:

from ipywidgets import IntSlider
IntSlider()

ipywidgets.ipynb:

```json { "cells": [ { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from ipywidgets import IntSlider\n", "\n", "IntSlider()" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.10" } }, "nbformat": 4, "nbformat_minor": 4 } ```

output:

```json {"cells": [{"cell_type": "code", "execution_count": 1, "metadata": {"execution": {"iopub.status.busy": "2020-03-15T20:11:39.500488Z", "iopub.execute_input": "2020-03-15T20:11:39.502425Z", "shell.execute_reply": "2020-03-15T20:11:39.555759Z", "iopub.status.idle": "2020-03-15T20:11:39.556131Z"}}, "outputs": [{"output_type": "display_data", "metadata": {}, "data": {"text/plain": "IntSlider(value=0)", "application/vnd.jupy ter.widget-view+json": {"version_major": 2, "version_minor": 0, "model_id": "5eda644bac78432fbc8b15c6d535d84f"}}}], "source": "from ipywidgets import IntSlider\n\nIntSlider()"}], "metadata": {"kernelspec": {"dis play_name": "Python 3", "language": "python", "name": "python3"}, "language_info": {"name": "python", "version": "3.6.10", "mimetype": "text/x-python", "codemirror_mode": {"name": "ipython", "version": 3}, "pygm ents_lexer": "ipython3", "nbconvert_exporter": "python", "file_extension": ".py"}, "widgets": {"application/vnd.jupyter.widget-state+json": {"state": {"2f8b466793ef448fa09311ed9397d53c": {"model_name": "LayoutMo del", "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "state": {"_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "displa y": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_ar eas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_he ight": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": nu ll, "width": null}}, "e0e4171b772142c68de1410e7c141e88": {"model_name": "SliderStyleModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_model_module": "@jupyter-widg ets/controls", "_model_module_version": "1.5.0", "_model_name": "SliderStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "desc ription_width": "", "handle_color": null}}, "5eda644bac78432fbc8b15c6d535d84f": {"model_name": "IntSliderModel", "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "state": {"_dom_clas ses": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "IntSliderModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version" : "1.5.0", "_view_name": "IntSliderView", "continuous_update": true, "description": "", "description_tooltip": null, "disabled": false, "layout": "IPY_MODEL_2f8b466793ef448fa09311ed9397d53c", "max": 100, "min": 0, "orientation": "horizontal", "readout": true, "readout_format": "d", "step": 1, "style": "IPY_MODEL_e0e4171b772142c68de1410e7c141e88", "value": 0}}}, "version_major": 2, "version_minor": 0}}}, "nbformat": 4, "nbformat_minor": 4} ```

In your example, interact is actually a VBox at the top level, which has two children, one of which is the IntSlider. It looks like the logic for storing the widget state in the notebook in nbconvert and nclient doesn't store all the states of the widget hierarchy. I opened an issue in jupyter/nbconvert#1214.

[chrisjsewell]

Thanks @davidbrochart good spot!

[davidbrochart]

I have updated the issue in jupyter/nbconvert#1214, the issue seems to be only related to interact.

[MSeal]

Posting on both threads -- doing this planned work should fix the Output widget object states, if that is the root cause here: #24

[choldgraf]

Awesome, thanks so much @MSeal - and I hope you enjoyed your vacation! It was a good time to take off work before the global economy comes to a halt 😆 😭

[MSeal]

#68 solved this and is out in the 0.4.0 release!