500 error (sometimes) when converting to dashboard

[parente]

The critical bit is in the render of the jinja template:

jinja2.exceptions.UndefinedError: \'IPython.nbformat.notebooknode.NotebookNode object\' has no attribute \'layout\

Best guess: the layout loop var is not defined on all iteration of the loop.

[E 17:47:53.751 NotebookApp] Uncaught exception GET /user/9RDX2WXLAjPc/bundle?type=dashboard&notebook=got_scotch_demo%2Fscotch_dashboard.ipynb (::ffff:45.37.178.151)
    HTTPServerRequest(protocol='http', host='jupyter.cloudet.xyz', method='GET', uri='/user/9RDX2WXLAjPc/bundle?type=dashboard&notebook=got_scotch_demo%2Fscotch_dashboard.ipynb', version='HTTP/1.1', remote_ip='::ffff:45.37.178.151', headers={'Accept-Encoding': 'gzip, deflate, sdch', 'Connection': 'close', 'X-Forwarded-For': '::ffff:45.37.178.151', 'Cookie': '_gat=1; _ga=GA1.2.1320566523.1442145202', 'X-Forwarded-Port': '80', 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8', 'Referer': 'http://jupyter.cloudet.xyz/user/9RDX2WXLAjPc/notebooks/got_scotch_demo/scotch_dashboard.ipynb', 'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.80 Safari/537.36', 'Upgrade-Insecure-Requests': '1', 'X-Forwarded-Proto': 'http', 'Accept-Language': 'en-US,en;q=0.8', 'Host': 'jupyter.cloudet.xyz'})
    Traceback (most recent call last):
      File "/opt/conda/lib/python3.4/site-packages/tornado/web.py", line 1413, in _execute
        result = method(*self.path_args, **self.path_kwargs)
      File "/opt/conda/lib/python3.4/site-packages/tornado/web.py", line 2721, in wrapper
        return method(self, *args, **kwargs)
      File "/opt/conda/lib/python3.4/site-packages/urth/dashboard/nbexts/bundle_handler.py", line 58, in get
        self._get_local_app(abs_nb_path)
      File "/opt/conda/lib/python3.4/site-packages/urth/dashboard/nbexts/bundle_handler.py", line 251, in _get_local_app
        md = self._create_app_bundle(abs_nb_path, bundle_root=os.path.join(self.nb_dir, 'local_dashboards'), overwrite=True, template_fn='local.tpl')
      File "/opt/conda/lib/python3.4/site-packages/urth/dashboard/nbexts/bundle_handler.py", line 133, in _create_app_bundle
        converter.to_php_app(abs_nb_path, bundle_dir, template_fn)
      File "/opt/conda/lib/python3.4/site-packages/urth/dashboard/converter/__init__.py", line 48, in to_php_app
        full_output = to_thebe_html(notebook_fn, {}, 'html', os.getcwd(), template_fn)
      File "/opt/conda/lib/python3.4/site-packages/urth/dashboard/converter/__init__.py", line 235, in to_thebe_html
        raise RuntimeError('nbconvert wrote to stderr: {}'.format(stderr))
    RuntimeError: nbconvert wrote to stderr: b'Traceback (most recent call last):\n  File "/opt/conda/bin/ipython", line 6, in <module>\n    sys.exit(start_ipython())\n  File "/opt/conda/lib/python3.4/site-packages/IPython/__init__.py", line 120, in start_ipython\n    return launch_new_instance(argv=argv, **kwargs)\n  File "/opt/conda/lib/python3.4/site-packages/IPython/config/application.py", line 574, in launch_instance\n    app.start()\n  File "/opt/conda/lib/python3.4/site-packages/IPython/terminal/ipapp.py", line 367, in start\n    return self.subapp.start()\n  File "/opt/conda/lib/python3.4/site-packages/IPython/nbconvert/nbconvertapp.py", line 278, in start\n    self.convert_notebooks()\n  File "/opt/conda/lib/python3.4/site-packages/IPython/nbconvert/nbconvertapp.py", line 401, in convert_notebooks\n    self.convert_single_notebook(notebook_filename)\n  File "/opt/conda/lib/python3.4/site-packages/IPython/nbconvert/nbconvertapp.py", line 372, in convert_single_notebook\n    output, resources = self.export_single_notebook(notebook_filename, resources)\n  File "/opt/conda/lib/python3.4/site-packages/IPython/nbconvert/nbconvertapp.py", line 324, in export_single_notebook\n    output, resources = self.exporter.from_filename(notebook_filename, resources=resources)\n  File "/opt/conda/lib/python3.4/site-packages/IPython/nbconvert/exporters/exporter.py", line 156, in from_filename\n    return self.from_notebook_node(nbformat.read(f, as_version=4), resources=resources, **kw)\n  File "/opt/conda/lib/python3.4/site-packages/IPython/nbconvert/exporters/html.py", line 65, in from_notebook_node\n    return super(HTMLExporter, self).from_notebook_node(nb, resources, **kw)\n  File "/opt/conda/lib/python3.4/site-packages/IPython/nbconvert/exporters/templateexporter.py", line 214, in from_notebook_node\n    output = self.template.render(nb=nb_copy, resources=resources)\n  File "/opt/conda/lib/python3.4/site-packages/jinja2/environment.py", line 989, in render\n    return self.environment.handle_exception(exc_info, True)\n  File "/opt/conda/lib/python3.4/site-packages/jinja2/environment.py", line 754, in handle_exception\n    reraise(exc_type, exc_value, tb)\n  File "/opt/conda/lib/python3.4/site-packages/jinja2/_compat.py", line 37, in reraise\n    raise value.with_traceback(tb)\n  File "/opt/conda/lib/python3.4/site-packages/urth/dashboard/converter/jinja_templates/thebe.tpl", line 83, in top-level template code\n    data-gs-x={{layout.col}} data-gs-y={{layout.row}}\n  File "/opt/conda/lib/python3.4/site-packages/jinja2/environment.py", line 408, in getattr\n    return getattr(obj, attribute)\njinja2.exceptions.UndefinedError: \'IPython.nbformat.notebooknode.NotebookNode object\' has no attribute \'layout\'\n\nIf you suspect this is an IPython bug, please report it at:\n    https://github.com/ipython/ipython/issues\nor send an email to the mailing list at ipython-dev@scipy.org\n\nYou can print a more detailed traceback right now with "%tb", or use "%debug"\nto interactively debug it.\n\nExtra-detailed tracebacks for bug-reporting purposes can be enabled via:\n    c.Application.verbose_crash=True\n\n'
[E 17:47:53.759 NotebookApp] {
      "Accept-Encoding": "gzip, deflate, sdch",
      "Connection": "close",
      "X-Forwarded-For": "::ffff:45.37.178.151",
      "Cookie": "_gat=1; _ga=GA1.2.1320566523.1442145202",
      "X-Forwarded-Port": "80",
      "X-Forwarded-Proto": "http",
      "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8",
      "Referer": "http://jupyter.cloudet.xyz/user/9RDX2WXLAjPc/notebooks/got_scotch_demo/scotch_dashboard.ipynb",
      "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.80 Safari/537.36",
      "Upgrade-Insecure-Requests": "1",
      "Accept-Language": "en-US,en;q=0.8",
      "Host": "jupyter.cloudet.xyz"
    }
[E 17:47:53.759 NotebookApp] 500 GET /user/9RDX2WXLAjPc/bundle?type=dashboard&notebook=got_scotch_demo%2Fscotch_dashboard.ipynb (::ffff:45.37.178.151) 970.95ms referer=http://jupyter.cloudet.xyz/user/9RDX2WXLAjPc/notebooks/got_scotch_demo/scotch_dashboard.ipynb

[parente]

Reproduced by trying to deploy the test/test_html_javascript today.

[parente]

test/test_no_layout has the same problem and is probably the source of the problem: template regression when cells have no layout metadata.

[parente]

Found it. Looks like we're not always clearing out the metadata.urth.dashboard object in the notebook document when we hide a cell from the dashboard view. Rather, we just clear the layout. But the thebe.tpl only checks if object existence down to the dashboard property level and then bombs trying to read the layout.

I'll put in the PR. Check for layout existence too.

/cc @dalogsdon @jhpedemonte