Developing and exploring templated apps in a notebook

[jbednar]

#2271 implements a mechanism for adding items to a global template object:

import panel as pn, numpy as np, holoviews as hv
pn.extension(theme='dark', template='material', sizing_mode='stretch_width')

xs = np.linspace(0, np.pi)

def sine(freq, phase):
    return hv.Curve((xs, np.sin(xs*freq+phase))).opts(
        responsive=True, min_height=400, title="Sine")

def cosine(freq, phase):
    return hv.Curve((xs, np.cos(xs*freq+phase))).opts(
        responsive=True, min_height=400, title="Cosine")

freq = pn.widgets.FloatSlider(name="Frequency", start=0, end=10, value=2)
phase = pn.widgets.FloatSlider(name="Phase", start=0, end=np.pi)

pn.Column(pn.pane.Markdown("## Settings"), freq, phase).servable(area='sidebar')

pn.Row(hv.DynamicMap(pn.bind(sine,   freq=freq, phase=phase),
       hv.DynamicMap(pn.bind(cosine, freq=freq, phase=phase)
).servable(title='Sine Explorer')

This approach lets a developer explore and debug each component one at a time in a notebook, while supporting separate deployment of the complete templated app using panel serve. What about debugging, developing, or using the entire app, with all the components combined, while you're working in the notebook? That's not addressed in the above code snippet. I think there are the following options:

1. Add pn.state.template.show() to the notebook to launch the full application in a separate browser tab.
   Pros: shows app exactly as it will look when launched separately, with all styling and layout intact.
   Cons: Requires leaving Jupyter and switching browser tabs. Not always possible, e.g. in hosted notebook environments. Not suitable for e.g. online docs. Template widget state won't match the state of the notebook cells, since it's an independent server.
2. Display pn.state.template directly in the notebook
   Pros: Shows full app within a single notebook.
   Cons: Will almost never have reasonable formatting, because Jupyter and the template end up fighting for control over the styling and layout.
3. Add a new function to Panel like pn.state.template.show() but using an iframe to show the full templated app within a Jupyter cell.
   Pros: Shows full app within a single notebook.
   Cons: May not always be possible to start the separate server? App formatting won't necessarily match the fully deployed app due to Jupyter's big margins. Template widget state won't match the state of the notebook cells, since it's an independent server.
4. As a user, add a code cell to the end of your notebook with a notebook-only view of the components of your app, e.g. pn.Row(pn.Column(title,pn.WidgetBox(sidebar_items),main_items))
   Pros: Resulting app should synchronize with the separate cells in the notebook, allowing incremental development and debugging.
   Cons: Requires naming each of the items that go into the template (notice that in the code example above the title, sidebar, etc. items have not been given names, but now we'd need to name everything so we can refer to it later). Very likely that the user would fail to catch one or more components that were added to the sidecar or main areas, put them in the wrong order, etc.; requires a lot of attention and care from the user! Resulting layout won't exactly match deployed app (not just in styling, but also e.g. modals won't work).
5. Add a new component or function to Panel pn.state.template_layout (or similar) that is the equivalent of 4. but assembling items from pn.state.template automatically.
   Pros: Same as for 4., while requiring less code, user effort, and user expertise than 4.
   Cons: Resulting layout won't match deployed app.

Option 1 is already supported. I don't see much use for 2, and 4 seems like a big hassle. But I would vote to add support for 3 and 5. Both of those seem independently useful, with 3 showing an approximation to the full app with styling (letting you debug the template layout and styling), and 5 showing the components without any templated styling (letting you debug the actual functionality while working naturally with the full set of components in your app in a notebook context).

[MarcSkovMadsen]

Regarding. 2. The whole point of why I have spent so much time on web components is that they work great in notebooks.

there are several advanced templates that work great in the notebook. The Fast Templates used to work in the notebook before moving to Panel. The Material Template Notebook in the gallery works great. I.ve also seen a bootstrap based app for power plants that works great in the notebook.

The aim must be to get the templates working in the notebook. The current limitation I believe is the bundling of assets that are not served in the notebook.

[MarcSkovMadsen]

Great looking templates for notebooks is key as that provides an easy way to develop data and ml tools for Jupyter users.

[philippjfr]

Thanks for the great summary. I'll weigh in on the other points later but just wanted to point out that these statements aren't quite accurate:

Template widget state won't match the state of the notebook cells, since it's an independent server.
May not always be possible to start the separate server?

Here's a little demo demonstrating both:

slider = pn.widgets.FloatSlider(end=10)
server = pn.serve(slider, websocket_origin='*', show=False)

iframe = pn.pane.HTML(f'<iframe src="http://localhost:{server.port}" style="border:none;"></iframe>')

pn.Column(
    slider,
    iframe
)

To make this work in remote Jupyter environments we'd have to integrate it with the jupyter server extension somehow so we can proxy the port in Jupyter. I'd like to use that approach to replace the Template notebook repr, which effectively boils down to going with option 3. instead of option 2. Whether to also implement option 5. I don't have strong opinions about.

[philippjfr]

The current limitation I believe is the bundling of assets that are not served in the notebook.

No the problem is that various CSS frameworks will clash horribly with CSS of the notebook itself. While this may work okay for some templates in general it is not an option that can seriously be supported. Therefore the iframe approach is the only thing that can work reliably.

[philippjfr]

Oh, and I'd also like to suggest an option 6. For for JupyterLab and VSCode it would also be great to support previewing apps in another tab.

[MarcSkovMadsen]

The current limitation I believe is the bundling of assets that are not served in the notebook.

No the problem is that various CSS frameworks will clash horribly with CSS of the notebook itself. While this may work okay for some templates in general it is not an option that can seriously be supported. Therefore the iframe approach is the only thing that can work reliably.

Or using a framework built on web components. Or just being a little bit careful - primarily around Bootstrap.

[philippjfr]

Until you try messing with modals for example. I'm thoroughly unconvinced you provide a good experience this way.

[MarcSkovMadsen]

Ok ok 😏modals where not on my mind. You got me there.

[jbednar]

Yes, @philippjfr , I was hoping you'd correct any incorrect or confusing statements of mine above. Maybe you could add edits directly against any items that need clarification? I wasn't sure from your separate reply what the status of these two statements was:

1. Template widget state won't match the state of the notebook cells, since it's an independent server.
2. May not always be possible to start the separate server?

For 1, normally a widget displayed using the usual IPython rich display will be synchronized with what it's attached to. My worry was that a separately launched server might have another copy of the models and no longer be synchronized, making it less natural to debug in Jupyter using a separate cell with a widget and watching how that affects the full app. Depending on how you handle the separate server that may not be an issue, but it doesn't seem to be addressed by your screenshot, which doesn't show a normal widget in one cell controlling an iframed server in the other.

For 2, your reply seems to indicate you need to set up proxying that's not already there, which is precisely the sort of issue I was saying could be involved here, so I'm confused that you say the statement isn't quite accurate. Maybe you're saying that I should have said the Con was "May require special work to handle the separate server when there is only one port available"? If so please edit the list to say that more precisely!

[philippjfr]

Depending on how you handle the separate server that may not be an issue, but it doesn't seem to be addressed by your screenshot, which doesn't show a normal widget in one cell controlling an iframed server in the other.

When iframed it doesn't matter and doesn't know whether it's viewed in the same cell, a different cell, a different notebook or your mobile phone, it's a distinct view running on a separate server but will always be synced with all other views of the same Panel object(s).

Maybe you're saying that I should have said the Con was "May require special work to handle the separate server when there is only one port available"? If so please edit the list to say that more precisely!

Not being able to start the server and not being able to access the server are quite distinct issues. It's also not really to do with having one or more ports available, it's about the ability to access the host from the frontend. Not trying to be pedantic, just clear about the technical issues involved.

[othmane-kada]

Hi, is there any fixes yet?