Make it as easy as Gradiolite to embed Panel Pyodide into websites #5766

MarcSkovMadsen · 2023-10-28T03:12:28Z

Gradio recently released Gradiolite which makes it super simple to use Gradio in Pyodide and embed it into websites.

<html>
	<head>
		<script type="module" crossorigin src="https://cdn.jsdelivr.net/npm/@gradio/lite/dist/lite.js"></script>
		<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/@gradio/lite/dist/lite.css" />
	</head>
	<body>
		<gradio-lite>
		import gradio as gr

		def greet(name):
			return "Hello, " + name + "!"
		
		gr.Interface(greet, "textbox", "textbox").launch()
		</gradio-lite>
	</body>
</html>

I don't think its nearly as easy with Panel because

no documentation: Its not documented anywhere how to embed Panel apps in existing web sites in a simple way. That would be especially useful for lots of frameworks in the PyData ecosystem.
- Maybe an iframe is the way to go. If that is our recommendation, we should document it.
panel convert not for embedding: Works really well for single applications. But creates highly bloated files that you don't want to look inside and especially not want to copy-paste manually 10 times into a markdown or html file.
pydodide-workers: That is the best way of using Panel with Pyodide. But how to embed that easily inside a website is not clear to me.
pyscript: Pyscript is nice. But slow to load and the UX when loading is worse than Panel.
extensions: If you are using extensions like Tabulator. Its not described how and takes some work to get their js and css correctly added to the .html head.
templates. Requires so much css and js imported. design is maybe the solution here. Again we should document how to use and not use templates and designs.
maintaince: Its really hard to keep pyodide apps working unless you freeze and pin every requirement. I see these apps broken again and again.
panel docs/ sphinx/ nbsite.pyodide: The panel docs with Pyodide are nice. But for others to replicate that they would have to use Sphinx with nbsite which will only work for a subset of sites. For example some use MkDocs and some would not like to depend on an external dependency like nbsite.

I see different solutions

Improved documentation and examples
Improved panel convert that can create files/ code optimized for embedding into webpages
A panel-lite web component that hides all the complexity similiar to gradio-lite.
PyScript speed ups and UX improvements

Some things that would be super helpful too would be

To make it super easy to first show .png, .gif or .mp4+.png and a run button. If the user clicks the run button the Panel application is then embedding. The experience of the Run Button should be similar to the one in Panels documentation with loading indicator and some text explaining steps.
Start talking about Panel in Pyodide/ Pyscript as panel-lite. There is now jupyterlite, stlite for Streamlit and gradio-lite for Gradio. It will just confuse people we use terminology panelite for our version of Jupyterlite and have no terminology for Panel in Pyodide.

Additional Context

Improvements here could also make it much easier for Panel users to share and maintain their apps at pyscript.com.

The text was updated successfully, but these errors were encountered:

MarcSkovMadsen · 2023-10-28T07:11:48Z

Example Issue: Using Extensions

If you want to use extensions like Tabulator, then its not explained in https://panel.holoviz.org/how_to/wasm/standalone.html how to add support.

If users try extending the documented example without adding the relevant js and css dependencies manually, it will fail for them.

<!DOCTYPE html>
<html>
  <head>
    <script src="https://cdn.jsdelivr.net/pyodide/v0.23.4/full/pyodide.js"></script>

    <script type="text/javascript" src="https://cdn.bokeh.org/bokeh/release/bokeh-3.3.0.js"></script>
    <script type="text/javascript" src="https://cdn.bokeh.org/bokeh/release/bokeh-widgets-3.3.0.min.js"></script>
    <script type="text/javascript" src="https://cdn.bokeh.org/bokeh/release/bokeh-tables-3.3.0.min.js"></script>
    <script type="text/javascript" src="https://cdn.jsdelivr.net/npm/@holoviz/panel@1.3.0/dist/panel.min.js"></script>

  </head>
  <body>
    <div id="simple_app"></div>
    <script type="text/javascript">
      async function main(){
        let pyodide = await loadPyodide();
        await pyodide.loadPackage("micropip");
        const micropip = pyodide.pyimport("micropip");
        await micropip.install([
          "https://cdn.holoviz.org/panel/1.3.0/dist/wheels/bokeh-3.3.0-py3-none-any.whl",
          "https://cdn.holoviz.org/panel/1.3.0/dist/wheels/panel-1.3.0-py3-none-any.whl"]
        )
        pyodide.runPython(`
          import panel as pn
          import pandas as pd
          
          pn.extension("tabulator", sizing_mode="stretch_width")

          df = pd.DataFrame({
              'int': [1, 2, 3],
              'float': [3.14, 6.28, 9.42],
              'str': ['A', 'B', 'C'],
              'bool': [True, False, True],
          }, index=[1, 2, 3])

          pn.widgets.Tabulator(df).servable(target="simple_app")
	    `);
      }
      main();
    </script>
  </body>
</html>

pyodide.asm.js:9  future: <Task finished name='Task-46' coro=<write() done, defined at /lib/python3.11/site-packages/panel/io/pyodide.py:435> exception=ReferenceError: Tabulator is not defined>

MarcSkovMadsen · 2023-10-29T06:37:27Z

`panel convert` adds bloat

If it was possible for users to easily see which .js and .css files they need to add then its easier to user and embed Panel.

But if I run

 panel convert script.py --skip-embed

on

import panel as pn

pn.extension("tabulator")

pn.panel("hello").servable()

I get a highly bloated file. For example the tabulator_simple_min.css file is both imported from CDN and inlined.

There are extra divs and custom js compared to the simple description.

philippjfr · 2023-10-29T22:12:05Z

I think we need to make a clear distinction here, panel convert is designed for converting applications, it is not designed for embedding components in another page. For embedding I think we have two possibilities:

Roll out our own web component(s)
Build on top of the new version of PyScript which has support for web workers

I get a highly bloated file. For example the tabulator_simple_min.css file is both imported from CDN and inlined.

Seems like a bug that should be fixed

MarcSkovMadsen · 2023-10-30T03:23:44Z

After having played with it in #5769 (comment), I can see we could simplify a lot because panel convert also works in pyodide. So we can actually let people provide the code only, use find_requirements to find the requirements and panel convert to convert the code into a working template that we can embed in an iframe.

It can be as simple as

<panel-lite>
import panel as pn
pn.extension()

component = pn.panel("Hello World")
pn.template.FastListTemplate(title="Awesome", main=[component]).servable()
<panel-lite>

We just need a function await pn.io.pyodide.render(element, code) that can take panel code, convert it to a template and render it into the element.

In addition we can implement functionality to support hot reload. The key to providing hot reload is to store and reuse the pyodide object. Its amazingly fast.

For more details including what <py-script> should do to enable us to use see #5769 (comment).

MarcSkovMadsen · 2023-10-30T03:32:49Z

I think we need to make a clear distinction here, panel convert is designed for converting applications, it is not designed for embedding components in another page.

I think panel convert could and should support embedding:

Its much easier to run panel convert script.py --embed --to pyscript on the code you have than

going to https://panel.holoviz.org/how_to/wasm/standalone.html
find the relevant section
copy the code
create a new file
paste the html code
replace the panel code with your own panel code
still not be done because how do you get the pn.extension js and css dependencies added?

And if you are a python framework wanting to embed 5-200 examples on your website then you need to implement your own command to automate this.

philippjfr · 2023-10-30T08:00:56Z

I think panel convert could and should support embedding:

I'm not saying we shouldn't provide functionality for generate an embeddable snippet, I'm saying convert is likely the wrong entrypoint for this. I'm not sure you need a CLI utility for it, a Python function seems to cover the main use case.

MarcSkovMadsen · 2023-10-30T13:21:38Z

I think panel convert could and should support embedding:

I'm not saying we shouldn't provide functionality for generate an embeddable snippet, I'm saying convert is likely the wrong entrypoint for this. I'm not sure you need a CLI utility for it, a Python function seems to cover the main use case.

Then you misunderstood me. I'm using the python function behind panel convert. I'm of course not trying to run panel convert in the browser.

It works. The main issue to solve is to only load one Pyodide instance. And to share it across iframes if you have multiple iframes embedding Panel pyodide apps in your app. Sharing across iframes seems hard as currently import js imports the js window where Pyodide was initially loaded. Not where its running.

gshotwell · 2023-12-18T13:51:59Z

Another great option would be to create a quarto extension for Panel Pyodide. I've found that the Shiny quarto extension has made it a lot easier for people to get started with WASM, and we'd be happy to help if you all were interested in that route. This would also be great because it might make it a bit easier to embed panel apps in blog posts.

MarcSkovMadsen · 2023-12-18T15:10:55Z

The understanding i lack is how to load Pyodide once, But use and reuse it over and over again on the main page and inside iframes.

The iframe is important for panel to be able to show our apps in the full templates and individualy styled.

I believe it can be done, But how?

Do you know @gshotwell ?

gshotwell · 2023-12-19T14:19:01Z

Oh yeah we definitely ran into that with both the R and python quarto versions, but it's now resolved. This is how we're able to host all of our examples with WASM without blowing up our network costs or creating a bunch of lag. The extension separates each application on a page and displays them in an iframe so they don't inherit global themes. For example see these: https://shiny.posit.co/py/docs/ui-styling.html

I'm not actually sure of the mechanics of how this is accomplished, but let me ask folks who worked on the extensions.

gshotwell · 2023-12-19T17:08:24Z

From Winston and George:

The way we do this is to load Pyodide in a JS Web/Service worker then distribute the work via message passing communication using tools like window.postMessage(). In Shinylive it’s launched in a web worker, and then each iframe talks to the web worker via an interface that looks like a WebSocket, but instead of sending messages across the network, it sends messages across JS threads via postMessage().

MarcSkovMadsen · 2023-12-20T04:49:28Z

Thanks

Note for self: should try recreate iframe on app rerender instead of trying to reset iframe internals and render

MarcSkovMadsen · 2023-12-23T14:42:56Z

From Winston and George:

The way we do this is to load Pyodide in a JS Web/Service worker then distribute the work via message passing communication using tools like window.postMessage(). In Shinylive it’s launched in a web worker, and then each iframe talks to the web worker via an interface that looks like a WebSocket, but instead of sending messages across the network, it sends messages across JS threads via postMessage().

Thanks @gshotwell . I tried getting quarto installed. My feedback is that its difficult for a Pythonista to get it installed 😄

I will not be allowed to install via the windows installer for security reasons
I'm normally a pip user but quarto cannot be installed that way.
I normally stay away from using conda. But I tried running conda install -c conda-forge r-quarto and it did not install because I need Pearl installed. So I had to run conda install -c conda-forge r-quarto perl.

I would recommend describing how a python user most easily get up and running with quarto in https://quarto.org/docs/computations/python.html.

gshotwell · 2023-12-23T16:21:04Z

That's great feedback @MarcSkovMadsen, I'll pass it on. Does installing the dev version work?

git clone https://github.com/quarto-dev/quarto-cli
cd quarto-cli
./configure.sh

Edit: looks like you can use pip to build quarto, not quite sure why it's not on pypi.

pip install git+ssh://git@github.com/quarto-dev/quarto-cli.git

MarcSkovMadsen · 2023-12-25T05:38:35Z

pip install does not work though. After a pip install I get.

$ quarto preview script.qmd --port 5008
ERROR: NotFound: No such file or directory (os error 2)
Path: /opt/conda/share/quarto/formats/html/bslib/components/scss/mixins/_mixins.scss

Stack trace:
Path: /opt/conda/share/quarto/formats/html/bslib/components/scss/mixins/_mixins.scss
    at readTextFileSync (deno:runtime/js/40_read_file.js:42:16)
    at Object.Deno.readTextFileSync (file:///home/jovyan/repos/private/docs-experiments/.venv/lib/python3.11/site-packages/quarto_cli/quarto-1.4.533/bin/quarto.js:5623:25)
    at bslibComponentMixins (file:///home/jovyan/repos/private/docs-experiments/.venv/lib/python3.11/site-packages/quarto_cli/quarto-1.4.533/bin/quarto.js:42998:17)
    at layerQuartoScss (file:///home/jovyan/repos/private/docs-experiments/.venv/lib/python3.11/site-packages/quarto_cli/quarto-1.4.533/bin/quarto.js:108756:9)
    at resolveBootstrapScss (file:///home/jovyan/repos/private/docs-experiments/.venv/lib/python3.11/site-packages/quarto_cli/quarto-1.4.533/bin/quarto.js:108796:22)
    at boostrapExtras (file:///home/jovyan/repos/private/docs-experiments/.venv/lib/python3.11/site-packages/quarto_cli/quarto-1.4.533/bin/quarto.js:109998:25)
    at themeFormatExtras (file:///home/jovyan/repos/private/docs-experiments/.venv/lib/python3.11/site-packages/quarto_cli/quarto-1.4.533/bin/quarto.js:111806:16)
    at Object.formatExtras (file:///home/jovyan/repos/private/docs-experiments/.venv/lib/python3.11/site-packages/quarto_cli/quarto-1.4.533/bin/quarto.js:111269:111)
    at async runPandoc (file:///home/jovyan/repos/private/docs-experiments/.venv/lib/python3.11/site-packages/quarto_cli/quarto-1.4.533/bin/quarto.js:68671:60)
    at async renderPandoc (file:///home/jovyan/repos/private/docs-experiments/.venv/lib/python3.11/site-packages/quarto_cli/quarto-1.4.533/bin/quarto.js:69969:26)

./configure.sh won't work for my use case. On windows sh scripts are not supported. And linux is running inside a pod where users cannot install things in this way. It needs to be pre-installed which is a prioritization of another teams time away.

gshotwell · 2023-12-26T18:25:58Z

Drag, would you mind raising a quarto issue? https://github.com/quarto-dev/quarto-cli/issues

MarcSkovMadsen added the type: enhancement Minor feature or improvement to an existing feature label Oct 28, 2023

MarcSkovMadsen added this to the Wishlist milestone Oct 28, 2023

flxmr mentioned this issue Apr 24, 2024

Pyodide Conversion: Bundle all the necessary ressources to load from a single webserver #6789

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make it as easy as Gradiolite to embed Panel Pyodide into websites #5766

Make it as easy as Gradiolite to embed Panel Pyodide into websites #5766

MarcSkovMadsen commented Oct 28, 2023 •

edited

Loading

MarcSkovMadsen commented Oct 28, 2023 •

edited

Loading

MarcSkovMadsen commented Oct 29, 2023 •

edited

Loading

philippjfr commented Oct 29, 2023

MarcSkovMadsen commented Oct 30, 2023 •

edited

Loading

MarcSkovMadsen commented Oct 30, 2023 •

edited

Loading

philippjfr commented Oct 30, 2023 •

edited

Loading

MarcSkovMadsen commented Oct 30, 2023 •

edited

Loading

gshotwell commented Dec 18, 2023

MarcSkovMadsen commented Dec 18, 2023 •

edited

Loading

gshotwell commented Dec 19, 2023 •

edited

Loading

gshotwell commented Dec 19, 2023

MarcSkovMadsen commented Dec 20, 2023

MarcSkovMadsen commented Dec 23, 2023 •

edited

Loading

gshotwell commented Dec 23, 2023 •

edited

Loading

MarcSkovMadsen commented Dec 25, 2023 •

edited

Loading

gshotwell commented Dec 26, 2023 •

edited

Loading

Make it as easy as Gradiolite to embed Panel Pyodide into websites #5766

Make it as easy as Gradiolite to embed Panel Pyodide into websites #5766

Comments

MarcSkovMadsen commented Oct 28, 2023 • edited Loading

Additional Context

MarcSkovMadsen commented Oct 28, 2023 • edited Loading

Example Issue: Using Extensions

MarcSkovMadsen commented Oct 29, 2023 • edited Loading

panel convert adds bloat

philippjfr commented Oct 29, 2023

MarcSkovMadsen commented Oct 30, 2023 • edited Loading

MarcSkovMadsen commented Oct 30, 2023 • edited Loading

philippjfr commented Oct 30, 2023 • edited Loading

MarcSkovMadsen commented Oct 30, 2023 • edited Loading

gshotwell commented Dec 18, 2023

MarcSkovMadsen commented Dec 18, 2023 • edited Loading

gshotwell commented Dec 19, 2023 • edited Loading

gshotwell commented Dec 19, 2023

MarcSkovMadsen commented Dec 20, 2023

MarcSkovMadsen commented Dec 23, 2023 • edited Loading

gshotwell commented Dec 23, 2023 • edited Loading

MarcSkovMadsen commented Dec 25, 2023 • edited Loading

gshotwell commented Dec 26, 2023 • edited Loading

MarcSkovMadsen commented Oct 28, 2023 •

edited

Loading

MarcSkovMadsen commented Oct 28, 2023 •

edited

Loading

MarcSkovMadsen commented Oct 29, 2023 •

edited

Loading

`panel convert` adds bloat

MarcSkovMadsen commented Oct 30, 2023 •

edited

Loading

MarcSkovMadsen commented Oct 30, 2023 •

edited

Loading

philippjfr commented Oct 30, 2023 •

edited

Loading

MarcSkovMadsen commented Oct 30, 2023 •

edited

Loading

MarcSkovMadsen commented Dec 18, 2023 •

edited

Loading

gshotwell commented Dec 19, 2023 •

edited

Loading

MarcSkovMadsen commented Dec 23, 2023 •

edited

Loading

gshotwell commented Dec 23, 2023 •

edited

Loading

MarcSkovMadsen commented Dec 25, 2023 •

edited

Loading

gshotwell commented Dec 26, 2023 •

edited

Loading