Python files as static assets ? #425

JCapul · 2022-12-06T14:09:48Z

Hi @whitphx
Awesome project ! Thanks.

Is there a way to have streamlit python files as static assets, instead of embedding them in the HTML file ?

whitphx · 2022-12-07T03:10:47Z

Hi, thank you.

stlite itself does not provide that feature, but you can download and pass the hosted Python files to stlite by yourself like the example below.

<!DOCTYPE html>
<html>
  <head>
    <meta charset="UTF-8" />
    <meta http-equiv="X-UA-Compatible" content="IE=edge" />
    <meta
      name="viewport"
      content="width=device-width, initial-scale=1, shrink-to-fit=no"
    />
    <title>stlite app</title>
    <link
      rel="stylesheet"
      href="https://cdn.jsdelivr.net/npm/@stlite/mountable/build/stlite.css"
    />
  </head>
  <body>
    <div id="root"></div>
    <script src="https://cdn.jsdelivr.net/npm/@stlite/mountable/build/stlite.js"></script>
    <script>
      async function main() {
        const scriptUrl = "https://gist.githubusercontent.com/whitphx/850a985c1db91744b04f24fcde1a5637/raw/022df00377b5e78329879a30c21dda55dc35720a/streamlit_hello_world.py";
        const scriptContent = await fetch(scriptUrl).then(res => res.text());
        stlite.mount({
          files: {
            "streamlit_app.py": scriptContent,
          },
          entrypoint: "streamlit_app.py"
        },
          document.getElementById("root")
        );
      }
      main();
    </script>
  </body>
</html>

JCapul · 2022-12-07T12:35:58Z

thanks for your reply !

Actually, I've just found a workaround:

I turned my streamlit app into a python package with a main() function
then I build it as a wheel and put the wheel alongside the index.html
in the index.html, I add the wheel URL in the stlite mount requirements and a tiny python script that import my package and run the main function
then I can serve the app (index.html + wheel) with a simple http server (like python http.server for local dev)

Et voila !

PS: I noticed from the console in debug mode that the task "setting the loggers" takes quite a while, not sure if it's an expected behaviour.

whitphx · 2022-12-07T13:41:38Z

That's good.

Then I imagine your case is there are so many source files that you don't want to either embed all of them into index.html or write fetch() for each in index.html.
Am I correct? If so, stlite is currently not providing a convenient way to solve it, and workaround like yours is the only way.
Do you think it would be better if there are some solution for it?

PS: I noticed from the console in debug mode that the task "setting the loggers" takes quite a while, not sure if it's an expected behaviour.

Thank you. It is an expected behavior, but I admit its log label is confusing.
I will fix it -> #427

JCapul · 2022-12-07T14:24:19Z

Then I imagine your case is there are so many source files that you don't want to either embed all of them into index.html or write fetch() for each in index.html.

Exactly.

As for a more convenient way to do it, maybe something like what pyscript provides for local python modules would be nice, but even with pyscript It seems it's limited to individual modules (but I might be wrong, haven't checked thoroughly).

By using a proper packaging approach like my workaround, you get all the benefits that comes with it, including dependency management, embedding of data assets or other resources (ref to #426). So I am not sure it would be worth the hassle, considering also that it's quite straight forward to make it work. So personnaly, I think I would be fine with my approach.

whitphx · 2022-12-14T14:37:04Z

Thank you for your great suggestion.
Let me suggest the following approach inspired by yours.

How about the new files option signature and the new archives option like below? Does it seem to satisfy your needs?

stlite.mount(
  {
    entrypoint: "streamlit_app.py",
    files: {
      // 👇 This signature is already available.
      "streamlit_app.py": `
import streamlit as st
from .data import some_func

st.write(some_func())
`,
      // 👇 This is a new API to download a remote file `./data.py` to the local `data.py`.
      "data.py": {
        url: "./data.py"
      },
    },
    archives: [
      // 👇 Downloads an archive file from a URL `./some_package.tar.gz` and unpack it into `.`.
      {
        url: "./some_package.tar.gz",
        dest: ".",
      }
    ]
  },
  document.getElementById("root")
);

A file download API ({ url: "./data.py" } in the example above) covers what PyScript's local modules feature does, which just downloads the specified files via HTTP to the virtual file system too.
The archives option covers what your workaround did.
- FYI, such archiving approach is a recommended way by Pyodide official: https://pyodide.org/en/stable/usage/faq.html#how-can-i-load-external-files-in-pyodide

If you want to take the advantage of the Wheel packaging like what you said such as dependency management,
you can also specify the wheel file URL in the requirements field like below instead of the files field with the unpack option.

stlite.mount(
  {
    entrypoint: "streamlit_app.py",
    requirements: [
      // 👇 Set the wheel file URL.
      "https://yoursite/your/package.whl"
    ],
    files: {
      "streamlit_app.py": `
import streamlit as st
from .data import some_func

st.write(some_func())
`,
    },
  },
  document.getElementById("root")
);

BTW,
The link you pasted above referring to PyScript's local modules seems to have been changed:
https://docs.pyscript.net/latest/reference/elements/py-config.html#local-modules

JCapul · 2022-12-14T20:51:59Z

I think these would be great additions !
If I understand correctly, you would then be covering most of "external files" use cases IMO:

file download option -> a single-file streamlit app
archive option -> a multi-files strealmlit app with no external dependencies
requirements + wheel URL -> packaged streamlit app with external dependencies

My use case had some dependencies, so I did the latter trick. But without deps, I would definitely go with the archive option.

Just a couple of questions:

in both file download and archive options, you're giving local URL examples (e.g. ./data.py), so I understand that the target file or archive could be a static asset of the site, am I correct ?
in requirements, can the wheel URL be pointing to a static asset of the site ? like this
requirements: ["./path/to/package.whl"]

Thanks !

whitphx · 2022-12-15T17:52:50Z

Thank you, I hope it solves many problems.

in both file download and archive options, you're giving local URL examples (e.g. ./data.py), so I understand that the target file or archive could be a static asset of the site, am I correct ?

Yes, but not limited to it.
Sorry it was confusing as I just used the Pyodide sample, but please just think that the URL like ./data.py would be simply passed to fetch() to download. So such relative URLs will be resolved based on the current page URL, and absolute URLs like https://yoursite/file.py can also be used.

in requirements, can the wheel URL be pointing to a static asset of the site ? like this
requirements: ["./path/to/package.whl"]

Currently it may be no because the requirement list will be passed to micropip.install() but it does not support relative URLs (this is the reason you had to specify the URL of the wheel file in the requirements field).
However, I think it's possible to inject preprocessing to convert relative URLs to be absolute ones before passing them to micropip.
Do you think such relative path resolution is beneficial?

JCapul · 2022-12-19T13:39:08Z

Thank you for the answers.

I'd personnaly consider the relative URLs in requirements as a handy feature.

Though I am a bit confused about whether it's already feasible in micropip, for instance this issue: pyodide/pyodide#867
or this comment: pyodide/pyodide#2731 (comment)

whitphx · 2022-12-22T13:15:05Z

Thank you,

hmm, I think it didn't work when I tested roughly, but there might be something wrong.

I think the details will be revealed during the development ✌️

cameronraysmith · 2023-03-06T05:47:08Z

Actually, I've just found a workaround:

I turned my streamlit app into a python package with a main() function

then I build it as a wheel and put the wheel alongside the index.html

in the index.html, I add the wheel URL in the stlite mount requirements and a tiny python script that import my package and run the main function

then I can serve the app (index.html + wheel) with a simple http server (like python http.server for local dev)

Et voila !

@JCapul I would be interested to see the example of #425 (comment) if there is a pointer to an open-source copy you are able to share. No worries if not as your description is ultimately quite clear! Thank you!

@whitphx #425 (comment) sounds like a very helpful feature set! Thank you!

whitphx · 2023-06-15T15:11:04Z

The archives and files.url options has been added since 0.32.0.
Please try it out, and let us know if there are something wrong.
Thanks!

whitphx mentioned this issue Dec 7, 2022

Can you import external assets into stlite in web mode? #426

Closed

whitphx mentioned this issue Dec 7, 2022

Change the boot-up log labels to be clearer #427

Closed

whitphx added the Priority: Medium label Feb 10, 2023

whitphx mentioned this issue Jun 14, 2023

Add an archives option to load and unpack archive files into the virtual file system #567

Merged

whitphx closed this as completed in #567 Jun 15, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Python files as static assets ? #425

Python files as static assets ? #425

JCapul commented Dec 6, 2022

whitphx commented Dec 7, 2022

JCapul commented Dec 7, 2022 •

edited

whitphx commented Dec 7, 2022

JCapul commented Dec 7, 2022

whitphx commented Dec 14, 2022 •

edited

JCapul commented Dec 14, 2022

whitphx commented Dec 15, 2022 •

edited

JCapul commented Dec 19, 2022

whitphx commented Dec 22, 2022

cameronraysmith commented Mar 6, 2023 •

edited

whitphx commented Jun 15, 2023 •

edited

Python files as static assets ? #425

Python files as static assets ? #425

Comments

JCapul commented Dec 6, 2022

whitphx commented Dec 7, 2022

JCapul commented Dec 7, 2022 • edited

whitphx commented Dec 7, 2022

JCapul commented Dec 7, 2022

whitphx commented Dec 14, 2022 • edited

JCapul commented Dec 14, 2022

whitphx commented Dec 15, 2022 • edited

JCapul commented Dec 19, 2022

whitphx commented Dec 22, 2022

cameronraysmith commented Mar 6, 2023 • edited

whitphx commented Jun 15, 2023 • edited

JCapul commented Dec 7, 2022 •

edited

whitphx commented Dec 14, 2022 •

edited

whitphx commented Dec 15, 2022 •

edited

cameronraysmith commented Mar 6, 2023 •

edited

whitphx commented Jun 15, 2023 •

edited