GitHub Action to build pyodide packages #2719
jobovy
started this conversation in
Show and tell
Replies: 1 comment 3 replies
-
Thanks @jobovy! I think this can be simplified a lot soon. In fact the set of files needed to build packages is pretty small, under 2mb. Some partial progress on this: I have been working on adding CI to pyo3, and there I manage to build Pyodide packages out of tree, not in the docker image, and without building Pyodide. Do you have an example repo where you are using this buildpackage-action? I could make a PR on your repo trying to prototype better out of tree builds. |
Beta Was this translation helpful? Give feedback.
3 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I created a simple GitHub Action to build
pyodide
wheels of Python packages in a workflow:https://github.com/jobovy/pyodide-buildpackage-action
The intended use cases are:
pyodide
wheels of the latest development version for packages in thepyodide
distribution or of a new release of such a package to have it available before it gets updated inpyodide
.pyodide
wheels for packages that aren't in the distributionBecause of use case 1., the basic flow of the action is that it takes a
meta.yaml
file (e.g., that for a package already inpyodide
) and replaces the URL with another URL (default: the commit that triggered the workflow that uses this action). The action is implemented in a modular fashion that allows on to (relatively) easily build multiple packages (e.g., all of the dependencies of a package).Some comments about the implementation:
pyodide
version/tag/branch, copy in an updatedmeta.yaml
, build the package and store the resulting wheel.WORKDIR
and replaces it with its own working directory, the action can't use the built version ofpyodide
, but has to re-build it from scratch. If there was some way to avoidWORKDIR
in the Docker image, then this could potentially be avoided.pyodide
takes a long time, I implemented caching so subsequent build are sped up. This works well, but the cache is very large, a few GB. So it fills up much of the cache available to a repository.pyodide
(mainlyscipy
), by default I only build the package and anylibrary/sharedlibrary
dependencies. That does mean that if a package needs another non-library/sharedlibrary
dependency during build time that this would fail. I'm not sure whether that's actually ever the case, but I imagine it is. In that case, specifying which are build-time dependencies and which are run-time dependencies in themeta.yaml
file would be useful.I hope this is useful to other users and I would appreciate any feedback!
Beta Was this translation helpful? Give feedback.
All reactions