Add micropip + pyodide-py recipe #20961

yuvipanda · 2022-11-01T19:34:49Z

Generated by grayskull. Useful to try to use micropip to install packages from pypi when using https://github.com/jupyterlite/xeus-python-kernel and mamba / emscriptenforge

Checklist

Generated by grayskull. Useful to try to use micropip to install packages from pypi when using https://github.com/jupyterlite/xeus-python-kernel and mamba / emscriptenforge

conda-forge-linter · 2022-11-01T19:35:16Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipes/micropip) and found it was in an excellent condition.

wolfv · 2022-11-01T19:47:55Z

I think these lines might be an issue https://github.com/pyodide/micropip/blob/8c164d2bc4d3e140339257fcbe8a0d7e500d6521/micropip/_compat_in_pyodide.py#L5-L8 but we can think abotu that later.

CC @DerThorsten

yuvipanda · 2022-11-01T19:53:59Z

@wolfv hmm, the errors make me think this wasn't as unvendored from pyodide as I had thought. Which makes sense maybe.

rth · 2022-11-01T20:02:36Z

recipes/micropip/meta.yaml

+    - python >=3.7
+    - pip
+  run:
+    - python >=3.7


Thanks for doing this @yuvipanda!

Could you please add the following runtime requirements,
https://github.com/pyodide/pyodide/blob/main/packages/micropip/meta.yaml#L16

Looks like there were missing pyodide/micropip#21 but we also didn't have many users of this package outside of Pyodide so far :)

And also https://pypi.org/project/pyodide-py/ is another dependency, you would likely need to package it as well (it's just another pure Python package). The source code is in https://github.com/pyodide/pyodide/tree/main/src/py

Finally, Python 3.7+ is misleading, it actually needs 3.10+ . We could support a broader range of versions if there is demand for it.

conda-forge-linter · 2022-11-01T20:17:13Z

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipes/micropip, recipes/pyodide-py) and found it was in an excellent condition.

yuvipanda · 2022-11-01T20:17:21Z

@rth thank you very much! I've added pyodide-py as a package in the same PR (lmk if it should be split into a different one), and set min supported version to 3.10

rth · 2022-11-01T20:18:42Z

And also pypi.org/project/pyodide-py is another dependency

This is only used here and emscipten forge could have some runtime detection to use the available JS/Py FFI and not rely on pyodide. Still to make the CI pass, for now, probably the easiest is to package pyodide-py.

It should not have any dependencies aside from the Python 3.10+ (and its stdlib).

rth · 2022-11-01T20:19:38Z

Thanks! LGTM. Let's see if CI passes.

yuvipanda · 2022-11-01T20:33:34Z

@rth looks like it passed, yay! Do you have rights to merge? :D

yuvipanda · 2022-11-01T20:47:38Z

@rth @wolfv uh, I should probably add other people as maintainers here as well. Can I add both of you? :D

westurner · 2022-11-02T03:39:54Z

BLD,TST: micropip: A micropip check command
- https://pypi.org/project/pyodide-micropip-test/
BLD,TST: meta.yml: add micropip check test command

westurner · 2022-11-02T03:45:03Z

Is it justified to bundle pytest?

yuvipanda · 2022-11-02T03:53:40Z

@westurner the way I see it, it's super early in micropip's life, and getting this here would enable testing having both conda and pip packages mixed together in jupyterlite (which is currently not possible). I am not particularly involved in micropip, just a random user trying to get noarch packages from conda-forge and pure wheels from pypi working :)

My hope is that this can get merged, and issues with health may be worked on one at a time.

I also don't see they have pytest bundled in their source (https://github.com/pyodide/micropip/tree/main/micropip/). I also don't understand the comment in #20961 (comment). Can you explain?

westurner · 2022-11-02T03:54:34Z

Is it justified to bundle pytest?

"Packages built in Pyodide"
https://pyodide.org/en/stable/usage/packages-in-pyodide.html :

pytest 7.1.2
pytest-benchmark

https://beta.mamba.pm/channels/emscripten-forge?tab=packages&size=25&index=2 :

pytest n/a 7.1.1
pytest_driver n/a 0.5.1
pytest_driver_node n/a 0.5.1

https://github.com/emscripten-forge/recipes#adding-packages

westurner · 2022-11-02T03:58:53Z

recipes/micropip/meta.yaml

+  imports:
+    - micropip
+  commands:
+    - pip check


Is there a micropip check command?

From #20961 (comment) :

BLD,TST: micropip: A micropip check command

https://pypi.org/project/pyodide-micropip-test/

BLD,TST: meta.yml: add micropip check test command

@westurner I don't think there is.

@wolfv what would micropip check check?.

yuvipanda · 2022-11-02T04:00:29Z

@westurner The pyodide-py package does not bundle pytest. I just verified this by installing pyodide-py in a blank venv, and the only files it installs are:

./lib/python3.10/site-packages/pyodide_py-0.22.0a1.dist-info
./lib/python3.10/site-packages/pyodide_py-0.22.0a1.dist-info/RECORD
./lib/python3.10/site-packages/pyodide_py-0.22.0a1.dist-info/WHEEL
./lib/python3.10/site-packages/pyodide_py-0.22.0a1.dist-info/top_level.txt
./lib/python3.10/site-packages/pyodide_py-0.22.0a1.dist-info/REQUESTED
./lib/python3.10/site-packages/pyodide_py-0.22.0a1.dist-info/INSTALLER
./lib/python3.10/site-packages/pyodide_py-0.22.0a1.dist-info/METADATA
./lib/python3.10/site-packages/_pyodide
./lib/python3.10/site-packages/_pyodide/_base.py
./lib/python3.10/site-packages/_pyodide/_core_docs.py
./lib/python3.10/site-packages/_pyodide/_importhook.py
./lib/python3.10/site-packages/_pyodide/__init__.py
./lib/python3.10/site-packages/_pyodide/__pycache__
./lib/python3.10/site-packages/_pyodide/__pycache__/_base.cpython-310.pyc
./lib/python3.10/site-packages/_pyodide/__pycache__/_core_docs.cpython-310.pyc
./lib/python3.10/site-packages/_pyodide/__pycache__/docstring.cpython-310.pyc
./lib/python3.10/site-packages/_pyodide/__pycache__/_importhook.cpython-310.pyc
./lib/python3.10/site-packages/_pyodide/__pycache__/__init__.cpython-310.pyc
./lib/python3.10/site-packages/_pyodide/docstring.py
./lib/python3.10/site-packages/pyodide
./lib/python3.10/site-packages/pyodide/_package_loader.py
./lib/python3.10/site-packages/pyodide/console.py
./lib/python3.10/site-packages/pyodide/_run_js.py
./lib/python3.10/site-packages/pyodide/code.py
./lib/python3.10/site-packages/pyodide/__init__.py
./lib/python3.10/site-packages/pyodide/__pycache__
./lib/python3.10/site-packages/pyodide/__pycache__/_core.cpython-310.pyc
./lib/python3.10/site-packages/pyodide/__pycache__/_state.cpython-310.pyc
./lib/python3.10/site-packages/pyodide/__pycache__/_run_js.cpython-310.pyc
./lib/python3.10/site-packages/pyodide/__pycache__/_package_loader.cpython-310.pyc
./lib/python3.10/site-packages/pyodide/__pycache__/code.cpython-310.pyc
./lib/python3.10/site-packages/pyodide/__pycache__/webloop.cpython-310.pyc
./lib/python3.10/site-packages/pyodide/__pycache__/console.cpython-310.pyc
./lib/python3.10/site-packages/pyodide/__pycache__/http.cpython-310.pyc
./lib/python3.10/site-packages/pyodide/__pycache__/__init__.cpython-310.pyc
./lib/python3.10/site-packages/pyodide/_core.py
./lib/python3.10/site-packages/pyodide/_state.py
./lib/python3.10/site-packages/pyodide/ffi
./lib/python3.10/site-packages/pyodide/ffi/__init__.py
./lib/python3.10/site-packages/pyodide/ffi/__pycache__
./lib/python3.10/site-packages/pyodide/ffi/__pycache__/wrappers.cpython-310.pyc
./lib/python3.10/site-packages/pyodide/ffi/__pycache__/__init__.cpython-310.pyc
./lib/python3.10/site-packages/pyodide/ffi/wrappers.py
./lib/python3.10/site-packages/pyodide/http.py
./lib/python3.10/site-packages/pyodide/webloop.py

pyodide-py isn't all of pyodide, just a bunch of shims

yuvipanda · 2022-11-02T04:02:17Z

@westurner #20961 (comment) has more information on what the pyodide-py package is.

rth · 2022-11-02T08:17:47Z

No, I don't have merge permissions here. Yes, please add me as a co-maintainer.

ocefpaf · 2022-11-02T09:37:18Z

Dumb question: What is the value of micropip outside of the browser? I thought it was only need inside a pyodide distribution.

rth · 2022-11-02T09:48:28Z

What is the value of micropip outside of the browser?

Not much, this is mostly so that emsctipen-forge could use it via the noarch channel as far as I understand.

The pyodide-py is more relevant, as people can import pyodide in notebooks and then those notebooks would fail to work outside of the browser without this package being installed. Jupyterlite tries to smooth this issue by obfuscating most of this, but it still can be an issue, which is the initial reason we uploaded it to PyPI

ocefpaf · 2022-11-02T09:52:45Z

Thanks @rth!

ocefpaf · 2022-11-02T09:53:39Z

Useful to try to use micropip to install packages from pypi when using https://github.com/jupyterlite/xeus-python-kernel and mamba / emscriptenforge

Argh. The problem with so many boilerplate PR/Issues. @yuvipanda had already answered my question and I missed it. Sorry Yuvi!

rth · 2022-11-02T09:55:18Z

Thanks @ocefpaf !

yuvipanda · 2022-11-02T16:04:51Z

THANK YOU SO MUCH, @ocefpaf and @rth!

Add micropip recipe

21544be

Generated by grayskull. Useful to try to use micropip to install packages from pypi when using https://github.com/jupyterlite/xeus-python-kernel and mamba / emscriptenforge

yuvipanda mentioned this pull request Nov 1, 2022

Better integration with conda/conda-forge for building packages pyodide/pyodide#795

Open

rth reviewed Nov 1, 2022

View reviewed changes

rth mentioned this pull request Nov 1, 2022

Specify runtime requirements pyodide/micropip#21

Closed

2 tasks

yuvipanda added 3 commits November 1, 2022 13:07

Add two missing package requirements

0a40471

Set min version to 3.10

aeb2650

Add pyodide-py as a dependency

2793acf

yuvipanda changed the title ~~Add micropip recipe~~ Add micropip + pyodide-py recipe Nov 1, 2022

westurner reviewed Nov 2, 2022

View reviewed changes

ocefpaf merged commit 329f629 into conda-forge:main Nov 2, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add micropip + pyodide-py recipe #20961

Add micropip + pyodide-py recipe #20961

yuvipanda commented Nov 1, 2022

conda-forge-linter commented Nov 1, 2022

wolfv commented Nov 1, 2022

yuvipanda commented Nov 1, 2022

rth Nov 1, 2022

rth Nov 1, 2022 •

edited

Loading

conda-forge-linter commented Nov 1, 2022

yuvipanda commented Nov 1, 2022

rth commented Nov 1, 2022 •

edited

Loading

rth commented Nov 1, 2022

yuvipanda commented Nov 1, 2022

yuvipanda commented Nov 1, 2022

westurner commented Nov 2, 2022 •

edited

Loading

westurner commented Nov 2, 2022

yuvipanda commented Nov 2, 2022

westurner commented Nov 2, 2022

westurner Nov 2, 2022

yuvipanda Nov 2, 2022

westurner Nov 2, 2022

yuvipanda commented Nov 2, 2022

yuvipanda commented Nov 2, 2022

rth commented Nov 2, 2022

ocefpaf commented Nov 2, 2022

rth commented Nov 2, 2022

ocefpaf commented Nov 2, 2022

ocefpaf commented Nov 2, 2022

rth commented Nov 2, 2022

yuvipanda commented Nov 2, 2022

Add micropip + pyodide-py recipe #20961

Add micropip + pyodide-py recipe #20961

Conversation

yuvipanda commented Nov 1, 2022

conda-forge-linter commented Nov 1, 2022

wolfv commented Nov 1, 2022

yuvipanda commented Nov 1, 2022

rth Nov 1, 2022

Choose a reason for hiding this comment

rth Nov 1, 2022 • edited Loading

Choose a reason for hiding this comment

conda-forge-linter commented Nov 1, 2022

yuvipanda commented Nov 1, 2022

rth commented Nov 1, 2022 • edited Loading

rth commented Nov 1, 2022

yuvipanda commented Nov 1, 2022

yuvipanda commented Nov 1, 2022

westurner commented Nov 2, 2022 • edited Loading

westurner commented Nov 2, 2022

yuvipanda commented Nov 2, 2022

westurner commented Nov 2, 2022

westurner Nov 2, 2022

Choose a reason for hiding this comment

yuvipanda Nov 2, 2022

Choose a reason for hiding this comment

westurner Nov 2, 2022

Choose a reason for hiding this comment

yuvipanda commented Nov 2, 2022

yuvipanda commented Nov 2, 2022

rth commented Nov 2, 2022

ocefpaf commented Nov 2, 2022

rth commented Nov 2, 2022

ocefpaf commented Nov 2, 2022

ocefpaf commented Nov 2, 2022

rth commented Nov 2, 2022

yuvipanda commented Nov 2, 2022

rth Nov 1, 2022 •

edited

Loading

rth commented Nov 1, 2022 •

edited

Loading

westurner commented Nov 2, 2022 •

edited

Loading