Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add command to create Pyodide virtual environment #2976

Merged
merged 100 commits into from
Sep 12, 2022

Conversation

hoodmane
Copy link
Member

@hoodmane hoodmane commented Aug 16, 2022

This is work in progress. To use:

  1. pip install pyodide-build
  2. pyodide venv .venv-pyodide
  3. source .venv-pyodide/bin/active
  4. pip install .[tests] or pip install -r test_requirements.txt etc
  5. python -m pytest

@rth
Copy link
Member

rth commented Aug 17, 2022

Very cool! We should probably pre-install micropip here by default, same way as venv pre-installs pip.

It's very impressive that python -m virtualenv works and it's important to have the python executable for a number of applications (e.g. IDE) but I wonder what should be the final user interface for this,
a) I have the impression virtualenv is being more or less deprecated in favor of stdlib venv https://stackoverflow.com/a/47559925/1791279
b) users probably shouldn't have to download the repo to install it. Maybe in the future, we should put this either in some Python (e.g. pyodide-cli) or npm package?

@hoodmane
Copy link
Member Author

It seems to me that the standard library venv is missing a lot of features present in virtualenv. Most critically:

The venv module does not offer all features of this library, to name just a few more prominent:

  • ...
  • cannot create virtual environments for arbitrarily installed python versions (and automatically discover these),
  • ...

https://virtualenv.pypa.io/en/latest/

I think for this reason it is unusable for us. I have just added a script
create_pyodide_venv.py that sets everything up rather nicely.

@henryiii
Copy link
Contributor

henryiii commented Sep 7, 2022

You should add it to pyproject.toml at

module = [

@henryiii
Copy link
Contributor

henryiii commented Sep 7, 2022

(One of the runs is not strictly checked for imports, and it seems this file is being checked in both MyPy runs. You might want to exclude it from one of them.)

Copy link
Member

@rth rth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking forward to this! A couple of comments otherwise LGTM.

.circleci/config.yml Show resolved Hide resolved
]

for pip in other_pips:
pip.unlink()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe missing_ok=True?

dedent(
f"""
#!/bin/sh
{environment} exec {sys.executable} -m pyodide_build.out_of_tree "$@"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will have to change once we migrate to the pyodide-cli but OK to do it in a separate MR.

src/py/pyodide/_package_loader.py Outdated Show resolved Hide resolved
src/tests/test_cmdline_runner.py Outdated Show resolved Hide resolved
src/tests/test_cmdline_runner.py Outdated Show resolved Hide resolved
@hoodmane
Copy link
Member Author

hoodmane commented Sep 7, 2022

@henryiii I figured out the issue: if venv.py is imported from a test then the test mypy will process it. But we can fix this by using importlib.importmodule to import it because then mypy can't understand what happened.

@henryiii
Copy link
Contributor

henryiii commented Sep 8, 2022

Then you can put if typing.TYPE_CHECKING so the easier for type checking import doesn't run at runtime.

Not following closely, but build supports both venv and virtualenv, and virtualenv is much easier to support - as a package, it pushes fixes, while venv has lots of issues.

@hoodmane
Copy link
Member Author

hoodmane commented Sep 8, 2022

@rth @ryanking13 getting the error:

__main__.py: error: argument --rt/--runtime: invalid choice: 'src' (choose from 'firefox', 'chrome', 'safari', 'node', 'host', 'firefox-no-host', 'chrome-no-host', 'safari-no-host', 'node-no-host')

@ryanking13
Copy link
Member

@rth @ryanking13 getting the error:

__main__.py: error: argument --rt/--runtime: invalid choice: 'src' (choose from 'firefox', 'chrome', 'safari', 'node', 'host', 'firefox-no-host', 'chrome-no-host', 'safari-no-host', 'node-no-host')

#3084 will fix it.

@hoodmane hoodmane changed the title Set up command line runner Add command to create Pyodide virtual environment Sep 12, 2022
@hoodmane hoodmane merged commit b6d3281 into pyodide:main Sep 12, 2022
@hoodmane hoodmane deleted the cmdline-runner branch September 12, 2022 18:18
@ryanking13
Copy link
Member

Thanks a lot!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants