Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce editablePackageSources to mkPoetryEnv #101

Merged
merged 3 commits into from
May 8, 2020

Conversation

infinisil
Copy link
Contributor

@infinisil infinisil commented May 8, 2020

This introduces a new parameter to mkPoetryEnv which can be used to specify sources for additional python packages that should be available in the resulting environment, loading directly from the mutable source directory. This allows you to call binaries that need to load python modules (like python -m or gunicorn) for development.

This editablePackageSources parameter could be read from the packages entry in pyproject.toml (or inferred if it doesn't exist), and that's what poetry install does, but this would be rather complex to implement correctly, so passing it manually seems to make more sense to me.

The implementation of this works the same way as poetry install works: .pth files (somewhat documented here) are created in the environments site-packages directory, which reference the mutable directory and lets python know to find the modules there.

This PR is sponsored by Niteo

@infinisil infinisil force-pushed the editablePackages branch 2 times, most recently from 1890a9a to 754ede3 Compare May 8, 2020 20:25
@adisbladis adisbladis merged commit bb06c87 into nix-community:master May 8, 2020
@infinisil infinisil deleted the editablePackages branch May 8, 2020 22:24
@infinisil
Copy link
Contributor Author

Hm turns out this doesn't quite work in all cases.

  • Using a pth file makes import's work by having additional paths in sys.path
  • But pkg_resources.require() which is apparently often used to support plugins, doesn't work with just that. To support this an .egg-info directory needs to be generated, and an .egg-link file pointing to the .egg-info needs to be installed in the site packages directory. See here for a bit more info.

Now poetry does this by generating a setup.py (or using the one from the project if it exists), and then calling pip install -e /path/to/setup.py (which is a bit nasty). I believe python /path/to/setup.py develop is used underneath.

But after some digging on the .egg-info format (e.g. the docs here) it looks like we can easily generate that directory statically. The only thing that's required is a PKG-INFO file within it, containing basic metadata as described in PEP 314.

In addition, an entry_points.txt file can be present, which is just a direct conversion from the tool.poetry.plugins key to INI.

Both of these files can therefore relatively easily be generated from the pyproject.toml. I'll try to implement this soon.

@infinisil infinisil mentioned this pull request May 23, 2020
@infinisil
Copy link
Contributor Author

I implemented this now in #110

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants