Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Install and use dependencies automatically for new Python projects #4775

Merged
merged 6 commits into from
Jun 9, 2020

Conversation

justinvp
Copy link
Member

@justinvp justinvp commented Jun 8, 2020

Automatically create a virtual environment and install dependencies in it with pulumi new and pulumi policy new for Python templates.

This will save a new virtualenv runtime option in Pulumi.yaml (PulumiPolicy.yaml for policy packs):

runtime:
  name: python
  options:
    virtualenv: venv

virtualenv is the path to a virtual environment that Pulumi will use when running python commands.

Existing projects are unaffected and can opt-in to using this by setting virtualenv, otherwise, they'll continue to work as-is.

Commits in this PR:

  1. Create virtual environment, install deps, and set virtualenv runtime option with pulumi new and pulumi policy new
  2. Use the virtual environment from the Python language plugin
  3. Use the virtual environment for Python dynamic providers
  4. Add some tests

Fixes #4645


When Luke and I chatted about this a while back, he was wondering how we could make it easy to use for existing projects that don't already have the virtual environment created (e.g. git clone examples repo and trying to get started with an example). As a separate change, I think we should consider an additional CLI command or gesture to make this easier. Something like pulumi stack prepare or pulumi install that would "prepare a project". Or potentially have pulumi stack init prompt to create the virtual environment and install dependencies automatically (if interactive). Opened #4777 to track this.

@justinvp justinvp force-pushed the justin/python_venv branch 2 times, most recently from 59766a7 to da406fb Compare June 8, 2020 15:26
Comment on lines 82 to 85
args := []string{host.ServerAddr()}
for k, v := range options {
args = append(args, fmt.Sprintf("-%s=%v", k, v))
}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To make this work for dynamic providers in Python, I needed some way to pass the virtualenv runtime option to the pulumi-resource-pulumi-python provider plugin. This change passes along all runtime options as command line flags to providers. Are we OK doing this? (I looked and all our providers should be able to handle the additional command line flags; they'll be ignored). Should we prefix these flag names with runtime- or something else to make it more clear where these are coming from?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Environment variables seem like a safer option here--have you given those a try?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Environment variables should be able to work for this. I can update the PR to pass them as PULUMI_RUNTIME_<OPTION>.

Note that this does mean the provider plugins would be receiving these values differently than the language and analyzer plugins, which receive them as command line flags.

Copy link
Member

@pgavlin pgavlin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Structure looks good overall. Just one comment re: the use of envvars vs. command line args.

Copy link
Member

@lukehoban lukehoban left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉

Lots of docs and examples we'll want to update once we roll this out. In particular would be great to make sure we document the flag itself in https://www.pulumi.com/docs/intro/languages/python/ so we have a canonical reference to point to around this setting.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Install dependencies automatically for new Python projects
3 participants