New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Install multiple requirements files via pip and virtualenv states #4648
Comments
I am going to approve this for the future, thanks for the request |
It'd be pretty sweet if we could get a cwd on virtualenv.managed like there is on pip.installed. I typically have my requirements files split out by environment like this: Base file:
Environment file:
The blank |
@yellottyellott Agreed. Just ran into this issue too. |
I would like to see this as well. |
It looks like we are getting hit by this as well.
requirements.txt
requirements/production.txt
requirements/base.txt
Even with -cwd: and chaining requirements.txt, it keeps using /tmp and not the correct path. Jason Swindle |
Going to look into fixing this. |
Confirmed that the cwd option for the virtualenv state does work using the example state above with pillar data. Can someone include an example state where its not working? |
Multiple requirements appears to work as well. Simply separate them by a comma on in the state. |
I'm working with Jason Swindle above on the same project. When we attempt to run anything that chains requirement files with the -r option in a requirements file the cwd location is not being respected though it is for any file explicitly called from salt. We have requirements.txt in the root of our project directory: /opt/apps/django-project/requirements.txt Its contents contains -r requirements/production.txt which in turn has -r base.txt. Using this method we are chaining the various requirements for different environments keeping common requirements in base.txt and secific environment requirements in each spcecific environment file. When attempting to use virtualenv.managed as seen in Jason's comment above it finds /opt/apps/django-project/requirements.txt file appropriately and attempts to install the requirements in the file. Since that file is present to really only to act as a proxy for PAAS installs it should then hand it off to requirements/production.txt locatied in /opt/apps/django-project/requirements/production.txt. Instead the logs show it is searching /tmp/requirements/production.txt When using virtualenv directly this functionality works as expected it is only when using salt that we encounter this problem. If you need further information let me know. I can provide any amount of debug needed. |
That helps make the situation clearer :) I'll replicate that scenario and see what happens. Thanks. |
Can you provide the state file and pillar files you're using? I tested with the following and it worked without error: State: Pillar: /opt/apps/moo/requirements: /opt/apps/moo/requirements.txt: /opt/apps/moo/requirements/production.txt: /opt/apps/moo/requirements/base.txt: |
Here you go, I hope this helps. :) pillar
sls
Error
|
So unless I'm mistaken, this thread has gone fairly off-topic. Can one of you create a new thread for the cwd stuff? This thread should only be used for the multiple requirements files feature request. |
From my testing with both scenarios, both the cwd option and being able to specify multiple requirement files either by chaining them or specifying multiple requirement files inside the state file, works. |
Can someone else verify on the latest |
Howdy, I am trying to but #9587 sadly is stopping me. Jason Swindle |
Looks like #9587 has been resolved. Can anyone else reproduce these issues? Looks like this has been fixed on develop. |
Since this has been fixed on develop, I am closing this one. If this issue comes up again, leave a comment and we can definitely re-open it and address any problems. |
Can you please clarify what this means?
The documentation for this state still seems to indicate that only one requirements file may be given. |
@chris-martin For specifying multiple requirements files the code over here: https://github.com/saltstack/salt/blob/develop/salt/modules/pip.py#L200 seems to indicate you'd want to use:
For the chaining of requirements files I believe @garethgreenaway is referring to something like this: https://stackoverflow.com/questions/11704287/split-requirements-files-in-pip Where a file contains two additional file references. (which is disgusting by the way @garethgreenaway ;) ) Docs probably need to be updated @jfindlay. This issue should either be re-opened and noted as docs, or a new one created to confirm functionality and document it. |
+1 still an issue... |
For those that are watching this thread, I've got a patch to Summary: With the default method of copy + chown for handling requirements files and a non-root user argument, the existing code takes the referenced requirements file, copies it to a temp file, does its user-switch and then runs against the temp file. The patched module, instead, uses a temp directory filtering on @BaseApi, I know this is an ancient thread. If you'd prefer a new issue (and you are available to even see this), I'd be happy to make the pull req against a new issue. |
Looks like your pull request was submitted and accepted! No need for a new issue as far as I'm concerned. Thanks @gladiatr72 ! (I should also note to be careful of username typos..... @basepi vs @BaseApi, or I might miss something in the future. :) ) |
Just going to chime in and say that while the pip module now supports multiple requirements file, the virtualenv.managed state function still insists that the Would a fix for this be appropriate against current release branches, or would it have to go into develop? |
This is typically something that we would put in develop, since it changes the behavior of a state instead of being a bugfix. Thanks for asking :) |
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue. |
We have some projects where we have more than one requirements file (a core one which everywhere should install, and a site-specific one). At the moment, we have to write out a full state for each file:
It is also painful in a virtualenv definition:
And, of course, we then either have to introduce dependencies between the requirements states or depend on all of the requirements states where we need them.
Preferable would be a list of requirements files:
This would boil down to
pip install -r main_requirements -r other_requirements
.The text was updated successfully, but these errors were encountered: