New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow creating requirements.txt files without dev-packages #942

Closed
tuukkamustonen opened this Issue Oct 19, 2017 · 16 comments

Comments

Projects
None yet
8 participants
@tuukkamustonen
Copy link

tuukkamustonen commented Oct 19, 2017

To install a set of pinned packages in remote systems, there is:

pipenv install --system  # won't install dev dependencies

However, this requires both Pipfile and Pipfile.lock to be present, and also pipenv to be installed in the target system. It's way more inconvenient than just being able to (maybe first copy package wheel into remote system and) say pip install <mypackage> (and letting pip to pull in the pinned dependencies specified in setup.py's install_requires).

(Or is there some benefit in using pipenv install --system instead?)

To be able to generate pinned requiremets.txt that excludes development packages, add --no-dev option to pipenv lock, that seems to by default include also dev packages. Or maybe even change to default to match that of pipenv install, and exclude dev-packages by default?

As per #209 (comment) this is already supported in code:

from pipenv.project import Project
from pipenv.utils import convert_deps_to_pip

pfile = Project(chdir=False).parsed_pipfile
requirements = convert_deps_to_pip(pfile['packages'], r=False)

Although there is workaround for this need already, supporting it right in the cli would/could be nice.

@erinxocon

This comment has been minimized.

Copy link
Contributor

erinxocon commented Oct 19, 2017

I think we could add a method that doesn't add dev packages tot he requirements file. It's tough because normally you'd just have everything in your requirements.txt or maybe had two for production and dev dependencies. The reason pipenv install --system instead of just pip is that with the PIpfile.lock, you get a deterministic build every time you install.

@tuukkamustonen

This comment has been minimized.

Copy link
Author

tuukkamustonen commented Oct 19, 2017

The reason pipenv install --system instead of just pip is that with the PIpfile.lock, you get a deterministic build every time you install.

Yeah, but you get the same result if you depend on requirements.txt (read into install_requires) that was created by pip-compile from requirements.in (from pip-tools package). Right? I don't see any difference.

maybe had two for production and dev dependencies.

Yeah, it's pretty common practice to have separate requirements.txt and dev-requirements.txt (or rather requirements.in and dev-requirements.in if using pip-tools).

@techalchemy

This comment has been minimized.

Copy link
Member

techalchemy commented Oct 19, 2017

@tuukkamustonen I think if you prefer using piptools with pip-compile directly you are certainly able to do that, pipenv uses pip-tools to build lockfiles so technically you can do that yourself, if you like. If you don't see the difference between pipenv and using pip-compile from pip-tools then you likely wont get anything out of using pipenv.

@piotr-dobrogost

This comment has been minimized.

Copy link

piotr-dobrogost commented Oct 19, 2017

@tuukkamustonen
Btw, you should not read requirements.txt into install_requireshttps://caremad.io/posts/2013/07/setup-vs-requirement/

@tuukkamustonen

This comment has been minimized.

Copy link
Author

tuukkamustonen commented Oct 19, 2017

@techalchemy Wasn't aware of the internal use of pip-tools. Neat.

I'm not sure what you mean though - pip-tools doesn't support reading Pipfile, so I cannot mix-and-match pipenv and pip-tools usage directly from cli. With some wrapper code surely, but I wouldn't want to write that. And I would like to use (or at least try) Pipfile as it seems to be the next blessed thing(tm).

@piotr-dobrogost I'm not doing that for libraries (that I use as dependency in other projects) - merely for the final deliverables (web apps, in my case) that I deploy to remote servers.

@piotr-dobrogost

This comment has been minimized.

Copy link

piotr-dobrogost commented Oct 19, 2017

Related – #916 (feature request: pipenv lock -r without the dev packages)

@erinxocon

This comment has been minimized.

Copy link
Contributor

erinxocon commented Oct 24, 2017

There is indeed a --deploy option now for not installing dev packages.

@erinxocon erinxocon closed this Oct 24, 2017

@tuukkamustonen

This comment has been minimized.

Copy link
Author

tuukkamustonen commented Oct 24, 2017

@erinxocon Ok so my initial assumption:

pipenv install --system # won't install dev dependencies

Was not correct. The above line does install dev packages. So it actually requires --deploy flag also. But the documentation for it only states:

Abort if the Pipfile.lock is out–of–date, or Python version is wrong.

Should it be updated?

Also, this ticket was about supporting dev-package exclusion in pipenv lock -r reqs.txt (through something like pipenv lock --no-dev -r reqs.txt). Would it be possible to add such option as well?

@techalchemy

This comment has been minimized.

Copy link
Member

techalchemy commented Oct 24, 2017

The documentation is out of date. I actually knew we had a —deploy flag but didn’t have a chance to figure out what it does. If you had a PR with updates to the documentation I’m sure it would be accepted

As for lock -r it only locks things which are installed. I realize that is not entirely helpful but the last piece of functionality doesn’t exist yet.

@tuukkamustonen

This comment has been minimized.

Copy link
Author

tuukkamustonen commented Oct 25, 2017

As for lock -r it only locks things which are installed.

So things that show up in pip list? Doesn't match the behavior

mkdir foo
cd foo
pipenv install jinja2  # installs also markupsafe as dependency
pipenv uninstall jinja2
pipenv lock -r

Yields:

markupsafe==1.0 --hash=sha256:a6be69091dac236ea9c6bc7d012beab42010fa914c459791d627dad4910eb665
jinja2==2.9.6 --hash=sha256:2231bace0dfd8d2bf1e5d7e41239c06c9e0ded46e70cc1094a0aa64b0afeb054  --hash=sha256:ddaa01a212cd6d641401cb01b605f4a4d9f37bfc93043d7f760ec70fb99ff9ff

Do you mean that it should exclude jinja2 from the output?

I will look into PR once I understand how things are supposed to work :|

the last piece of functionality doesn’t exist yet.

Can we re-open this ticket then?

@erinxocon

This comment has been minimized.

Copy link
Contributor

erinxocon commented Oct 25, 2017

So here's the syntax for the new functionality

pipenv lock -r will give you just the default and vcs deps

pipenv lock -r -d will give you just the development deps and development vcs deps

Will update the documentation.

@erinxocon erinxocon closed this Oct 25, 2017

@wooyek

This comment has been minimized.

Copy link

wooyek commented Dec 14, 2017

@erinxocon I'm confused. Should it work in 9.0.0?

Calling pipenv lock -r gives me all requirements, including the development ones.

@nateprewitt

This comment has been minimized.

Copy link
Member

nateprewitt commented Dec 14, 2017

@wooyek this was a regression in 9.0.0 and will be fixed in the next release.

@gregbrowndev

This comment has been minimized.

Copy link

gregbrowndev commented Sep 18, 2018

Is it now not possible to generate a requirements.txt with both production and dev dependencies in one file?

For context I'm developing inside a Docker container so want to install everything into the system environment not in a virtual env. I've avoided using pipenv install --dev --system in the Dockerfile, since it lead to a bootstrapping issue where the lock file was out of sync with the Pipfile so the image didn't build properly, so I had to remove all of that from my Dockerfile and go into the container and install using pipenv install --dev within a virtual env just so the lock file would update in my local project.

@uranusjr

This comment has been minimized.

Copy link
Member

uranusjr commented Sep 18, 2018

-r -d

@gregbrowndev

This comment has been minimized.

Copy link

gregbrowndev commented Oct 2, 2018

-r -d

@uranusjr thanks but that creates a requirements file with just the development dependencies. I think the best solution is to use --system option and ensure the lock file is up-to-date.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment