New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Production-only dependencies #335

Closed
dahlia opened this Issue Apr 27, 2017 · 18 comments

Comments

Projects
None yet
10 participants
@dahlia
Copy link

dahlia commented Apr 27, 2017

I've used the practice of having dev-requirements.txt, requirements.txt, and prod-requirements in my work. While the former two can be replaced by Pipfile's [dev-packages] and [packages], but it seems I can't find any correspondents for Pipfile.

The usual purpose of prod-requirements.txt is, for example, installing packages related to AWS or raven to record server-side exceptions.

I want to sweepingly introduce Pipenv to my work, but the lack of [prod-packages] (or whatever we call it) prevent it from leave a single list of packages. I need to have both Pipenv and prod-requirements.txt which share the many common items, and also need to edit both files when I change dependencies. It's also hard to keep deployment scripts straightforward.

So I suggest Pipfile to have [prod-packages] list and pipenv install to have -p/--prod option. Do this suggestion make sense?

@nateprewitt

This comment has been minimized.

Copy link
Member

nateprewitt commented Apr 27, 2017

Hey @dahlia, thanks for taking the time to open this. This is more of a question for the Pipfile project which is where the Pipfile specification lives. I know that when this has been asked previously Kenneth has expressed disinterest in supporting any more sections at this time. I'd take a look at issues there and possibly open one if you can't find satisfactory information.

I'd say this is unlikely to be implemented at this time unless Kenneth has changed his mind.

@dahlia

This comment has been minimized.

Copy link
Author

dahlia commented Apr 27, 2017

Thanks for your answer.

I wonder how Pipenv users currently workaround this problem. Is maintaining both Pipfile and prod-requirements.txt is the convention?

@kenneth-reitz

This comment has been minimized.

Copy link
Contributor

kenneth-reitz commented Apr 27, 2017

[dependencies] should be your prod dependencies.

@kenneth-reitz

This comment has been minimized.

Copy link
Contributor

kenneth-reitz commented Apr 27, 2017

[dev-dependencies] should be everything else.

@tony

This comment has been minimized.

Copy link

tony commented Oct 12, 2017

FWIW, until pipenv, I kept a requirements/production.txt file.

I don't use gunicorn, dj-database-url or django-redis-cache on my local environment, but I do on prod.

I'll probably bite the bullet use [dependencies] for now. I also had to merge my requirements/test.txt into [dev-dependencies].

@MzHub

This comment has been minimized.

Copy link

MzHub commented Oct 12, 2017

Maybe if pipenv had a parameter for specifying the Pipfile name/path (and the resulting .lock would be based on that)?

Or is it against the design of pipenv?

@nateprewitt

This comment has been minimized.

Copy link
Member

nateprewitt commented Oct 13, 2017

@MzHub, we technically have a PIPENV_PIPFILE environment variable which will allow you to specify a path for a Pipfile. I wouldn't advise using this for the above scenario though. Unfortunately, it's highly unlikely we'll add any support for additional sections or multiple Pipfiles for a single project.

@uranusjr

This comment has been minimized.

Copy link
Member

uranusjr commented Oct 13, 2017

@tony There’s nothing stopping you from keeping requirements/production.txt. I still keep it (but remove the -r part), and do this in production:

$ pipenv install --skip-lock
$ pipenv run pip install -r requirements/production.txt

This issue has been raised quite some times now (mine is #620, there are a few more), and I still think the current approach is not good enough. I don’t know how the Composer community make things work, but all package managers I work with (Bundler, Cargo, NPM) have some sort of ability to achieve this. Given how Python packaging ecosystem works, it would be impossible for Pipfile to replace requirements.txt, but become another thing to learn besides it.

@lorinkoz

This comment has been minimized.

Copy link

lorinkoz commented May 15, 2018

@kennethreitz

[dependencies] should be your prod dependencies.
[dev-dependencies] should be everything else.

Question, every time I need to pipenv install <package> what should I do (best practice to comply with your statement) if:

  1. is dev only -> assumed pipenv install <package> --dev
  2. is production only -> assumed pipenv install <package>
  3. is both dev and production -> run both commands????

Thanks!

@nateprewitt

This comment has been minimized.

Copy link
Member

nateprewitt commented May 15, 2018

Hi @lorinkoz, if a package is required in “both” sections, it’s a production package. packages will always be installed and dev-packages are additive for when you need to do development tasks. You only need to place the package in one section depending on whether it will be needed for the final deployment. Hopefully that clears things up.

@techalchemy

This comment has been minimized.

Copy link
Member

techalchemy commented May 15, 2018

As a small point of additional interest, if you do put it in both sections, the one in dev-packages will just be overriden by the one in packages

@lorinkoz

This comment has been minimized.

Copy link

lorinkoz commented May 16, 2018

@nateprewitt @techalchemy Thanks for both replies :)

I still have a question. I have some production-only packages (not harmful in development but neither required.) Is there a way to flag production-only packages? Quick example: I don't use django-redis in all development environments because some team members are on Windows and somehow installing it is a pain?

This is just an example, I know local environments should resemble as much as possible production environments but sometimes you need to bend the rules a little bit. In those cases we can always resort to temporarily remove the package from the Pipfile, but I was hoping for a permanent solution.

Thanks again!

@kenneth-reitz

This comment has been minimized.

Copy link
Contributor

kenneth-reitz commented May 16, 2018

There is not.

@techalchemy

This comment has been minimized.

Copy link
Member

techalchemy commented May 16, 2018

However you can use the following to only install it if not using windows:

django-redis = {version = "*", markers = "os_name != 'nt'"}

@petarGitNik

This comment has been minimized.

Copy link

petarGitNik commented Jun 29, 2018

@techalchemy How to skip package if production and development OS are the same (same Python version, etc)? I've looked into env markers in PEP508 but I don't see any custom markers that could be used. Something like the following would be very nice to have:

mod_wsgi = { version = "*", my_custom_env_var = "== 'production'" }

@techalchemy

This comment has been minimized.

Copy link
Member

techalchemy commented Jun 29, 2018

The packages section is meant for production dependencies while dev-packages is meant for dev dependencies

@hjwp

This comment has been minimized.

Copy link

hjwp commented Aug 30, 2018

+1 for the ability to specify production-only dependencies. for small projects, i don't want to use postgres (+ install psycopg2) on my dev environment. i can have it in CI if I really want.

@hjwp

This comment has been minimized.

Copy link

hjwp commented Aug 30, 2018

for now, using platform_release might work, since that's different in dev and prod:

"psycopg2" = { version = "*", platform_release = "=='4.4.0-1065-aws'" }
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment