New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature request: pipenv bundle command #986

Closed
nicholasbishop opened this Issue Oct 27, 2017 · 9 comments

Comments

Projects
None yet
7 participants
@nicholasbishop

nicholasbishop commented Oct 27, 2017

(Hope it's OK to file a feature request here, didn't see any explicit documentation regarding that.)

tl;dr: I'd like to use pipenv to generate a bundle suitable for AWS Lambda

For reference, here is the AWS Lambda documentation for how to create a bundle with pip:
http://docs.aws.amazon.com/lambda/latest/dg/lambda-python-how-to-create-deployment-package.html#deployment-pkg-for-virtualenv

There are a couple existing pipenv issues that touch on this use case:

But neither of those directly requests this feature, so I figured it might be helpful to do so.

@erinxocon

This comment has been minimized.

Contributor

erinxocon commented Oct 29, 2017

Hi @nicholasbishop! Feature requests are certainly welcome. I don't think this is in the scope of pipenv. Those directions essentially just tell you to copy your site packages folder along with your code. Pip does nothing but install the code to the site packages folder. You can get the location of your virutalenv by doing (on unix) open (pipenv --venv) and then following those directions regarding copying the site package folder.

@erinxocon erinxocon closed this Oct 29, 2017

@erinxocon erinxocon added enhancement future and removed future labels Oct 29, 2017

@Zebradil

This comment has been minimized.

Zebradil commented Jan 8, 2018

I would vote for this feature. It is strange to have «check style» functionality and not to have some of the useful pip's functionality. I tried to migrate one of my AWS Lambda project to using pipenv, but it turned out that it's easier to use original venv functionality + pip + requirements.txt. Just because I can't specify exact directory of package installation with pipenv.

@uranusjr

This comment has been minimized.

Member

uranusjr commented Jan 8, 2018

@Zebradil If you mean pipenv check --style, that functionality is planned to be removed.

@joestump

This comment has been minimized.

joestump commented May 7, 2018

I found this example on GitHub that looks sensible for packaging Lambdas with pipenv. You could move those to a bundle "script" in the Pipfile and run pipenv run bundle.

@ashemedai

This comment has been minimized.

ashemedai commented Jun 5, 2018

@joestump Unfortunately that's not sensible (unless you work in an environment where there's a large homogeneity of systems). I've been through similar hoops in the last few days. You might encounter both lib/python3.6/site-packages and lib64/python3.6/site-packages that you need to add to the zip command. Then you also need to start adding checks to see whether or not the directories exist.

In the end the easiest remains, as @Zebradil pointed out:

build:
	mkdir -p $(BUILDDIR)
	$(pipenv) lock -r > $(BUILDDIR)/requirements.txt
	cp -R $(SRCDIR)/* $(BUILDDIR)
	$(pipenv) run pip install --isolated --disable-pip-version-check -r $(BUILDDIR)/requirements.txt -t $(BUILDDIR) -U

That way you can just point the CodeUri to build and use aws cloudformation package as it should.

This makes it rather platform-agnostic (had to get this working on a Linux, macOS, and Windows environment). Having to step out of pipenv with a run pip install feels weird. And pipenv --venv as @erinxocon mentioned does not help in any of these scenarios.

@ashemedai

This comment has been minimized.

ashemedai commented Jun 5, 2018

In addition, I do need to check that script section of Pipfile. That might actually allow me to keep some stuff local to Pipfile without having to bloat the Makefile with Python-specific things and allows us, where I work, to keep using our more standardised Lambda build pipeline (which supports C#, Go, Node.js, and Python).

@Zebradil

This comment has been minimized.

Zebradil commented Jun 5, 2018

At the end I got to the following command:

$ pipenv run pip install -r <(pipenv lock -r) --target _build/

Works with bash, doesn't work with pure shell.
But I see some sense in preserving requirements.txt in the bundle. Just as metadata of the package.

@ashemedai

This comment has been minimized.

ashemedai commented Jun 8, 2018

The problem was that if I want to try the redirection soltion in a Makefile the result became unreadable fast. And like you said, I figured that having the requirements.txt part of the zip in S3 would be an added benefit since, currently, we do not really do any release management on these functions. We always install the latest ones from the repository.

@joshcurago

This comment has been minimized.

joshcurago commented Sep 20, 2018

The problem was that if I want to try the redirection soltion in a Makefile the result became unreadable fast. And like you said, I figured that having the requirements.txt part of the zip in S3 would be an added benefit since, currently, we do not really do any release management on these functions. We always install the latest ones from the repository.

The easiest way around this is stuffing the requirements into a temp file. See below

$(eval TEMPFILE = $(shell mktemp))
pipenv lock -r > ${TEMPFILE}
pipenv run pip install -r ${TEMPFILE} --target _build/
rm -f ${TEMPFILE}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment