Not able to push new lambda function changes via amplify push. Error: Could not find a pipenv site-packages directory

[heng2j]

Note: If your issue/bug is regarding the AWS Amplify Console service, please log it in the
Amplify Console GitHub Issue Tracker

Describe the bug
A clear and concise description of what the bug is.

I am not able to do amplify push after making any changes to one of my lambda function. For example simply remove a single space.

I am receiving this error:

nstalling dependencies from Pipfile.lock (a65489)...
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
(node:52398) UnhandledPromiseRejectionWarning: Error: Could not find a pipenv site-packages directory at /Users/heng2j/.virtualenvs/collectorVideosProcessor-r9qVMPot/lib/python3.8/site-packages
    at Object.getPipenvDir (/usr/local/lib/node_modules/@aws-amplify/cli/node_modules/amplify-python-function-runtime-provider/src/util/pyUtils.ts:24:9)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
    at /usr/local/lib/node_modules/@aws-amplify/cli/node_modules/amplify-python-function-runtime-provider/src/util/packageUtils.ts:27:21
(node:52398) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:52398) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Amplify CLI Version
You can use amplify -v to check the amplify cli version on your system

4.39.0

To Reproduce
Steps to reproduce the behavior or terminal output if applicable

I am putting my python dependencies in another lambda layer for that lambda function at opt/python

And I am using runtime Python 3.7 instead. But I have tried to run Python 3.8

Expected behavior
A clear and concise description of what you expected to happen.

Successful push changes to the lambda function.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: Mac
• Node Version. You can use node -v to check the node version on your system
  v12.13.0

Additional context
Add any other context about the problem here.

[half2me]

@heng2j I'm having the exact same issue. Have you found a workaround yet?

[akshbhu]

Hi @half2me , @heng2j

It looks like pipenv is missing. I have a following questions:

1. Are you able to push functions before this error(selecting python as runtime)?

2. Can you make sure you have pipenv installed and included in your PATH variable(make sure python verison > 3.8)?

[half2me]

@akshbhu

1. I got this error the very first time I tried to push my python function.
2. I have it installed:

➜  ~ python3 --version
Python 3.9.1
➜  ~ pipenv --version
pipenv, version 2020.11.15

[akshbhu]

Can you try this:

1. Install pipenv using pip install --user pipenv.This does a user installation to prevent breaking any system-wide packages.
2. Run python -m site --user-base to get your user base binary directory, append bin to the path and add this path to your $PATH variable. You can echo $PATH if its added or not.
3. Try amplify push again.

Let me know if this works for you?

[heng2j]

For me, I checked that I only have python3.7 in my lambda function's virtualenv. So I have to renamed it manually to 3.8 to fix this issue.

/Users/heng2j/.virtualenvs/collectorVideosProcessor-r9qVMPot/lib/python3.8/site-packages

[kaustavghosh06]

@heng2j @half2me Are you'll unblocked on this issue?

[jhockett]

Closing due to lack of response.

[rt-demetri]

amplify pull did the trick for me. It must have recreated what was missing (obviously I already had python and pipenv). Think of it this way ... if you were to pick up on a brand new machine, you wouldn't have any of those files under ~/.local/share/virtualenvs or ~/.virtualenvs. That's technically what happened to me (migrated Amplify project to M1 Mac).

UPDATE: Scratch that. It appears I would have to run pull before push every time. It's no fix. I'll tell you what seems to reliably do the trick:

pipenv run amplify push

[bjm88]

thanks @rt-demetri this worked for me

pipenv run amplify push

[vincent38wargnier]

I had the same problem.
For me, the solution to solve it was to call "python3 -m site --user-base" as @akshbhu said. And then go inside the folder created, and copy the site-package folder into the path given by the pipenv error.

I needed this for an amplify lambda, the default path is not the path used by my lambda.

[bfleming-ciena]

I'm having a similar problem. When I add a new lambda function I have to cd into the folder and run pipenv shell, then amplify push works.

So I have to have a separate pipenv virtual environment for EVERY lambda function? Where are the docs on how to properly use pipenv and add a python lambda function with amplify. I should just be able to follow a doc and it should work.

[michaelbrewer]

But default i still get this issue and i can't do a amplify function build. Amplify CLI should fix this out of the box.

[michaelbrewer]

@bfleming-ciena i agree this is not resolved
@jhockett can you reopen this

[esotericzhang]

Also seeing this issue. Anyone find a workaround yet? Neither @rt-demetri or @bfleming-ciena solutions worked for me.

[ousjaguraga]

I had the same problem.
For me, the solution to solve it was to call "python3 -m site --user-base" as @akshbhu said. And then go inside the folder created, and copy the site-package folder into the path given by the pipenv error.

I needed this for an amplify lambda, the default path is not the path used by my lambda.

This worked for me, I don't normally reply lol. Thank you and @akshbhu.

[michaelbrewer]

Maybe @heitorlessa can help with a suggestion on how to fix this in the Amplify cli?

[viaSSH]

For me, I checked that I only have python3.7 in my lambda function's virtualenv. So I have to renamed it manually to 3.8 to fix this issue.

/Users/heng2j/.virtualenvs/collectorVideosProcessor-r9qVMPot/lib/python3.8/site-packages

I have also encountered this problem.
In my case, pipenv command makes ~/.local/share/virtualenvs/myPythonAmplify/lib/python3.8
But, I'm using python3.9
So, I change directory name from python3.8 to python3.9

I don't know why this works, but the problem is solved.
Now I can amplify push !!!

[dcchristopher]

For me, I checked that I only have python3.7 in my lambda function's virtualenv. So I have to renamed it manually to 3.8 to fix this issue.
/Users/heng2j/.virtualenvs/collectorVideosProcessor-r9qVMPot/lib/python3.8/site-packages

I have also encountered this problem.
In my case, pipenv command makes ~/.local/share/virtualenvs/myPythonAmplify/lib/python3.8
But, I'm using python3.9
So, I change directory name from python3.8 to python3.9

I don't know why this works, but the problem is solved.
Now I can amplify push !!!

Funny thing you should mention that. I think each first time you run Python in an amplify project, it leaves behind version information in a pipfile or something like that. Subsequent runs with a different version of Python will see the different version listed and you run into situations such as this issue. Another coworker unknowingly updated the pipfile (by means of running a multitude of different pip/pipenv commands) and suddenly I’m not getting the error anymore. Next time I’m in that code, I’ll take a look at the commit.

[viaSSH]

That's right. Changing "python3.8 to python3.9 version" Methods was only possible once....
After that, I use pipenv run amplify push It's Works!!

Maybe my Python version and pipfile have compatibility confilcts.

[magicalne]

Same problem just resolved.

There is a mistake in the project in my case. My college setup python 3.8 in the functions, but use 3.9 in the Pipefile under the root of project.

So pipenv kept looking for site-packages under python-3.9 instead of python-3.8.

After set python 3.8 in the pipefile, I followed what akshbhu said:

• pip install --user pipenv
• export PATH="$HOME/.cargo/bin:$HOME/.local/bin:$PATH"
• amplify push

It works now.

[kaustavghosh06]

Closing this issue.
Looks like moving from python3.8 to python3.9 version fixes this issue.

[michaelbrewer]

Maybe we can clarify how this should work in the AWS Amplify docs?

[KKodiac]

I personally had my machine have two versions of python3 (3.8 and 3.9).
This apparently caused an issue where my Pipfile indicated my version of python3 as 3.9
where amplify used the 3.8 version.

Resolved my issue by specifying python version as 3.8 in Pipfile and Pipfile.lock then ran
pipenv run amplify push

[github-actions]

This issue has been automatically locked since there hasn't been any recent activity after it was closed. Please open a new issue for related bugs.

Looking for a help forum? We recommend joining the Amplify Community Discord server *-help channels for those types of questions.