Issue with libpython2.7.so.1.0 when running on nodejs v10 runtime

[renatoargh]

Hello,

I am getting an error with the following code:

const shell = require('child_process').execSync

module.exports.handler = () => {
    const version = shell('/opt/awscli/aws --version')
    console.log(version)
}

The error is as following: Command failed: /opt/awscli/aws --version\n/opt/awscli/bin/python: error while loading shared libraries: libpython2.7.so.1.0: cannot open shared object file: No such file or directory\n

OBS: I got the layer by clicking this button:

What might have been wrong with my deployment?! Thank a lot for your work :)

[renatoargh]

PS.: It works well with v8

[pahud]

The aws-lambda-layer-awscli comes with /opt/awscli/bin/python which requires /lib64/libpython2.7.so.1.0, which unfortunately is not provided in nodejs10 lambda runtime.

bash-4.2# ldd /opt/awscli/bin/python
        linux-vdso.so.1 (0x00007ffd36dab000)
        libpython2.7.so.1.0 => /lib64/libpython2.7.so.1.0 (0x00007f1938d43000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f1938b25000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007f1938921000)
        libutil.so.1 => /lib64/libutil.so.1 (0x00007f193871e000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f19383d3000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f193801d000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f1939120000)

However, /lib64/libpython2.7.so.1.0 is provided in the v8 runtime.

[renatoargh]

Thanks @pahud
Do we plan to support it in the future?! Maybe by packaging some of these shared libraries?
My understanding is this regard is limited, otherwise, I could come up with a PR.

[pahud]

@renatoargh Sure. Feel free to submit your PR.

[jeromeheissler]

There is a PR somewhere ? Any idea to temporary solve this issue ?

[flesherj]

@pahud Has this issue been resolved? Is there a PR open for it?

[pahud]

Hi,

I will leave this issue open and any PR would be appreciated.

[pahud]

Just out of curiosity, as aws-lambda-layer-awscli is originally designed for lambda custom runtime with bash, is there any reason or scenario the nodejs runtime will use the aws CLI rather than the aws javascript sdk?

[flesherj]

We have one use case that leverages the aws s3 sync to copy the content of a directory to an s3 bucket. Apparently manually iterating over the directory contents and using the sdk to upload each file was too slow.

…

On Sat, Dec 28, 2019 at 18:12 Pahud Hsieh ***@***.***> wrote: Just out of curiosity, as aws-lambda-layer-awscli is originally designed for lambda custom runtime with bash, is there any reason or scenario the nodejs runtime will use the aws CLI rather than the aws javascript sdk? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#5?email_source=notifications&email_token=ABDI6ZSXAN5EROTUZNCD2SLQ27MM3A5CNFSM4IU6T7Z2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHYT4RA#issuecomment-569458244>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABDI6ZWTUL6Y34GOVCZUO7TQ27MM3ANCNFSM4IU6T7ZQ> .

[renatoargh]

My use case here is exactly the same as @flesherj

[joshua-at-aws]

Hi,

This is expected behavior for the newer Lambda runtimes which use an Amazon Linux 2 base, which include only a very minimal set of shared libraries.

For the short term, you may want to try using one of the non-deprecated runtimes which use the older Amazon Linux 2018.03 base.

For example, with python3.6 here is a trivial example:

def handler(event, context):
  my_output = subprocess.check_output(["/opt/awscli/aws","--version"], stderr=subprocess.STDOUT, shell=False)
  print(my_output.decode('utf8'))

However, in the long term I also recommend using your own custom runtime.

[Amri91]

Hi, I have the same issue. I am using this Layer https://github.com/aws-samples/aws-lambda-layer-kubectl but both use AWSCLI. Perhaps the same fix can be applied to both.

I am using AWSCLI to call EKS update-config command as well as S3 sync commands. I am using Javascript for some logic.

Just an FYI: https://docs.aws.amazon.com/lambda/latest/dg/runtime-support-policy.html
Node 8 has reached EOL.

[pahud]

Hi

Thanks for the feedback. I've successfully built an awscli layer that supports node12 lambda runtime

I will fork a new branch and share new assets in the next few hours.

[pahud]

please check this new README section in the new node12-runtime-support branch

https://github.com/aws-samples/aws-lambda-layer-awscli/tree/node12-runtime-support#node10-and-node12-aws-lambda-runtime-support

Let me know whether it works before we merge it into master.

Thanks.

[pahud]

I was working on AWS CLI v2 with python3 support for Lambda python3.8 runtime at #10 (comment)

And it seems to work with nodejs 12.x runtime as well

I am considering to start a new v2 branch to support AWS CLI v2 for all lambda runtimes.

[logicalicy]

please check this new README section in the new node12-runtime-support branch
https://github.com/aws-samples/aws-lambda-layer-awscli/tree/node12-runtime-support#node10-and-node12-aws-lambda-runtime-support
Let me know whether it works ...

Thanks for your node12-runtime-support branch! I've followed the steps and attempted to use the layer for my Node 12 runtime. But found that the layer unzips to 98,588,610 bytes, so unfortunately my use case happens to go just over the limit Edit: managed to reduce my package size. Does the size sound right to you? (Thought to double-check in case this is unexpected.)

Edit: Ran the two make commands and added the env vars — works great!

[pahud]

@logicalicy

Sounds like it works in your environemt. :-)

I just ran the make layer-build to generate the layer.zip. The zip file is about 12MB while the unzipped awscli dir is about 64MB.

You can customize your Dockerfile to generate the final layer.zip file.

[vhadianto]

@pahud thanks for this, layer-build-python27 works well in Node 12 runtime. Will be great if we can have this in master.

[pahud]

@vhadianto merged :-)

[vhadianto]

@vhadianto merged :-)

Awesome, thank you very much :)

[wlarch]

Wonderful thanks! Works as expected.

[Venus713]

Hi @wlarch
Would you give me the example that works fine on your side?
I am trying to execute awscli within lambda, but still failed.
So I need your help.
Thanks

[wlarch]

@Venus713

1. Clone the repository (https://github.com/aws-samples/aws-lambda-layer-awscli/) and checkout the node12-runtime-support branch
2. Edit the Makefile to have the adequate configuration
3. Execute make build will build your layer in a Docker container and zip its content. Building the layer consists of following the steps of executing commands in the Dockerfile.python27 file.
4. Once the build is complete you can execute make layer-upload and then make layer-publish in order to upload the zipped layer to an S3 Bucket and then publish the layer in your AWS Lambda custom layers.
5. Configure the Lambda function to use your custom layer either by command line on directly in the AWS Lambda Console.

You can then execute /opt/awscli/aws --version in your nodejs12.x Lambda runtime using Nodejs child_process.

I will probably write a fully explained article shortly. I found that a lot of searching and experimentation is needed to understand this examples repository.

[Venus713]

@wlarch
Thanks, but I am still getting same issue.
If you are available, then would you guide me on my machine via TeamViewer?

[Venus713]

Hi @wlarch @pahud @vhadianto
Would you please review my issue?

I cloned the https://github.com/aws-samples/aws-lambda-layer-awscli/ and then execute make build command and S3BUCKET="awscli-layers-dev-serverlessdeploymentbucket-y5hlybvmyu8s" LAMBDA_REGION="us-east-1" make sam-layer-package sam-layer-deploy command. So I've got following result.

aws --region us-east-1 cloudformation describe-stacks --stack-name "awscli-layer-stack" --query 'Stacks[0].Outputs'
[
    {
        "OutputKey": "LayerVersionArn",
        "OutputValue": "arn:aws:lambda:us-east-1:412511290092:layer:awscli-layer-stack:3",
        "Description": "ARN for the published Layer version",
        "ExportName": "LayerVersionArn-awscli-layer-stack"
    }
]
[OK] Layer version deployed.

And then I did execute a lambda function in nodejs12.x version.

const shell = require('child_process').execSync

exports.handler = () => {
    const version = shell('/opt/awscli/aws --version')
    console.log(version)
}

In result, It gives me following error.

{
  "errorType": "Error",
  "errorMessage": "Command failed: /opt/awscli/aws --version\n/opt/awscli/bin/python: error while loading shared libraries: libpython2.7.so.1.0: cannot open shared object file: No such file or directory\n",
  "trace": [
    "Error: Command failed: /opt/awscli/aws --version",
    "/opt/awscli/bin/python: error while loading shared libraries: libpython2.7.so.1.0: cannot open shared object file: No such file or directory",
    "",
    "    at checkExecSyncError (child_process.js:630:11)",
    "    at execSync (child_process.js:666:15)",
    "    at Runtime.exports.handler (/var/task/index.js:4:21)",
    "    at Runtime.handleOnce (/var/runtime/Runtime.js:66:25)"
  ]
}

Request ID:
"bcadd745-01ba-43e9-b55d-2baac1da3cbd"
...

Do you have any suggestion for me?

[Venus713]

Hi @logicalicy
Did you read my comment?
Do you have any suggestion?

[wlarch]

@Venus713 I encountered the same problem before. Did you git checkout node12-runtime-support branch before building the layer and publishing it ?

[Venus713]

@wlarch
Yes, I did the git checkout node12-runtime-support as you said me.
Then I followed the ReadMe.

[vhadianto]

Yes, I did the git checkout node12-runtime-support as you said me.
Then I followed the ReadMe.

@Venus713 have you ensured that your Lambda function uses the layer? Check in AWS Console.

[ja-agustin]

This also happens with dotnetcore3.1. any suggestions?