The current Chromium Binary doesn't work for AWS nodejs10.x runtime #37
Comments
|
@nickychow Latest version (1.16.0) ships with Node 10.x support. |
|
Sorry for the false report, I've unpublished 1.16.0 for the time being, Will need some more time to properly compile libnss. |
|
(same problem as OP), but I also want to express my gratitude to the author for publishing the package and the very rapid response above. |
|
So just to give a status update on this, it seems that under the new execution environment the following shared objects are required - and they add about 530 KiB after Brotli compression:
Despite patching up the Lambda environment with these additional dependencies, I still haven't managed to be able to boot up Chromium correctly, it seems to be related to changes in the Fontconfig configuration: I'm recompiling Chromium with the EDIT: The |
|
So yeah, on Node 8.10 we have: $ lambdash 'echo $AWS_EXECUTION_ENV && ls /etc/fonts/fonts.conf'
AWS_Lambda_nodejs8.10
/etc/fonts/fonts.confOn Node 10.x however: The contents of And the contents of |
|
@nickychow From what I could assess, the new Lambda environment is way more lightweight than the 8.10 counterpart: not only it comes with the outdated libnss dependencies you already noticed, I also wasn't able to locate any Fontconfig configuration file, nor any single font available on the system - grepped for *.ttf, *.otf, ... So it seems like a proper packaging for Node 10 would be substantially larger:
Besides that, quite some patching to the environment/files would be required:
In light of all of this, I no longer think the advantages of upgrading to Node 10 outweigh the drawbacks. I'm uploading the necessary shared objects (they should be rather stable/future-proof) in case someone wants to create a Node 10 compatibility layer on top of this, including whatever font faces are appropriate. |
|
It looks like AWS is going to force the new, lightweight environment for all lambdas pretty soon: https://aws.amazon.com/blogs/compute/upcoming-updates-to-the-aws-lambda-execution-environment/ If I’m not mistaken, this will cause problems similar to the ones mentioned above even on Node 8 lambdas. Has anyone managed to make this work in the new environment? |
|
@bhasto pretty sure that the forced update is to 2018.03 (of amazon linux 1), which is an update for sure, but not one that should impact this. That's not to say that there wont be another update later in the year that forces linux 2 on all lambda environments. |
|
@bhasto I did opt in to the new execution environment on a Node 8 Lambda with Layer However I do also share the concern about a potential update to Linux 2 on all environments, but if that time comes I'll definitely be creating and maintaining support Layers for it. |
This probably doesn't help, but could someone explain why Chrome works on Amazon Linux 2 and Chromium doesn't? |
|
@SergeyAvd I don't see why it wouldn't work, as long as (some of) the dependencies here are installed: chrome-aws-lambda/_/ansible/plays/chromium.yml Lines 86 to 141 in 9fe9b70 |
I'm sure this is already known, but As for the font, you can set the This should restore the previous functionality you have for asynchronously loading fonts into I'm testing out preparing a chrome headless binary for AL2 in a different way than the approach in this repo (using amazonlinux docker image to build shared objects and non-chrome binaries), but came across this same error and resolved it by setting the font config path. Not sure yet if everything else is working but I plan on publishing my repo if it works out. |
|
So I also tried to use the binary bundled with this repo but it doesn't appear to look at the I'm able to get the latest RPM as well as chromium Linux_x64 working in both AL2 and AL2018 docker containers, but get other errors when I try to do the same in lambda. |
|
@jensbodal This binary needs to be compiled with I'm not sure is that skips the |
|
Including It looks like using |
|
@jensbodal I think it's looking under What I do with the |
|
Oh wow that was unfortunate, I did everything but forgot to copy my I have not yet tested in lambda but have it working with the AL2018 docker image. My And before calling my test I run |
|
@jensbodal Great! Let me know how it goes! |
|
Is this something we need to worry about since the General Update starts tomorrow (16th)? https://aws.amazon.com/blogs/compute/upcoming-updates-to-the-aws-lambda-execution-environment/ |
|
I don't think so. The General Update happening on July 16th changes Amazon Linux 2017.09 to Amazon Linux 2018.03. I've tested my use case on 2018.03 and
|
|
Ah! This makes more sense. Thanks for clarifying. |
|
I am unable to use the pre-built layer created through make with puppeteer-core directly. Is it possible to do so? Is it necessary to do all the inflate steps packaged in Chromum class before using this layer directly with puppeteer? |
|
I tried to deconstruct your packaging logic t assemble a path based layer. I am taking this approach rather than using pre-built layer because it helps me in some ways manage development, testing and production environments. Things are working perfectly if I use Chromium.puppeteer.launch(). My target is to directly be able to use puppeteer-core. ==== Chromium.executablePath() , inflates (brotli) and creates
It returns /tmp/chromium as the executablePath. Which is used in puppeteer-core options to instantiate a headless chromium. With this understanding I tried to package the layer into a /layer folder. This was a wrong assumption. So I tried to deflate both tar files. into the /layer/lib folder. This gave all packaged *.so files into LD_LIBRARY_PATH , along with inflated chromium as executable. This approach produced another error. Error: Failed to launch chrome! I can see that libnss3.so is correctly getting used. However something is not suiting with /lib64/libnssutil3.so Any ideas? |
|
@devasur What I found out is that the order of the paths in environment variables matters: if (process.env.AWS_EXECUTION_ENV === 'AWS_Lambda_nodejs10.x') {
if (process.env.FONTCONFIG_PATH === undefined) {
process.env.FONTCONFIG_PATH = '/opt/lib';
}
if (process.env.LD_LIBRARY_PATH.startsWith('/opt/lib:') !== true) {
process.env.LD_LIBRARY_PATH = [...new Set(['/opt/lib', ...process.env.LD_LIBRARY_PATH.split(':')])].join(':');
}
}Try to look into version You should also move Chromium (and SwiftShader) into PS: Not sure it's clear but you need to inflate and untar all the files. |
I had the same issue when using the module in a layer with Node10.x runtime, but after I updated the layer to 1.20.3, the issue has gone and now everything is working fine! Thank you! |
|
I see. I changed the order. Don't know whether changing it post instantiation has any impact on an executing lambda. I am getting a socket hangup error now with rejigged layer. // browser = await chromium.puppeteer.launch({
// args: chromium.args,
// defaultViewport: chromium.defaultViewport,
// executablePath: await chromium.executablePath,
// headless: chromium.headless,
// });
browser = await puppeteer.launch({
args: ['--no-sandbox', '--disable-gpu', '--single-process'],
headless:true,
executablePath:executablePath
})2019-10-21T09:39:15.294Z 52c9f3fc-b4f8-4064-822c-3c71ecf64940 INFO LD_LIBRARY_PATH /opt/lib:/var/lang/lib:/lib64:/usr/lib64:/var/runtime:/var/runtime/lib:/var/task:/var/task/lib |
|
@alixaxel it's weird but I found solution for problem. to: |
|
@exentrich Are you sure? 🤔 It doesn't make much sense... |
|
@alixaxel : You are right. I realized I was just making my life miserable by trying to use puppeteer directly on AWS/Node10. Clearly you have done everything that was required to achieve everything I was trying to achieve :) - light packaging, layer usage etc. I have put together a small example of using your layer in a lambda here. |
|
I have the same issue as the OP, and the 1.20.3 version saved me! Thanks to the author for his great work! |
|
I got same error attaching this Lambda Layer I just followed how to create Lambda Layer manually and attached this Lambda Layer to my lambda function, it works well... my my service: chromium-layer # NOTE: update this with your service name
provider:
name: aws
runtime: nodejs10.x
region: ap-northeast-2
stage: ${opt:stage, 'dev'}
profile: !$!@!@#!@
environment:
NODE_PATH: "/opt/node_modules:/opt:/"
custom:
serverless-offline:
port: 6000
# on AWS environment: cd /opt && npm ls -> npm ERR! missing: ms@2.1.1, required by layer_packages@1.0.0
# https://github.com/serverless/serverless/issues/3915
package:
excludeDevDependencies: false
layers:
ChromiumPackage:
path: ./ # required, path to layer contents on disk
name: Layer-${self:provider.stage}-ChromiumPackage # optional, Deployed Lambda layer name
description: #$!@!@#!@#
compatibleRuntimes:
- nodejs10.x
retain: true |
|
What worked for me was to create layer using the artifact method. Path method did not work correctly for the pre-built zip created by make. You may check this example for a working project that uses this layer. |
|
I'm experiencing similar issue on AWS lambda with What's preferred: open a new bug report or post details here? Update: It turns out that the issue above was with the first invocation of the Lambda timing out. Looking at the code, I believe what was happening was that the binary inflation would be left incomplete after the first time the Lambda was invoked if it timed out. Subsequent calls would not try to inflate it again. Just by making sure that the first invocation does not timeout, it seems to have fixed the issue. For reference, I am using Node 10.x runtime environment. |
|
This is what I'm getting with latest version: UPDATE: it does work when I'm trying to use node10.x, but can't make it work with node12.x |
I had the same issue using Lambda with Nodejs12.x and chrome-aws-lambda version |
|
@hasansaghir Node 12.x support is only implemented in version 2.0.1 onwards. |
Trying fix from alixaxel/chrome-aws-lambda#37
|
version 6.0.0 supports Nodejs 14 |
and others
First of all, a big thank you for your such amazing work for this project.
We've been using the Chromium Binary file from this project as AWS Lambda Layer for a while. It works flawlessly on runtime 8.10. However, since AWS just announced supports for Nodejs10.x. We didn't hesitate to upgrade our lambda functions and it ended up breaking our builds.
The error we are getting from local tests is
error while loading shared libraries: libnss3.so: cannot open shared object file: No such file or directory. And I also noticed that AWS changed to use Amazon Linux 2 instead of Amazon Linux as Nodejs 10 Lambda Operation System. That might change a lot of shared libs or dependencies. Please find more information with the link below:https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html
I just hope that this project can add supports for AWS nodejs10.x runtime in the near future.
Cheers,
Nick
The text was updated successfully, but these errors were encountered: