[BUG] /tmp/chromium: error while loading shared libraries: libnss3.so: cannot open shared object file: No such file or directory

[pranavburnwal]

I am using the AWS layer method.
Already checked all existing issues for similarity.
Please see the screenshot below for more debugging and details.
getAllFilesAndFolders gets all files and prints them in an array.

Environment

• chromium Version: 122.0.0
• puppeteer / puppeteer-core Version: 22.5.0
• Node.js Version: 18.x
• Lambda / GCF Runtime: nodejs18.x

FYI Previously has tried: chromium:118.0 / puppeteer:21.4 - Same issue
My codebase is serverless typescript, which gets compiled by webpack and deployed.

Expected Behavior

Code should just run.

Current Behavior

Steps to Reproduce

			console.log("AWS_EXECUTION_ENV: "+process.env.AWS_EXECUTION_ENV)
            console.log('defaultPuppeteerTest> before ')
            console.log(await getAllFilesAndFolders('/tmp'));
            console.log(await getAllFilesAndFolders('/opt/nodejs/node_modules/@sparticuz/chromium'));
            chromium.setHeadlessMode = true;
            chromium.setGraphicsMode = false
            const executablePath = config.ENVIRONMENT === 'dev'
                ? config.CHROMIUM
                : await chromium.executablePath("/opt/nodejs/node_modules/@sparticuz/chromium/bin");
            
            console.log('defaultPuppeteerTest> after executablePath')
            console.log(await getAllFilesAndFolders('/tmp'));
            console.log(await getAllFilesAndFolders('/opt/nodejs/node_modules/@sparticuz/chromium'));
            let browser = await puppeteer.launch({
                args: config.ENVIRONMENT === 'dev' ? puppeteer.defaultArgs() : ['--disable-extensions', ...chromium.args],
                executablePath: executablePath,
                defaultViewport: {
                    width: 3839,
                    height: 2160,
                    deviceScaleFactor: 2
                },
                env: {
                    TZ: "Asia/Singapore",
                },

                ignoreHTTPSErrors: true,
            });
            const page = await browser.newPage();
            await page.goto("https://example.com");
            const pageTitle = await page.title();
            await browser.close();

[Sparticuz]

aws.tar is old (<=120). I would bet your Lambda layer isn't 122 like you think it is. You should have al.tar.br and al2023.tar.br

[pranavburnwal]

FYI Previously has tried: chromium:118.0 / puppeteer:21.4 - Same issue

@Sparticuz Yes, the screenshot is for this (above) version I mentioned.

I had tried the same with the given version 122 as well. Here is the screenshot of the same code for reference:

Edit: formatting

[pranavburnwal]

Also tried Method 2 from https://stackoverflow.com/a/72485784
Manually adding PATH and LD_LIBRARY_PATH with /tmp extracted files
Added this in the provider in serverless:

  environment:
    PATH: /var/lang/bin:/usr/local/bin:/usr/bin/:/bin:/opt/bin:/tmp
    LD_LIBRARY_PATH: /var/lang/lib:/lib64:/usr/lib64:/var/runtime:/var/runtime/lib:/var/task:/var/task/lib:/opt/lib:/tmp/al2/lib

Didnt work.

[Sparticuz]

console.log your LD_LIBRARY_PATH, after calling executablePath. Make sure /tmp/al2/lib is in it. The package puts it before all the other entries.

[pranavburnwal]

console.log your LD_LIBRARY_PATH, after calling executablePath. Make sure /tmp/al2/lib is in it. The package puts it before all the other entries.

It's there, still throws the same error. Super weird.
@Sparticuz What else can I debug to check?

[dmarreco]

Exactly same issue here. Using Esbuild.

[Sparticuz]

Maybe try using chromium-min variant? Otherwise Amazon might have silently updated their images

[noppa]

We also started facing this today. Puppeteer crashes, and the Chromium logs contain

[ERROR:nss_util.cc(124)] Error initializing NSS without a persistent database: NSS error code: -8023

The breakage seems to come between lambda images nodejs:18.v26 and nodejs:18.v28.
Maybe they have upgraded or done something else to NSS that breaks it? They have a open issue about a NSS-related CVE.

[hudsontavares]

@noppa - yes, they updated Node JS version yesterday

[james-certn]

We noticed this as well, we had to turn off Auto mode for Lambda runtime upgrades and pin to nodejs:18.v26. I assume something changed with regards to the underlying Amazon Linux image that's used to build the nodejs runtime, like the shared library is renamed or not installed.

What is the realistic path forward, I'm curious?

[Sparticuz]

Should just need to boot up the corresponding image on ec2 and grab the updated libs

[noppa]

Rebuilding the layer with the latest version (123.0.1) of @sparticuz/chromium fixed the issue for us. We were using version 117 before.

Plus we had to re-enable setGraphicsMode due to #247, which seems to make things a bit slower. But oh well, at least it works.

[Sparticuz]

Rebuilding the layer with the latest version of this lib fixed the issue for us (we were using 117 before).

Thank you for checking! I will push a fix soon.

[Sparticuz]

Can someone try this? I've updated the libnss3.so and libsoftokn3.so files. Github won't allow .br attachements, so please REMOVE .zip from the file.

al2.tar.br.zip