Bug: zlib data error on Fedora Linux when running NodejsNpmBuilder:NpmPack

[SingingBush]

Description:

When running sam build --template-file template.yaml in my project I get the following error

$ sam build --template-file template.yaml
Building codeuri: /home/username/project/src runtime: nodejs16.x metadata: {} architecture: x86_64 functions: ['ChimeSdkJoinLambda', 'ChimeSdkGetAttendee', 'GameInstanceQueueConsumerFunction', 'OnConnectFunction', 'AuthorizeFunction', 'OnDisconnectFunction', 'SendMessageFunction', 'OnMessageFunction', 'ChimeEventBridgeLambda']
Running NodejsNpmBuilder:NpmPack

Build Failed
Error: NodejsNpmBuilder:NpmPack - NPM Failed: npm ERR! code Z_DATA_ERROR
npm ERR! errno -3
npm ERR! zlib: invalid distance too far back

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/username/.npm/_logs/2022-06-13T13_22_33_417Z-debug-0.log
error Command failed with exit code 1.

Steps to reproduce:

Install aws-sam-cli-linux on Fedora 36 (or Arch Linux) then try to use it

Observed result:

Traceback (most recent call last):
  File "aws_lambda_builders/workflows/nodejs_npm/actions.py", line 59, in execute
  File "aws_lambda_builders/workflows/nodejs_npm/npm.py", line 85, in run
aws_lambda_builders.workflows.nodejs_npm.npm.NpmExecutionError: NPM Failed: npm ERR! code Z_DATA_ERROR
npm ERR! errno -3
npm ERR! zlib: invalid distance too far back

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

1. OS: Fedora Linux 36 (64-bit)
2. sam --version: SAM CLI, version 1.52.0
3. AWS region: eu-west-1

Add --debug flag to command you are running

[qingchm]

Thanks for reporting the issue! I've looked at a few similar issues online, my recommendation would be to try upgrading zlib and node version, would that be feasible for you? Let me know if this helps!

[SingingBush]

I found similar problems online before reporting and they generally had significantly older versions of node. The closest I've seen on stackoverflow is Sam build fails with: Build Failed Error: NodejsNpmBuilder:NpmPack - NPM Failed: npm ERR! code Z_DATA_ERROR.

My system is as up to date as it can be using the standard Fedora repos and I tried multiple nodejs versions that are available in Fedora 36 (from official fedora repo). Fedora defaults to the LTS nodejs releases so I'd expect to be able to use sam cli with nodejs 16.

Fedora currently has zlib 1.2.11 in the repository. Fedora has a bugzilla issue for updating to 1.2.12. I presume it's a similar situation for Arch.

I suspect that the zlib update to 1.2.12 maybe what I need but this project should really work with the LTS nodejs releases and zlib versions that are currently distributed with mainstream Linux distributions. If sam cli is using a newer version than what's generally available to users of current Linux distributions such as Fedora and Arch then that in itself should be considered a problem.

[mndeveci]

This error is coming from one of the steps in lambda builders, and in that step, we are running following npm command;

npm pack -q {package.json}

So if you try to run same npm command you should see same error message happening as well. I would recommend to check if there are any issues reported with npm cli on your operating system version/architecture.

Please let us know if you have further questions.

[torresxb1]

Closing as there hasn't been more information provided. Please reopen if you have further questions.

[marioavs]

The error is not present when running npm command, it generates a .tgz file:

npm pack -q file:/home/pathtotestproject/sam-app

The zlib version is already updated in Fedora 37, zlib-1.2.12

Do you think there is another way to isolate the way sam is invoking npm in order to report it somewhere else?

This error is coming from one of the steps in lambda builders, and in that step, we are running following npm command;

npm pack -q {package.json}

So if you try to run same npm command you should see same error message happening as well. I would recommend to check if there are any issues reported with npm cli on your operating system version/architecture.

[marioavs]

I had to change python version to 3.9 to be able to build aws-sam-cli from source, with make pr, and now the build command is working, no more "zlib: invalid distance too far back":

samdev build --template template.yaml

Something in the official release is affecting npm invocation.

[williamhaley]

Seeing the same behavior. Commenting here as the issue tracker says to do that rather than open a new issue.

I've been using sam build steadily to develop a sample program over the past couple weeks, and in the past two days or so it's been generating errors. See attached log with all stdout and stderr generated with --debug.

out.log

This seems most relevant

2022-12-20 21:07:17,697 | NODEJS installing in: /tmp/tmpbvf3mcgl
2022-12-20 21:07:17,697 | executing NPM: ['npm', 'install', '-q', '--no-audit', '--no-save', '--unsafe-perm', '--production']
2022-12-20 21:07:20,041 | NodejsNpmEsbuildBuilder:NpmInstall failed
Traceback (most recent call last):
  File "aws_lambda_builders/workflows/nodejs_npm/actions.py", line 111, in execute
  File "aws_lambda_builders/workflows/nodejs_npm/npm.py", line 85, in run
aws_lambda_builders.workflows.nodejs_npm.npm.NpmExecutionError: NPM Failed: npm WARN config production Use `--omit=dev` instead.
npm WARN tar zlib: invalid distance too far back
npm WARN tarball cached data for uuid@https://registry.npmjs.org/uuid/-/uuid-8.3.2.tgz (sha512-+NYs2QeMWy+GWFOEm9xnn6HCDp0l7QBD7ml8zLUmJ+93Q5NF0NocErnwkTkXVFNiX3/fpC6afS8Dhb/gz7R7eg==) seems to be corrupted. Refreshing cache.
npm WARN tar zlib: invalid distance too far back
npm WARN tarball cached data for strnum@https://registry.npmjs.org/strnum/-/strnum-1.0.5.tgz (sha512-J8bbNyKKXl5qYcR36TIO8W3mVGVHrmmxsd5PAItGkmyzwJvybiw2IVq5nqd0i4LSNSkB/sx9VHllbfFdr9k1JA==) seems to be corrupted. Refreshing cache.
npm WARN tar zlib: invalid distance too far back
npm WARN tarball cached data for tslib@https://registry.npmjs.org/tslib/-/tslib-1.14.1.tgz (sha512-Xni35NKzjgMrwevysHTCArtLDpPvye8zV/0E4EyYn43P7/7qvQwPh9BGkHewbMulVntbigmcT7rdX3BNo9wRJg==) seems to be corrupted. Refreshing cache.
npm WARN tar zlib: invalid distance too far back
npm WARN tarball cached data for fast-xml-parser@https://registry.npmjs.org/fast-xml-parser/-/fast-xml-parser-4.0.11.tgz (sha512-4aUg3aNRR/WjQAcpceODG1C3x3lFANXRo8+1biqfieHmg9pyMt7qB4lQV/Ta6sJCTbA5vfD8fnA8S54JATiFUA==) seems to be corrupted. Refreshing cache.
npm WARN tar zlib: invalid distance too far back

Linux archlinux 6.0.12-zen1-1-zen #1 ZEN SMP PREEMPT_DYNAMIC Thu, 08 Dec 2022 11:03:40 +0000 x86_64 GNU/Linux
Python 3.10.8
Node.js v18.12.1
npm 8.19.2
SAM CLI, version 1.67.0

Manually invoking the npm command seems fine.

$ npm install -q --no-audit --no-save --unsafe-perm --production
npm WARN config production Use `--omit=dev` instead.

up to date in 655ms

1 package is looking for funding
  run `npm fund` for details

Based on the error it seems like maybe the npm warning is a problem for the sam build process possibly.

• I tried downgrading Python as suggested in another comment, but that did not help. Neither did sam build --template template.yaml explicitly specifying the template file.
• I tried disabling nodenv (which I use for node management locally) and using system node (19.x), but that made no difference.
• I tried downgrading to node 17.9.1, but that did not help.
• I tried deleting node_modules and re-installing, but that did not help

sam build -u did work for me, which builds using a container rather than the host system.

[Baldomo]

This issue should be reopened. I get almost the same output as @williamhaley above (thank you for the detailed output which I could easily compare to mine).

My system:

Linux zephyrus 6.3.5-arch1-2-g14 #1 SMP PREEMPT_DYNAMIC Mon, 05 Jun 2023 10:18:49 +0000 x86_64 GNU/Linux
Python 3.11.3
Node v20.3.0
npm 8.19.2
SAM CLI, version 1.86.1
core/zlib 1:1.2.13-2

I also tried downgrading everything in various combinations but whit no success. I also cannot use a container with sam build -u because of the complexity of the project, which contains symlinks and shared code and layers.

This the log of a run with sam build --debug in the default SAM TypeScript template:

log.txt

A section view of the error:

Error: NodejsNpmBuilder:NpmInstall - NPM Failed: npm WARN config production Use `--omit=dev` instead.
npm WARN tar zlib: invalid distance too far back
npm WARN tarball cached data for esbuild@https://registry.npmjs.org/esbuild/-/esbuild-0.14.54.tgz (sha512-Cy9llcy8DvET5uznocPyqL3BFRrFXSVqbgpMJ9Wz8oVjZlh/zUSNbPRbov0VX7VxN2JH1Oa0uNxZ7eLRb62pJA==) seems to be corrupted. Refreshing cache.
npm WARN tarball tarball data for esbuild@https://registry.npmjs.org/esbuild/-/esbuild-0.14.54.tgz (sha512-Cy9llcy8DvET5uznocPyqL3BFRrFXSVqbgpMJ9Wz8oVjZlh/zUSNbPRbov0VX7VxN2JH1Oa0uNxZ7eLRb62pJA==) seems to be corrupted. Trying again.
npm WARN tarball tarball data for esbuild-linux-64@https://registry.npmjs.org/esbuild-linux-64/-/esbuild-linux-64-0.14.54.tgz (sha512-EgjAgH5HwTbtNsTqQOXWApBaPVdDn7XcK+/PtJwZLT1UmpLoznPd8c5CxqsH2dQK3j05YsB3L17T8vE7cp4cCg==) seems to be corrupted. Trying again.
npm WARN tar zlib: invalid distance too far back
npm WARN tarball tarball data for esbuild-linux-64@https://registry.npmjs.org/esbuild-linux-64/-/esbuild-linux-64-0.14.54.tgz (sha512-EgjAgH5HwTbtNsTqQOXWApBaPVdDn7XcK+/PtJwZLT1UmpLoznPd8c5CxqsH2dQK3j05YsB3L17T8vE7cp4cCg==) seems to be corrupted. Trying again.
npm WARN tar zlib: invalid distance too far back
npm WARN tarball tarball data for esbuild@https://registry.npmjs.org/esbuild/-/esbuild-0.14.54.tgz (sha512-Cy9llcy8DvET5uznocPyqL3BFRrFXSVqbgpMJ9Wz8oVjZlh/zUSNbPRbov0VX7VxN2JH1Oa0uNxZ7eLRb62pJA==) seems to be corrupted. Trying again.
npm ERR! code Z_DATA_ERROR
npm ERR! errno -3
npm ERR! zlib: invalid distance too far back

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/leo/.npm/_logs/2023-06-13T14_28_15_059Z-debug-0.log

[mndeveci]

Hi there,

Re-opening this issue to continue the discussion. Thanks for providing detailed outputs here. But it seems like the error is generated from NPM. I did a quick google search and it might be that your system's zlib might not be compatible with NPM or NodeJS version

• https://stackoverflow.com/questions/60288671/npm-install-on-synology-gives-zlib-invalid-distance-error
• invalid distance too far back nodejs/node#22839

Can you try searching NodeJS or NPM changelog's with your OS and see if you need to update zlib there?

Thanks!

[mndeveci]

Closing issue due to inactivity.

As I mentioned previously, it seems like the issue is coming from the operating system and not related to SAM CLI.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.