ZlibError when calling publish #2278

iamdriz · 2019-09-18T15:57:45Z

When we're trying to publish our packages we are getting the error:

ZlibError: zlib: Cannot read property 'length' of null

Expected Behavior

It should publish the packages with no errors.

Current Behavior

It updates all the packages.json versions but DOES NOT publish them to the registry.

Possible Solution

Unknown?

Steps to Reproduce (for bugs)

Call npx lerna publish

Your Environment

Mac OS El Capitan 10.11.5
NPM version: 5.6.0
Node version: 9.2.1
Lerna version: 3.16.4

This is the full output of the log when we call npx lerna publish --loglevel verbose

lerna info execute Skipping releases
lerna verb git [ 'commit',
lerna verb git   '-F',
lerna verb git   '/private/var/folders/r_/t905430x1zvd1sgymm76t6p51b940m/T/bb79cbbe-dbe2-4a40-bbf6-8368b1b00437/lerna-commit.txt' ]
lerna verb git [ 'tag', '@fui/fui-button@1.3.0', '-m', '@fui/fui-button@1.3.0' ]
lerna verb git [ 'tag',
lerna verb git   '@fui/fui-buttons@1.3.0',
lerna verb git   '-m',
lerna verb git   '@fui/fui-buttons@1.3.0' ]
lerna verb git [ 'tag',
lerna verb git   '@fui/fui-checkbox@1.3.0',
lerna verb git   '-m',
lerna verb git   '@fui/fui-checkbox@1.3.0' ]
lerna verb git [ 'tag', '@fui/fui-form@1.3.0', '-m', '@fui/fui-form@1.3.0' ]
lerna verb git [ 'tag', '@fui/fui-forms@1.3.0', '-m', '@fui/fui-forms@1.3.0' ]
lerna verb git [ 'tag', '@fui/fui-radio@1.3.0', '-m', '@fui/fui-radio@1.3.0' ]
lerna verb git [ 'tag', '@fui/fui-select@1.3.0', '-m', '@fui/fui-select@1.3.0' ]
lerna verb git [ 'tag', '@fui/fui-switch@1.3.0', '-m', '@fui/fui-switch@1.3.0' ]
lerna verb git [ 'tag',
lerna verb git   '@fui/fui-text-field@1.3.0',
lerna verb git   '-m',
lerna verb git   '@fui/fui-text-field@1.3.0' ]
lerna verb git [ 'tag', '@fui/fui-core@4.4.0', '-m', '@fui/fui-core@4.4.0' ]
lerna verb git [ 'tag', '@fui/fui@4.4.0', '-m', '@fui/fui@4.4.0' ]
lerna info git Pushing tags...
lerna info publish Publishing packages to npm...
lerna notice Skipping all user and access validation due to third-party registry
lerna notice Make sure you're authenticated properly ¯\_(ツ)_/¯
lerna verb getCurrentSHA f6328967d0453c363a0d22d8d99f07f62da4ffad
lerna verb pack-directory packages/components/buttons/fui-button
lerna verb pack-directory packages/components/forms/fui-checkbox
lerna verb pack-directory packages/components/forms/fui-form
lerna verb pack-directory packages/components/forms/fui-radio
lerna verb pack-directory packages/components/forms/fui-select
lerna verb pack-directory packages/components/forms/fui-switch
lerna verb pack-directory packages/components/forms/fui-text-field
lerna verb pack-directory packages/fui-core
events.js:136
      throw er; // Unhandled 'error' event
      ^

ZlibError: zlib: Cannot read property 'length' of null
    at Gzip.Zlib.(anonymous function) (/Users/cameron.drysdale/Projects/fui/node_modules/minizlib/index.js:126:21)
    at Gzip.write (/Users/cameron.drysdale/Projects/fui/node_modules/minizlib/index.js:247:21)
    at Object.source.on.chunk (/Users/cameron.drysdale/Projects/fui/node_modules/tar/lib/pack.js:340:18)
    at Object.emit (events.js:159:13)
    at Object.emit (/Users/cameron.drysdale/Projects/fui/node_modules/minipass/index.js:308:25)
    at Object.write (/Users/cameron.drysdale/Projects/fui/node_modules/minipass/index.js:102:17)
    at Object.[onread] (/Users/cameron.drysdale/Projects/fui/node_modules/tar/lib/write-entry.js:263:10)
    at fs.read (/Users/cameron.drysdale/Projects/fui/node_modules/tar/lib/write-entry.js:220:19)
    at FSReqWrap.wrapper [as oncomplete] (fs.js:676:17)

The text was updated successfully, but these errors were encountered:

iamdriz · 2019-09-19T10:43:36Z

We're gonna attempt updating our environment to the latest versions of everything to see if that fixes the problem but any help would be appreciated.

I wonder if the use of a proxy registry is the problem? If so how could we fix that?

iamdriz · 2019-09-19T13:30:31Z

Upgrade to Mojave and upgraded Node to the latest version and it worked fine.

phun-ky · 2019-10-02T12:41:38Z

I have a similar issue, reported first here: isaacs/minizlib#8

Not reproducable locally on ubuntu 18.04 and node v 8.1* or ubuntu bash with windows wsl, same node and npm version 6.5.0. Using latest lerna in build. node@10.16.0

cc @evocateur FYI

jescalan · 2020-05-21T19:09:22Z

Just got this error on osx with that latest version of node...

relevant parts of the error log:

211 error ZlibError: zlib: invalid code lengths set
211 error     at Unzip.<anonymous> (~/node_modules/minizlib/index.js:73:52)
211 error     at Unzip.emit (events.js:215:7)
211 error     at Zlib.zlibOnError [as onerror] (zlib.js:173:8)
211 error     at processChunkSync (zlib.js:416:12)
211 error     at Unzip.ZlibBase._processChunk (zlib.js:391:12)
211 error     at Unzip.write (~/node_modules/minizlib/index.js:140:30)
211 error     at Object.write (~/node_modules/tar/lib/parse.js:305:58)
211 error     at Yallist.<anonymous> (~/node_modules/minipass/index.js:374:18)
211 error     at Yallist.forEach (~/node_modules/minipass/node_modules/yallist/yallist.js:156:8)
211 error     at ReadStream.emit (~/node_modules/minipass/index.js:373:20)
211 error     at ReadStream.emit (~/node_modules/fs-minipass/index.js:154:22)
211 error     at ReadStream.write (~/node_modules/minipass/index.js:163:17)
211 error     at ReadStream.[_handleChunk] (~/node_modules/fs-minipass/index.js:131:19)
211 error     at ReadStream.[_onread] (~/node_modules/fs-minipass/index.js:109:32)
211 error     at ~/node_modules/fs-minipass/index.js:101:22

tylerlevine · 2020-05-29T21:37:08Z

I got this today as well on osx, node 10.16.3, lerna 3.20.2.

lerna ERR! ZlibError: zlib: too many length or distance symbols
lerna ERR!     at Unzip.ZlibBase.(anonymous function).on.er (/Users/tyler/src/bitgojs/node_modules/minizlib/index.js:73:52)
lerna ERR!     at Unzip.emit (events.js:203:15)
lerna ERR!     at Zlib.zlibOnError [as onerror] (zlib.js:165:8)
lerna ERR!     at processChunkSync (zlib.js:403:12)
lerna ERR!     at Unzip.ZlibBase._processChunk (zlib.js:378:12)
lerna ERR!     at Unzip.write (/Users/tyler/src/bitgojs/node_modules/minizlib/index.js:140:30)
lerna ERR!     at Object.write (/Users/tyler/src/bitgojs/node_modules/tar/lib/parse.js:305:58)
lerna ERR!     at Yallist.pipes.forEach.p (/Users/tyler/src/bitgojs/node_modules/minipass/index.js:374:18)
lerna ERR!     at Yallist.forEach (/Users/tyler/src/bitgojs/node_modules/yallist/yallist.js:156:8)
lerna ERR!     at ReadStream.emit (/Users/tyler/src/bitgojs/node_modules/minipass/index.js:373:20)
lerna ERR!     at ReadStream.emit (/Users/tyler/src/bitgojs/node_modules/fs-minipass/index.js:154:22)
lerna ERR!     at ReadStream.write (/Users/tyler/src/bitgojs/node_modules/minipass/index.js:163:17)
lerna ERR!     at ReadStream.[_handleChunk] (/Users/tyler/src/bitgojs/node_modules/fs-minipass/index.js:131:19)
lerna ERR!     at ReadStream.[_onread] (/Users/tyler/src/bitgojs/node_modules/fs-minipass/index.js:109:32)
lerna ERR!     at fs.read (/Users/tyler/src/bitgojs/node_modules/fs-minipass/index.js:101:22)
lerna ERR! lerna zlib: too many length or distance symbols

Downgrading to lerna 3.18.4 allowed me to publish successfully.

phun-ky · 2020-06-11T07:48:34Z

Got this again today, 3.22.0:

lerna info git Pushing tags...
lerna WARN gitPush fatal: the receiving end does not support --atomic push
lerna WARN gitPush fatal: the remote end hung up unexpectedly
lerna WARN gitPush error: failed to push some refs to 'https://dev.azure.com/if-it/If%20Design%20Hub/_git/guybrush'
lerna WARN gitPush 
lerna info gitPush --atomic failed, attempting non-atomic push
lerna info publish Publishing packages to npm...
lerna notice Skipping all user and access validation due to third-party registry
lerna notice Make sure you're authenticated properly ¯\_(ツ)_/¯
lerna ERR! ZlibError: zlib: invalid distance too far back
lerna ERR!     at Unzip.<anonymous> (/home/vsts/work/1/s/node_modules/minizlib/index.js:73:52)
lerna ERR!     at Unzip.emit (events.js:327:22)
lerna ERR!     at Zlib.zlibOnError [as onerror] (zlib.js:183:8)
lerna ERR!     at Zlib.writeSync (<anonymous>)
lerna ERR!     at processChunkSync (zlib.js:426:12)
lerna ERR!     at Unzip.ZlibBase._processChunk (zlib.js:401:12)
lerna ERR!     at Unzip.write (/home/vsts/work/1/s/node_modules/minizlib/index.js:140:30)
lerna ERR!     at Object.write (/home/vsts/work/1/s/node_modules/@lerna/get-packed/node_modules/tar/lib/parse.js:305:58)
lerna ERR!     at Yallist.<anonymous> (/home/vsts/work/1/s/node_modules/minipass/index.js:374:18)
lerna ERR!     at Yallist.forEach (/home/vsts/work/1/s/node_modules/minipass/node_modules/yallist/yallist.js:156:8)
lerna ERR!     at ReadStream.emit (/home/vsts/work/1/s/node_modules/minipass/index.js:373:20)
lerna ERR!     at ReadStream.emit (/home/vsts/work/1/s/node_modules/fs-minipass/index.js:154:22)
lerna ERR!     at ReadStream.write (/home/vsts/work/1/s/node_modules/minipass/index.js:163:17)
lerna ERR!     at ReadStream.[_handleChunk] (/home/vsts/work/1/s/node_modules/fs-minipass/index.js:131:19)
lerna ERR!     at ReadStream.[_onread] (/home/vsts/work/1/s/node_modules/fs-minipass/index.js:109:32)
lerna ERR! lerna zlib: invalid distance too far back

NMinhNguyen · 2020-07-01T17:02:54Z

@phun-ky apologies for the late response. I had seen this issue before when searching for similar problems since upgrading from Lerna v2 to v3, and I actually managed to solve it but forgot to reply here. If you're using Yarn, then add this as a Yarn resolution to your root package.json:

{
  "resolutions": {
    "npm-packlist": "1.1.12"
  }
}

The way I found this was we were using npm pack (similarly to https://reactjs.org/blog/2017/12/15/improving-the-repository-infrastructure.html#simulating-package-publishing) and it would never fail, however our publishes would fail very very frequently since upgrading to Lerna 3. It's been a while so I don't remember the full details now, but I think what happened was I looked at what npm pack did under the hood and it turned out it was using this npm-packlist dependency and an older version than what was introduced in Lerna v3. Since pinning down npm-packlist we haven't had a single failure (and before we'd sometimes get 3-5 in a row). Hope this helps!

Another thing you can do is defer git push until after a succesful publish:

npx lerna version --no-push --yes
npx lerna publish from-git --yes
# See https://github.com/lerna/lerna/blob/ea6ec639560ce71d9df8cbd7eef2d416e71f5442/commands/version/lib/git-push.js#L11
git push --follow-tags --no-verify origin master

carsonfarmer · 2020-07-01T20:46:28Z

The above worked for us just now. Thank you @NMinhNguyen 🙏

Very often running into the "Publish & Install" CI flow failing due to the error: ``` lerna ERR! ZlibError: zlib: invalid block type ``` Research brought me to lerna/lerna#2278 (comment), so I am now attempting to implement this fix to see if it solves the random failures.

benmarch · 2021-03-11T00:28:01Z

I'm using NPM not Yarn, but publishing before pushing upstream worked for me. Thanks @NMinhNguyen!

tbergquist-godaddy · 2021-11-25T09:20:25Z

I ran into the same failure as @tylerlevine, using lerna 4.0.0 and node 14.17.0 on our CI. I have not been able to reproduce it on my mac locally.

I did what @NMinhNguyen suggested, putting npm-packlist@1.1.12 in resolutions, and this made the pipeline pass. The subsequent commit in npm packlist is Follow symlinks in bundled scoped dependencies, so might this interfere with how lerna works?

This issue is closed, but I don't feel like pinning down a lower version of npm-packlist is a sustainable fix, it feels like more of a workaround.

phun-ky · 2021-12-09T18:29:33Z

@tbergquist-godaddy @NMinhNguyen getting a similar issue now with lerna 4, node 16 and npm 8:

isaacs/minizlib#8 (comment)

omar84 · 2022-05-04T23:26:24Z

@phun-ky did you ever solve this lerna error? if so, can you please share

phun-ky · 2022-05-07T07:52:39Z

We separated the versioning and publishing task in our ci to `lerna version` and `lerna publish` (with required parameters), and it solved it self.

iamdriz closed this as completed Sep 19, 2019

iamdriz mentioned this issue Sep 19, 2019

[Q] How to disable publish tarball file? #1940

Closed

phun-ky mentioned this issue Jun 11, 2020

ZlibError: zlib: invalid code lengths set isaacs/minizlib#8

Closed

iartemiev mentioned this issue Aug 25, 2020

ci: zlib fix using npm-packlist aws-amplify/amplify-js#6650

Merged

gribnoysup mentioned this issue Jun 25, 2021

ci(github): Fix windows failures during the build process mongodb-js/compass#2295

Merged

phun-ky mentioned this issue Dec 10, 2021

[BUG] Npm pack (via publish via lerna version) returns Bash exited with code '1' npm/cli#4157

Open

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ZlibError when calling publish #2278

ZlibError when calling publish #2278

iamdriz commented Sep 18, 2019

iamdriz commented Sep 19, 2019

iamdriz commented Sep 19, 2019

phun-ky commented Oct 2, 2019 •

edited

jescalan commented May 21, 2020 •

edited

tylerlevine commented May 29, 2020

phun-ky commented Jun 11, 2020

NMinhNguyen commented Jul 1, 2020 •

edited

carsonfarmer commented Jul 1, 2020

benmarch commented Mar 11, 2021

tbergquist-godaddy commented Nov 25, 2021

phun-ky commented Dec 9, 2021

omar84 commented May 4, 2022

phun-ky commented May 7, 2022 via email

ZlibError when calling publish #2278

ZlibError when calling publish #2278

Comments

iamdriz commented Sep 18, 2019

Expected Behavior

Current Behavior

Possible Solution

Steps to Reproduce (for bugs)

Your Environment

iamdriz commented Sep 19, 2019

iamdriz commented Sep 19, 2019

phun-ky commented Oct 2, 2019 • edited

jescalan commented May 21, 2020 • edited

tylerlevine commented May 29, 2020

phun-ky commented Jun 11, 2020

NMinhNguyen commented Jul 1, 2020 • edited

carsonfarmer commented Jul 1, 2020

benmarch commented Mar 11, 2021

tbergquist-godaddy commented Nov 25, 2021

phun-ky commented Dec 9, 2021

omar84 commented May 4, 2022

phun-ky commented May 7, 2022 via email

phun-ky commented Oct 2, 2019 •

edited

jescalan commented May 21, 2020 •

edited

NMinhNguyen commented Jul 1, 2020 •

edited