zlib: Error: invalid stored block lengths

[Swaagie]

Unpacking (larger) zipped archives from github repositories will result in error: invalid stored block lengths.

I was at least able to reproduce this issue with the packed releases 1 and 2. Any zip file of respectable size should produce the error. However, on smaller distro's (3) the error is not occurring.

1. https://github.com/TryGhost/Ghost/releases/download/0.3.3/Ghost-0.3.3.zip
2. https://github.com/primus/primus/archive/1.4.4.zip
3. https://github.com/Swaagie/smith/archive/v1.0.zip

var fs = require('fs')
  , zlib = require('zlib')
  , file = process.env.HOME + '/Downloads/dist.zip'; // any packed repo with: git archive HEAD -o dist.zip 

fs.createReadStream(file).pipe(zlib.createInflateRaw());

I know this does not lead to any useful output in itself, but it is a simplification of the unzip package process.

This seems to be related to the two different underlying package libraries.

• git archive uses the libarchive library
• node.js uses the zlib library

Repackaging the files with zip results in the same error, I couldn't find the underlying library that is used by zip however. Will follow up on this tomorrrow

Repacking the files with tar and using createUnzip completes without errors.

Does anyone know of other languages/command line tools that bind into the zlib library so node.js could actually be excluded or not?

Edit: this same error is also present in node version 0.8.25 which makes me suspect the zlib library, is there an update available for this library?

System specs:
64 bit, node 0.10.21, ubuntu 13.10

[Swaagie]

Also tested against master, e.g. v0.11.8-pre, same error is still occuring

[tjfontaine]

I'm not really sure what to do here, except that zip contains a header that you have to seek beyond before you can start your inflate stream

var fs = require('fs');
var zlib = require('zlib');

var file = process.argv[2];

var zs = zlib.createInflateRaw();
var fstream = fs.createReadStream(file);

function doIt() {
  if (fstream.read(2))
    fstream.pipe(zs);
  else
    setImmediate(doIt);
}

setImmediate(doIt);

zs.on('error', function(err) {
  console.error(err.errno, err);
});

And it works fine. Closing as this is mostly beyond the scope of node, if you can find a more minimal test case that can show a problem with our usage of zlib please let us know.

[Swaagie]

Ok, will try what you posted here, how could I make this testcase more minimal btw? Stripping out the streams etc. or?

For future reference:
I updated to the most recent zlib (1.2.5) available in the master branch of chromium did not resolve the issue nor did updating to the source from zlib.net (1.2.8) change anything.

[tjfontaine]

I guess my assertion is that if you're hitting this it's because you're not at the right offset of the stream, which is a programmer error and not necessarily a node or zlib issue.

[Swaagie]

you're right your snippet and specifcally just reading the first two bytes before actually piping works fine. However then I can transfer this issue to the unzip module since it all started with using that :)

edit: thx for the response and insight, appreciated

[jeffpar]

I'm getting this "invalid stored block lengths" error with this zip file using the node-unzip module. I don't think it's a signature problem, because node-unzip's parse.js successfully reads the signature itself (0x04034b50).

[afollestad]

I got this error today, with manual piping