New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

small file zip error #112

Closed
qlqllu opened this Issue Mar 19, 2014 · 13 comments

Comments

Projects
None yet
3 participants
@qlqllu

qlqllu commented Mar 19, 2014

When I use jszip to zip a small file, result zip file may be not correct on some OS.
On some OS, it's OK.

The small file is a small .js file, contains only one line

test_order_load_value = "a";

This is the error when unzip the file:
zip-error
This is my OS info:
win8

@dduponchel

This comment has been minimized.

Show comment
Hide comment
@dduponchel

dduponchel Mar 19, 2014

Collaborator

That's strange. How did you generate the file ? With compression ? Does other zip files work ? (I don't access to a Windows 8 right now).

Collaborator

dduponchel commented Mar 19, 2014

That's strange. How did you generate the file ? With compression ? Does other zip files work ? (I don't access to a Windows 8 right now).

@Stuk

This comment has been minimized.

Show comment
Hide comment
@Stuk

Stuk Mar 19, 2014

Owner

@qlqllu could you also post the zip file and unzipped text file somewhere?

Owner

Stuk commented Mar 19, 2014

@qlqllu could you also post the zip file and unzipped text file somewhere?

@Stuk Stuk added the bug label Mar 19, 2014

@qlqllu

This comment has been minimized.

Show comment
Hide comment
@qlqllu

qlqllu Mar 20, 2014

This is my code that does zip:

exports.zipFolderSync = function(folderPath, zipFile) {
  var zip = new JSZip();
  folderPath = path.normalize(folderPath);
  visitFolderFiles(folderPath, function(filePath, fileName, isDirectory){
    if(isDirectory){
      zip.folder(filePath.substr(folderPath.length, filePath.length));
    }else{
      var fileContent;
      fileContent = fs.readFileSync(filePath);
      zip.file(filePath.substr(folderPath.length, filePath.length), fileContent);
    }
  });

  var data = zip.generate({type: 'nodebuffer', compression: 'DEFLATE'});
  fs.writeFileSync(zipFile, data, 'binary');
};

@Stuk
Sorry for that I don't know how to upload the error zip file.

qlqllu commented Mar 20, 2014

This is my code that does zip:

exports.zipFolderSync = function(folderPath, zipFile) {
  var zip = new JSZip();
  folderPath = path.normalize(folderPath);
  visitFolderFiles(folderPath, function(filePath, fileName, isDirectory){
    if(isDirectory){
      zip.folder(filePath.substr(folderPath.length, filePath.length));
    }else{
      var fileContent;
      fileContent = fs.readFileSync(filePath);
      zip.file(filePath.substr(folderPath.length, filePath.length), fileContent);
    }
  });

  var data = zip.generate({type: 'nodebuffer', compression: 'DEFLATE'});
  fs.writeFileSync(zipFile, data, 'binary');
};

@Stuk
Sorry for that I don't know how to upload the error zip file.

@dduponchel

This comment has been minimized.

Show comment
Hide comment
@dduponchel

dduponchel Mar 20, 2014

Collaborator

I reproduced the bug on Windows 8.1 with the following script :

var JSZip = require("jszip");
var fs = require("fs");

var zip = new JSZip();
zip.file("a.js", 'test_order_load_value = "a";\n');
var data = zip.generate({type: 'nodebuffer', compression: 'DEFLATE'});
fs.writeFileSync("issue_112.zip", data, 'binary');

I don't get the same error message but it seems to be the same issue (or at least related).
issue_112

Collaborator

dduponchel commented Mar 20, 2014

I reproduced the bug on Windows 8.1 with the following script :

var JSZip = require("jszip");
var fs = require("fs");

var zip = new JSZip();
zip.file("a.js", 'test_order_load_value = "a";\n');
var data = zip.generate({type: 'nodebuffer', compression: 'DEFLATE'});
fs.writeFileSync("issue_112.zip", data, 'binary');

I don't get the same error message but it seems to be the same issue (or at least related).
issue_112

@dduponchel

This comment has been minimized.

Show comment
Hide comment
@dduponchel

dduponchel Mar 20, 2014

Collaborator

I don't get the issue with compression: 'STORE'. With DEFLATE the file seems correct (unzip and 7zip decompress it without any issue) but Windows 8.1 can't open it. I tested with pako as DEFLATE implementation and I get no error.
@qlqllu could you do two tests ?

  • one test with compression: 'STORE'
  • one test with pako (see below)

To test with pako, you can do the following :
in your terminal, npm install pako@0.1.1
in your js code, before any JSZip usage :

var pako = require("pako");
JSZip.compressions.DEFLATE.compress = function (input) {
  return pako.deflateRaw(input);
};
Collaborator

dduponchel commented Mar 20, 2014

I don't get the issue with compression: 'STORE'. With DEFLATE the file seems correct (unzip and 7zip decompress it without any issue) but Windows 8.1 can't open it. I tested with pako as DEFLATE implementation and I get no error.
@qlqllu could you do two tests ?

  • one test with compression: 'STORE'
  • one test with pako (see below)

To test with pako, you can do the following :
in your terminal, npm install pako@0.1.1
in your js code, before any JSZip usage :

var pako = require("pako");
JSZip.compressions.DEFLATE.compress = function (input) {
  return pako.deflateRaw(input);
};
@qlqllu

This comment has been minimized.

Show comment
Hide comment
@qlqllu

qlqllu Mar 21, 2014

@dduponchel
Thank you very much. It works! And it's more faster!

qlqllu commented Mar 21, 2014

@dduponchel
Thank you very much. It works! And it's more faster!

@qlqllu

This comment has been minimized.

Show comment
Hide comment
@qlqllu

qlqllu Mar 21, 2014

@dduponchel
I find a new error if I use pako: when unzip file, there is one error about one gif file.
This error is reproducible in specific machine(MA). The zip generated in MA will have unzip error in any machine.

This is the gif file:
ajax-loader

This is my OS info:
image

qlqllu commented Mar 21, 2014

@dduponchel
I find a new error if I use pako: when unzip file, there is one error about one gif file.
This error is reproducible in specific machine(MA). The zip generated in MA will have unzip error in any machine.

This is the gif file:
ajax-loader

This is my OS info:
image

@dduponchel

This comment has been minimized.

Show comment
Hide comment
@dduponchel

dduponchel Mar 21, 2014

Collaborator

I can't reproduce the bug with your gif file.
If you try with the following code, can you reproduce the error ?

var fs = require("fs");
var JSZip = require("jszip");
var pako = require("pako");
JSZip.compressions.DEFLATE.compress = function (input) {
  return pako.deflateRaw(input);
};
var zip = new JSZip();

var gif = fs.readFileSync("image.gif"); // <- change the file name
zip.file("image.gif", gif);

var data = zip.generate({type: 'nodebuffer', compression: 'DEFLATE'});
fs.writeFileSync("issue_112.zip", data);

If you still have the error, could send me (by email for example) the generated zip file ?

Regarding pako, it will be soon included inside JSZip see #102.

Collaborator

dduponchel commented Mar 21, 2014

I can't reproduce the bug with your gif file.
If you try with the following code, can you reproduce the error ?

var fs = require("fs");
var JSZip = require("jszip");
var pako = require("pako");
JSZip.compressions.DEFLATE.compress = function (input) {
  return pako.deflateRaw(input);
};
var zip = new JSZip();

var gif = fs.readFileSync("image.gif"); // <- change the file name
zip.file("image.gif", gif);

var data = zip.generate({type: 'nodebuffer', compression: 'DEFLATE'});
fs.writeFileSync("issue_112.zip", data);

If you still have the error, could send me (by email for example) the generated zip file ?

Regarding pako, it will be soon included inside JSZip see #102.

@qlqllu

This comment has been minimized.

Show comment
Hide comment
@qlqllu

qlqllu Mar 24, 2014

My UT can't pass when using pako.

var fs = require('fs');
var path = require('path');
var assert = require('assert');
var jsZIP = require('jszip');
var pako = require("pako");

jsZIP.compressions.DEFLATE.compress = function (input) {
  return pako.deflateRaw(input);
};

/* global describe, it, __dirname */
describe('Animated GIF compression', function() {
  it('Compression animated.gif', function() {
    var zip = new jsZIP();
    var fileContent = fs.readFileSync(
      path.join(__dirname, 'animated.gif'), {
        encoding: 'base64'
      }
    );
    zip.file('animated.gif', fileContent, {
      base64: true,
      date: new Date("Nov 25, 2013")
    });
    var data = zip.generate({
      type: 'nodebuffer',
      compression: 'DEFLATE'
    });
    var zipContent = fs.readFileSync(
      path.join(__dirname, 'animated.zip')
    );
    assert.strictEqual(data.toString('binary'), zipContent.toString('binary'));
  });

  it('Compression ajax-loader.gif', function() {
    var zip = new jsZIP();
    var fileContent = fs.readFileSync(path.join(__dirname, 'ajax-loader.gif'));
    zip.file('ajax-loader.gif', fileContent, {
      date: new Date("Nov 25, 2013")
    });
    var data = zip.generate({
      type: 'nodebuffer',
      compression: 'DEFLATE'
    });
    var zipContent = fs.readFileSync(path.join(__dirname, 'ajax-loader.zip'));
    assert.strictEqual(data.toString('binary'), zipContent.toString('binary'));
  });
});

The first UT is from this issue: #89,
The second is from @dduponchel 's comment.

Could you point out the error in my UT, or could you have a try?
Thanks!

qlqllu commented Mar 24, 2014

My UT can't pass when using pako.

var fs = require('fs');
var path = require('path');
var assert = require('assert');
var jsZIP = require('jszip');
var pako = require("pako");

jsZIP.compressions.DEFLATE.compress = function (input) {
  return pako.deflateRaw(input);
};

/* global describe, it, __dirname */
describe('Animated GIF compression', function() {
  it('Compression animated.gif', function() {
    var zip = new jsZIP();
    var fileContent = fs.readFileSync(
      path.join(__dirname, 'animated.gif'), {
        encoding: 'base64'
      }
    );
    zip.file('animated.gif', fileContent, {
      base64: true,
      date: new Date("Nov 25, 2013")
    });
    var data = zip.generate({
      type: 'nodebuffer',
      compression: 'DEFLATE'
    });
    var zipContent = fs.readFileSync(
      path.join(__dirname, 'animated.zip')
    );
    assert.strictEqual(data.toString('binary'), zipContent.toString('binary'));
  });

  it('Compression ajax-loader.gif', function() {
    var zip = new jsZIP();
    var fileContent = fs.readFileSync(path.join(__dirname, 'ajax-loader.gif'));
    zip.file('ajax-loader.gif', fileContent, {
      date: new Date("Nov 25, 2013")
    });
    var data = zip.generate({
      type: 'nodebuffer',
      compression: 'DEFLATE'
    });
    var zipContent = fs.readFileSync(path.join(__dirname, 'ajax-loader.zip'));
    assert.strictEqual(data.toString('binary'), zipContent.toString('binary'));
  });
});

The first UT is from this issue: #89,
The second is from @dduponchel 's comment.

Could you point out the error in my UT, or could you have a try?
Thanks!

@dduponchel

This comment has been minimized.

Show comment
Hide comment
@dduponchel

dduponchel Mar 24, 2014

Collaborator

zlib.js and pako don't have the same implementation and won't produce the same result : that's why your unit tests won't pass !

Collaborator

dduponchel commented Mar 24, 2014

zlib.js and pako don't have the same implementation and won't produce the same result : that's why your unit tests won't pass !

@dduponchel

This comment has been minimized.

Show comment
Hide comment
@dduponchel

dduponchel Mar 25, 2014

Collaborator

@qlqllu sent me an email with the file ajax-loader.gif and the generated zip file. The generated zip file is corrupted : after decompression, the content doesn't match its crc32.
I reproduced the error : I get exactly the same (corrupted) zip but only when I force the version 0.1.0 of pako. With pako 0.1.1, the bug have been fixed and I get a good zip file.

Could you double check that you use the version 0.1.1 of pako ?

Collaborator

dduponchel commented Mar 25, 2014

@qlqllu sent me an email with the file ajax-loader.gif and the generated zip file. The generated zip file is corrupted : after decompression, the content doesn't match its crc32.
I reproduced the error : I get exactly the same (corrupted) zip but only when I force the version 0.1.0 of pako. With pako 0.1.1, the bug have been fixed and I get a good zip file.

Could you double check that you use the version 0.1.1 of pako ?

@qlqllu

This comment has been minimized.

Show comment
Hide comment
@qlqllu

qlqllu Mar 26, 2014

@dduponchel
Yes, I use pako 0.1.0. After I upgrade to 0.1.1, the error is gone!
Thank you very much!

qlqllu commented Mar 26, 2014

@dduponchel
Yes, I use pako 0.1.0. After I upgrade to 0.1.1, the error is gone!
Thank you very much!

@dduponchel

This comment has been minimized.

Show comment
Hide comment
@dduponchel

dduponchel Apr 9, 2014

Collaborator

@Stuk a bit late but... Should I prepare a v2.2.1 ?

Collaborator

dduponchel commented Apr 9, 2014

@Stuk a bit late but... Should I prepare a v2.2.1 ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment