New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bad zipfile offset #38

Closed
andrecaribe opened this Issue Aug 3, 2013 · 13 comments

Comments

Projects
None yet
2 participants
@andrecaribe

andrecaribe commented Aug 3, 2013

Hi,

Sometimes my zip file has errors like this:

file #17:  bad zipfile offset (local header sig):  784439
file #18:  bad zipfile offset (local header sig):  802437
testing: medias/images/q_444_256x256.jpg   OK
testing: medias/images/q_445_256x256.jpg   OK
testing: data.json                OK
At least one error was detected in B1L12.zip.

See complete unzip -t here: https://gist.github.com/andrecaribe/6144876

I think that my problem is on archive.append method (https://gist.github.com/andrecaribe/6144848) what is inside the loop. Is it? How to make the best way?

@andrecaribe

This comment has been minimized.

Show comment
Hide comment
@andrecaribe

andrecaribe Aug 3, 2013

I tried to append some images manually but I got same result.

archive
  .append(fs.createReadStream('/var/www/files/exercises/questions/612_256x256.jpg'), { name: 't_001.jpg' })
  .append(fs.createReadStream('/var/www/files/exercises/questions/613_256x256.jpg'), { name: 't_002.jpg' })
  .append(fs.createReadStream('/var/www/files/exercises/questions/614_256x256.jpg'), { name: 't_003.jpg' })
  .append(fs.createReadStream('/var/www/files/exercises/questions/615_256x256.jpg'), { name: 't_004.jpg' })
  .append(fs.createReadStream('/var/www/files/exercises/questions/616_256x256.jpg'), { name: 't_005.jpg' });

The images are not corrupted according to the jpeginfo and exiftool.

andrecaribe commented Aug 3, 2013

I tried to append some images manually but I got same result.

archive
  .append(fs.createReadStream('/var/www/files/exercises/questions/612_256x256.jpg'), { name: 't_001.jpg' })
  .append(fs.createReadStream('/var/www/files/exercises/questions/613_256x256.jpg'), { name: 't_002.jpg' })
  .append(fs.createReadStream('/var/www/files/exercises/questions/614_256x256.jpg'), { name: 't_003.jpg' })
  .append(fs.createReadStream('/var/www/files/exercises/questions/615_256x256.jpg'), { name: 't_004.jpg' })
  .append(fs.createReadStream('/var/www/files/exercises/questions/616_256x256.jpg'), { name: 't_005.jpg' });

The images are not corrupted according to the jpeginfo and exiftool.

@ctalkington

This comment has been minimized.

Show comment
Hide comment
@ctalkington

ctalkington Aug 3, 2013

Member

What OS, platform, node version? I've only seen one other report of such errors and that was on Mac OSX which I unfortunately am unable to test locally.

Member

ctalkington commented Aug 3, 2013

What OS, platform, node version? I've only seen one other report of such errors and that was on Mac OSX which I unfortunately am unable to test locally.

@andrecaribe

This comment has been minimized.

Show comment
Hide comment
@andrecaribe

andrecaribe Aug 4, 2013

The package is generated on Debian Wheezy (7.1), Node v0.10.13
I test zip files on Debian and Mac OS X 10.7.5

UPDATE:
Node was compiled with:

{ 'target_defaults': { 'cflags': [],
  'default_configuration': 'Release',
  'defines': [],
  'include_dirs': [],
  'libraries': []},
  'variables': { 'clang': 0,
    'gcc_version': 47,
    'host_arch': 'x64',
    'node_install_npm': 'true',
    'node_prefix': '',
    'node_shared_cares': 'false',
    'node_shared_http_parser': 'false',
    'node_shared_libuv': 'false',
    'node_shared_openssl': 'false',
    'node_shared_v8': 'false',
    'node_shared_zlib': 'false',
    'node_tag': '',
    'node_unsafe_optimizations': 0,
    'node_use_dtrace': 'false',
    'node_use_etw': 'false',
    'node_use_openssl': 'true',
    'node_use_perfctr': 'false',
    'node_use_systemtap': 'false',
    'python': '/usr/bin/python',
    'target_arch': 'x64',
    'v8_enable_gdbjit': 0,
    'v8_no_strict_aliasing': 1,
    'v8_use_snapshot': 'true'}}

andrecaribe commented Aug 4, 2013

The package is generated on Debian Wheezy (7.1), Node v0.10.13
I test zip files on Debian and Mac OS X 10.7.5

UPDATE:
Node was compiled with:

{ 'target_defaults': { 'cflags': [],
  'default_configuration': 'Release',
  'defines': [],
  'include_dirs': [],
  'libraries': []},
  'variables': { 'clang': 0,
    'gcc_version': 47,
    'host_arch': 'x64',
    'node_install_npm': 'true',
    'node_prefix': '',
    'node_shared_cares': 'false',
    'node_shared_http_parser': 'false',
    'node_shared_libuv': 'false',
    'node_shared_openssl': 'false',
    'node_shared_v8': 'false',
    'node_shared_zlib': 'false',
    'node_tag': '',
    'node_unsafe_optimizations': 0,
    'node_use_dtrace': 'false',
    'node_use_etw': 'false',
    'node_use_openssl': 'true',
    'node_use_perfctr': 'false',
    'node_use_systemtap': 'false',
    'python': '/usr/bin/python',
    'target_arch': 'x64',
    'v8_enable_gdbjit': 0,
    'v8_no_strict_aliasing': 1,
    'v8_use_snapshot': 'true'}}
@ctalkington

This comment has been minimized.

Show comment
Hide comment
@ctalkington

ctalkington Aug 4, 2013

Member

should really be fine, do you get same unzip test results on both platforms? ive only seen it on Mac.

Member

ctalkington commented Aug 4, 2013

should really be fine, do you get same unzip test results on both platforms? ive only seen it on Mac.

@ctalkington

This comment has been minimized.

Show comment
Hide comment
@ctalkington

ctalkington Aug 4, 2013

Member

do the zips open with Mac's Archive tool?

you might also try listening for outputs close event, finalize doesn't always promise that other streams have completed. also, not sure the point of the fs.writeFile you wrapped it all in?

Member

ctalkington commented Aug 4, 2013

do the zips open with Mac's Archive tool?

you might also try listening for outputs close event, finalize doesn't always promise that other streams have completed. also, not sure the point of the fs.writeFile you wrapped it all in?

@andrecaribe

This comment has been minimized.

Show comment
Hide comment
@andrecaribe

andrecaribe Aug 5, 2013

Yes, same results on both.

I moved the response to output.on('close', ... and moved the code to out of fs.writeFile for tests. Same results. Sometimes - if the zip file don't have "bad zipfile offset" - the file open with success.
If has error, Mac's Archive tool shows an message:

'Unable to unarchive "XYZ.zip" into "FOLDER".
(Error 2 - No such file or directory exists.)'

Mac's Terminal and Debian 7.1 unzip, extract only files without bad offset.

I tried to use tar instead of zip and it worked fine, but I can not give output with tar or tar.gz

Node 0.10.13 changes some things about zlib. That will influence the archiver? http://blog.nodejs.org/2013/07/09/node-v0-10-13-stable/

andrecaribe commented Aug 5, 2013

Yes, same results on both.

I moved the response to output.on('close', ... and moved the code to out of fs.writeFile for tests. Same results. Sometimes - if the zip file don't have "bad zipfile offset" - the file open with success.
If has error, Mac's Archive tool shows an message:

'Unable to unarchive "XYZ.zip" into "FOLDER".
(Error 2 - No such file or directory exists.)'

Mac's Terminal and Debian 7.1 unzip, extract only files without bad offset.

I tried to use tar instead of zip and it worked fine, but I can not give output with tar or tar.gz

Node 0.10.13 changes some things about zlib. That will influence the archiver? http://blog.nodejs.org/2013/07/09/node-v0-10-13-stable/

@ctalkington

This comment has been minimized.

Show comment
Hide comment
@ctalkington

ctalkington Aug 5, 2013

Member

hum don't think node version is the issue. could you email me the file you are testing locally?

Member

ctalkington commented Aug 5, 2013

hum don't think node version is the issue. could you email me the file you are testing locally?

@andrecaribe

This comment has been minimized.

Show comment
Hide comment
@andrecaribe

andrecaribe Aug 5, 2013

Yes, of course. What you need? I created a simple test and got the same error. I'll send you.

andrecaribe commented Aug 5, 2013

Yes, of course. What you need? I created a simple test and got the same error. I'll send you.

@ctalkington

This comment has been minimized.

Show comment
Hide comment
@ctalkington

ctalkington Aug 5, 2013

Member

ok, send along that project dir (just need js and package.json, no node_modules) and the zip you get from running it so I can see if the binary contents vary in any key spots.

EDIT: email is on my GH profile.

Member

ctalkington commented Aug 5, 2013

ok, send along that project dir (just need js and package.json, no node_modules) and the zip you get from running it so I can see if the binary contents vary in any key spots.

EDIT: email is on my GH profile.

@ctalkington

This comment has been minimized.

Show comment
Hide comment
@ctalkington

ctalkington Aug 11, 2013

Member

sorry for delay. will try out this coming week.

Member

ctalkington commented Aug 11, 2013

sorry for delay. will try out this coming week.

@ctalkington

This comment has been minimized.

Show comment
Hide comment
@ctalkington

ctalkington Aug 17, 2013

Member

the zip you sent definitely fails 7zip and unzip on Windows for me. wasnt able to do much with the test since I dont have local files to match.

Member

ctalkington commented Aug 17, 2013

the zip you sent definitely fails 7zip and unzip on Windows for me. wasnt able to do much with the test since I dont have local files to match.

@ctalkington

This comment has been minimized.

Show comment
Hide comment
@ctalkington

ctalkington Aug 17, 2013

Member

can you do me a favor and try running your test with:

var archive = archiver('zip', { zlib: { level: 0 } } );
Member

ctalkington commented Aug 17, 2013

can you do me a favor and try running your test with:

var archive = archiver('zip', { zlib: { level: 0 } } );
@ctalkington

This comment has been minimized.

Show comment
Hide comment
@ctalkington

ctalkington Aug 18, 2013

Member

checkout #32 and the latest release.

Member

ctalkington commented Aug 18, 2013

checkout #32 and the latest release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment