Join GitHub today
GitHub is home to over 20 million developers working together to host and review code, manage projects, and build software together.
corrupt zip file generation #32
First off, thanks for the hard work, it was great to find an archive module that functions well with node's streams.
However, after getting setup, I realized I was having an issue on one of my linode boxes where corrupt zip archives were being generated. I didn't have the time right now to track down why they were corrupt, but I did isolate what was causing it, and I stripped down part of the app to what was breaking:
Most importantly, this was the culprit:
To my surprise, this fixed the issue. Now again, this wasn't affecting my local machine, but was affecting my production linode box; it's 100% reproducible there.
I don't have time right now to dig into the issue, but I wanted to leave this issue in case you wanted to poke around and see if something is obviously wrong (nothing immediately jumped at me scrolling through
Edit: if it helps, both my local and the remote machines are running v0.10.11.
This comment has been minimized.
This comment has been minimized.Show comment Hide comment
since archiver can handle being passed files right after each other and then dealing dealing with it internally 1 by 1. you really shouldn't need async at all on that part and there is a nice little helper for lazy loading streams that should ease open file limits.
bit of update from discoveries, this corruption seems to happen more when you are writing multiple zips at once and only when the source is a buffer and goes through zlib.
i messed with this some more, it appears if JUST the line to checksum is removed from the
also to be noted this corruption also happens randomly and some times its none, one, two, or all the zips that end up corrupt. It almost makes me feel like its a global leak or there is some memory corruption going on from all the buffers being created. yet that doesn't really fit with the
EDIT: after the latest commits the