Size Constraints - Malloc Error

[brainmap]

Hi Brian,

Thanks for the great work with this. Unfortunately, I'm running into a malloc error when trying to zip or unzip very large files (685MB zipped, 2.1GB unzipped). I'm running this on a machine w/ 4GB ram, so I think it may actually a limitation in the memory size that ruby itself allows. Are there any plans to implement a buffer to unzip larger files? Unfortunately I'm not familiar with C well enough to edit the bindings myself, hoping this is on your radar. Thanks again!

Erik Kastman

[brianmario]

Hi Erik,

I've been wanting to refactor the C code of this gem for a while but haven't had the time. It has quite a few bugs and uses parts of the Ruby C API that it shouldn't be. I suspect the bug you're hitting has to do with that, or something related.
I'll do my best to get around to fixing it up, but for now maybe search Github for "bz2" (the original gem sources).

-Brian

[brainmap]

Thanks Brian - I'll definitely take a look at it. I suspect that the original sources will also have this memory problem, but if not (and it has to do with your implementation specifically) i'll post a stack trace for you. Thanks,

Erik

[brianmario]

Nice, that would be awesome.
Thanks Erik