Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

gz.inflate() error -5 #1

Closed
wants to merge 4 commits into from

3 participants

@SaltwaterC

Sometimes gz.inflate() throws an exception (gz error -5). This is a simple workaround.

@Woodya
Owner

thanks. i think the try should only surround the inflate call. else you surround the emit call, itself surrounds the callback execution.

@SaltwaterC

I don't get it. Can you post some code?

gunzipstream.js
@@ -15,8 +15,14 @@ var GunzipStream = function(readStream, enc) {
self.gz.init({encoding: enc});
self.ondata = function(data) {
- var inflated = self.gz.inflate(data);
- self.emit('data', inflated);
+ try {
+ var inflated = self.gz.inflate(data);
+ self.emit('data', inflated);
@Woodya Owner
Woodya added a note

i'm thinking pull this line out of the try/catch

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
gunzipstream.js
@@ -15,8 +15,14 @@ var GunzipStream = function(readStream, enc) {
self.gz.init({encoding: enc});
self.ondata = function(data) {
- var inflated = self.gz.inflate(data);
- self.emit('data', inflated);
+ try {
+ var inflated = self.gz.inflate(data);
+ self.emit('data', inflated);
+ } catch (err) {
+ self.gz.end();
+ self.emit('error', err);
+ self._cleanup();
@Woodya Owner
Woodya added a note

add "return;" here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
gunzipstream.js
@@ -15,8 +15,14 @@ var GunzipStream = function(readStream, enc) {
self.gz.init({encoding: enc});
self.ondata = function(data) {
- var inflated = self.gz.inflate(data);
- self.emit('data', inflated);
+ try {
+ var inflated = self.gz.inflate(data);
+ self.emit('data', inflated);
+ } catch (err) {
+ self.gz.end();
+ self.emit('error', err);
+ self._cleanup();
+ }
@Woodya Owner
Woodya added a note

and here, do the 'data' emit

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@SaltwaterC

I get the concept now. The thing I don't get is: why. Is there something that I'm missing or something that's isn't stated into the node.js Events docs?

@lbdremy

Great, I wanted post an issue about this.
I get randomly sometimes this error. Do you have an idea why?
Thanks

@SaltwaterC

If you have some input example, I guess somebody could take a stab at debugging it. I didn't have the chance to save the resource that gave me the -5 headache.

@Woodya
Owner

ok. sorry for my prolonged absence. it's more concise to simply:

        return self.onerror(err);

in the catch() section, so i'm going to do that.
can anyone explain what is up with modules being deprecated? does that mean those are no longer part of the pkg?
is there a write up on this somewhere?

@SaltwaterC

npm 1.0.x complains about the deprecation every time I run npm install or npm update into a directory where gzbz2 is installed.

@Woodya
Owner

hey guys, have you looked further into the -5 (buff_error) issue?

from the zlib spec

INFLATE:
Z_BUF_ERROR if no progress is possible or if there was not enough room in the output buffer when Z_FINISH is used. Note that Z_BUF_ERROR is not fatal, and inflate() can be called again with more input and more output space to continue decompressing.

DEFLATE:
Z_BUF_ERROR if no progress is possible (for example avail_in or avail_out was zero). Note that Z_BUF_ERROR is not fatal, and deflate() can be called again with more input and more output space to continue compressing.

it seems that perhaps this is recoverable, perhaps not. http://www.gzip.org/zlib/zlib_faq.html#faq05

if you can reproduce the error scenario, we/i could determine if it can be handled in the c code.
it's happening in the GzipInflate (from your bug title) rather than GzipEnd, which is a bummer, as the zlib comments indicate that simply means "no progress can be made"

i think it's possible that there is an issue with the loop handling of buf_errr wrt compress.cc#331
if ret = BUF_ERR, then out_avail will be CHUNK, but that incrmements out_len by zero and moves on.
it's possible that it should re-attempt inflate() with avail_out = 2*CHUNK to see if that will succeed.

thoughts?
really, if you have a file and code to re-create this consistently, that'd be a huge help.
-w

@SaltwaterC

I can try to find some input that triggers this condition, but most probably would be some sort of brute force. It could take a while. I found this bug by chance (same as various node.js bugs). The joys of working with a large data set ...

@SaltwaterC SaltwaterC closed this
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Aug 4, 2011
  1. @SaltwaterC
Commits on Aug 22, 2011
  1. @SaltwaterC
  2. @SaltwaterC
Commits on Aug 25, 2011
  1. @SaltwaterC

    Removed the deprecated "modules" from package.json. Makes npm to shut…

    SaltwaterC authored
    … up about the deprecation warning.
This page is out of date. Refresh to see the latest.
Showing with 8 additions and 4 deletions.
  1. +8 −1 gunzipstream.js
  2. +0 −3  package.json
View
9 gunzipstream.js
@@ -15,7 +15,14 @@ var GunzipStream = function(readStream, enc) {
self.gz.init({encoding: enc});
self.ondata = function(data) {
- var inflated = self.gz.inflate(data);
+ try {
+ var inflated = self.gz.inflate(data);
+ } catch (err) {
+ self.gz.end();
+ self.emit('error', err);
+ self._cleanup();
+ return;
+ }
self.emit('data', inflated);
};
self.onclose = function() {
View
3  package.json
@@ -12,9 +12,6 @@
},
"engine": [ "node >=0.4.0" ],
"main": "build/default/gzbz2",
- "modules": {
- "gunzipstream": "./gunzipstream",
- "bunzipstream": "./bunzipstream"},
"directories.lib": "build/default/",
"scripts": {
"build" : "node-waf configure build",
Something went wrong with that request. Please try again.