corrupted component downloads

[domachine]

i'm not quite sure but since 0.16.7 i'm recognizing problems with corrupted dependencies. component install fetches dependencies but they have syntax errors in their included scripts. they're then copied to the build file and are recognized as syntax errors by the browser. it's really hard to reproduce those errors since they appear irregularly in different dependencies when trying multiple dependency installs. is somebody else recognizing similar behaviour?

i'm using node 0.10.12 with component 0.16.7

[chemzqm]

I've got the same problem several times. I found the script fetched by component install was injected with other characters like }; during the network transfer.
I thought it might be a network problem in China, so I backup the components folder every time before I run make clean.

[timoxley]

@chemzqm @domachine try 0.16.5

[domachine]

0.16.5 has this nasty bug with superagent. see #364.

[yields]

i'm using node 0.10.12, component 0.16.7 and it happened to me only once. i don't remember what i did, probably make clean build.

[domachine]

we tried it on 2 VMs now and it's all the same. component fetches the files: some are fine but many are corrupted and cause syntax errors in the build.js file. any ideas?

[ianstormtaylor]

this happened to me yesterday as well for about an hour, not sure what i did either, but it disappeared again.

[tj]

stream all the things. will look at this tonight

[ianstormtaylor]

nvm, it's pretty regular now. happens a lot to me with component/event and component/classes, although that could easily just be in my case, downloading ~40 components into /components

component 0.16.7
node 0.8.8

[pgherveou]

Same here, notice the same things with component/event and
component/classes

On Mon, Jul 15, 2013 at 7:12 PM, Ian Storm Taylor
notifications@github.comwrote:

nvm, it's pretty regular now. happens a lot to me with component/eventand
component/classes, although that could easily just be in my case,
downloading ~40 components into /components

component 0.16.7
node 0.8.8

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/374#issuecomment-21017139
.

PG

[domachine]

@ianstormtaylor the same for me. it's not really usable. i downgraded to 0.16.5 with superagent 0.10.0.

[lancejpollard]

Getting this too, it seems pretty random. Sometimes it injects extra characters in one place, sometimes in another place, sometimes not. It seems to happen at roughly the same few points in the code though.

Any updates?

[yields]

@viatropos what versions of node, component are you using.
i'm trying to get this to happen to me again so i can see what's going on.

[lancejpollard]

@yields component v0.16.7, node v0.10.4

[lancejpollard]

Haven't had much time to dig in, but seems like it's probably happening because of the this.piped in superagent:

https://github.com/visionmedia/superagent/blob/f25d32432cf04d992f65e82187ab002d65444c5f/lib/node/index.js#L691-L696

    if (self.piped) {
      res.on('end', function(){
        self.emit('end');
      });
      return;
    }

https://github.com/visionmedia/superagent/blob/f25d32432cf04d992f65e82187ab002d65444c5f/lib/node/index.js#L399-L410

Request.prototype.pipe = function(stream, options){
  this.piped = true; // HACK...
  this.buffer(false);
  this.end().req.on('response', function(res){
    if (/^(deflate|gzip)$/.test(res.headers['content-encoding'])) {
      res.pipe(zlib.createUnzip()).pipe(stream, options);
    } else {
      res.pipe(stream, options);
    }
  });
  return stream;
};

console.logging some when the streams each close shows that the first block of code executes before the gzip stream is finished. It seems that the end shouldn't be called until the gzip finishes. This could explain why only part of the file is written (seemingly random, but most of the time at the same position in the file). Haven't used the latest streams much yet tho, so might be missing something.

Seems like maybe it needs to be using the close event instead of end? Or is there a bigger problem with node.js streams in general?

[lancejpollard]

following @domachine's suggestion solved the problem, good temporary workaround:

$ npm uninstall component -g
$ npm install component@0.16.5 -g
$ cd /usr/local/share/npm/lib/node_modules/component
$ npm install superagent@0.10.0 -S

[tj]

im just going to remove the stream bullshit from superagent, I'll patch this pretty soon with regular http/https