encoding: null results in undefined body #42

Closed
quanghoc opened this Issue Feb 26, 2013 · 1 comment

Comments

Projects
None yet
2 participants

I am using this simple example and found a problem:

var Crawler = require("crawler").Crawler;

var c = new Crawler({
    "maxConnections":10,
    "callback":function(error,result,$) {
        console.log(result.body);
        //console.log($('body').text());
    }
});
c.queue("http://google.com");

In this particular line:

var req = request(_.pick.apply(this,[ropts].concat(requestArgs)), function(error,response,body) {

The input JSON was as following:

{ uri: 'http://google.com',
  method: 'GET',
  headers:
   { 'Accept-Encoding': 'gzip',
     'User-Agent': 'node-crawler/0.2.3' },
  encoding: null,
  timeout: 60000 }

As you see, I have to delete "encoding: null" in order for body variable to work.

Why is it "null" and how to fix this for websites like google.com?

The platform I am working on is Windows, btw (if that makes a different).

Collaborator

sylvinus commented Feb 26, 2013

Thanks for the report! I just pushed 0.2.4 which fixes this issue.

sylvinus closed this Feb 26, 2013

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment