Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

ReferenceError not being caught by page.onError and phantom.onError #11144

Open
malexmave opened this Issue Mar 17, 2013 · 18 comments

Comments

Projects
None yet

Consider the following code:

var page = require('webpage').create(),
    system = require('system'),
    address, set = {};
try {   

        if (system.args.length === 1) {
            console.log('Usage: netdomain.js <some URL>');
            phantom.exit(1);
        } else {
            address = system.args[1];

            page.onResourceReceived = function (res) {
                var data = JSON.stringify(res.url, undefined, 4).split("/")[2];
                if (~data.indexOf(".")) {  // Check if String contains "." (if not, it is a false positive)
                    if (data.substring(0,4) == "www.") { // Remove www., if it is there
                        data = data.substring(4);
                    }
                    set[data] = true; // Save data
                }
            };
            page.onError = function(err) {
            };
            phantom.onError = function(err) {
            };

            page.open(address, function (status) {
                if (status !== 'success') {
                    console.log('FAIL to load the address');
                    phantom.exit(1);
                }
                console.log(Object.keys(set).toString());
                phantom.exit();
            });
        }
        // END Modified netlog.js example
}
catch(err) {
        console.log(Object.keys(set).toString());
        phantom.exit();
}

It works well on many pages (almost 50 000 tried so far) but on very few pages (victoriassecret.com, hsamuel.co.uk and a few others, I can provide a full list, if needed) it will still throw errors that are not caught. The error type that still comes through is a ReferenceErrors ("Can't find variable: [varname]" on victoriassecret.com, for example).

This might be related to #10511, although the user over there theorized that phantom.onError should work for this, which it does not.

Please let me know if you need any further information.

Owner

ariya commented Mar 17, 2013

In the code example above, where did you dump that ReferenceError message?

On page.open(). If I comment out the whole onRessourceReceived-Block, it will still throw the error.

Owner

ariya commented Mar 17, 2013

I still don't understand. Do you mean that this line:

console.log(Object.keys(set).toString());

shows `ReferenceError: bla bla'?

No, the error is thrown even if I comment out all output.

var page = require('webpage').create(),
    system = require('system'),
    address, set = {};
try {

    if (system.args.length === 1) {
        console.log('Usage: netdomain.js <some URL>');
        phantom.exit(1);
    } else {
        address = system.args[1];

        page.onResourceReceived = function (res) {
            //var data = JSON.stringify(res.url, undefined, 4).split("/")[2];
        //if (~data.indexOf(".")) {  // Check if String contains "." (if not, it is a false positive)
            //if (data.substring(0,4) == "www.") { // Remove www., if it is there
        //  data = data.substring(4);
          //  }
            //set[data] = true; // Save data
        //}
        };
        page.onError = function(err) {
        };
        phantom.onError = function(err) {
        };

        page.open(address, function (status) {
            if (status !== 'success') {
                console.log('FAIL to load the address');
            phantom.exit(1);
            }
        //console.log(Object.keys(set).toString());
            phantom.exit();
        });
    }
    // END Modified netlog.js example
}
catch(err) {
    console.log(Object.keys(set).toString());
    phantom.exit();
}
Owner

ariya commented Mar 17, 2013

Still not clear at all. Please provide:

  • an example URL
  • a full error message

so that it can be reproduced.

example URL is victoriassecret.com, full Error message:

ReferenceError: Can't find variable: jQuery

  http://www.victoriassecret.com/themes/base/candice/js/plugins-bottom.js:11
ReferenceError: Can't find variable: VSD

  http://www.victoriassecret.com/themes/base/candice/js/email-signup.js:77
ReferenceError: Can't find variable: VSD

  http://www.victoriassecret.com/themes/base/candice/js/email-signup.js:45
  http://www.victoriassecret.com/themes/base/js/jquery.js:26
  http://www.victoriassecret.com/themes/base/js/jquery.js:33

Owner

ariya commented Mar 18, 2013

Thanks for the error message, that definitely gives some insight.

You can add another uncaught error to this list:
SyntaxError: Parse Error appears twice on http://alwazer.com, with no further data being displayed.

Contributor

davidrapin commented Jun 6, 2014

Any news on this one?
Same problem here. I have phantom.onError and page.onError callbacks set up, but I still get this:

ReferenceError: Can't find variable: jQuery
  http://newsletters.francetv.fr/cnil/js/cnil-min-v1.2.1.js:1
SyntaxError: Parse error

I get this on http://france2.fr when triggering some JS callbacks after onLoad.

jorisd commented Nov 10, 2014

Bump.

I'm getting same errors, while scraping https://m.facebook.com/trader.games?v=feed

Bump.

The common denominator seems to be a colon (:) in URLs. I have the same problem when scraping URLs like msn.com or tumblr.com. They use colons (:) in their URLs (eg. "http://www.msn.com/de-de/:926"), which seems to cause the ReferenceError. Afaik colon characters are valid in HTTP URLs.

hansod1 commented Nov 11, 2014

(I appear to be having this issue as well, PhantomJS 1.9.7, OSX 10.10)

Those colons are printed by PhantomJS to tell you where the Javascript error is taking place and are not indicative of the URL that PhantomJS is using to request the resource. If you log the requests using onResourceRequested, you'll see that there are no colons in the URLs.

Still, I'm also having this issue on tumblr.com, phantomJS prints several errors to std out and doesn't load the page properly. These errors do not occur when I visit the page using another browser. Any ideas would be appreciated.

It probably goes without saying, but I also needed to pass in --ssl-protocol=any in order to load the page, since tumblr uses a different version than the phantomJS default.

Here's an example of the errors printed by phantomJS after loading up tumblr.com:

TypeError: instanceof called on an object with an invalid prototype property.

  https://secure.assets.tumblr.com/assets/scripts/polyfills.js?_v=bcc832ec0df74eda743c7f9c36d71cde:2
  https://secure.assets.tumblr.com/client/prod/app/vendor/index.js?_v=79938966b2144678dd0dc50bd03b8cc5:3
  https://secure.assets.tumblr.com/client/prod/app/vendor/index.js?_v=79938966b2144678dd0dc50bd03b8cc5:3
  https://secure.assets.tumblr.com/client/prod/app/vendor/index.js?_v=79938966b2144678dd0dc50bd03b8cc5:3
  https://secure.assets.tumblr.com/client/prod/app/vendor/index.js?_v=79938966b2144678dd0dc50bd03b8cc5:3
  https://secure.assets.tumblr.com/client/prod/app/vendor/index.js?_v=79938966b2144678dd0dc50bd03b8cc5:3
  https://secure.assets.tumblr.com/client/prod/app/vendor/index.js?_v=79938966b2144678dd0dc50bd03b8cc5:3
  https://secure.assets.tumblr.com/client/prod/app/vendor/index.js?_v=79938966b2144678dd0dc50bd03b8cc5:3
  https://secure.assets.tumblr.com/client/prod/app/vendor/index.js?_v=79938966b2144678dd0dc50bd03b8cc5:3
  https://secure.assets.tumblr.com/client/prod/app/vendor/index.js?_v=79938966b2144678dd0dc50bd03b8cc5:13
  https://secure.assets.tumblr.com/client/prod/app/vendor/index.js?_v=79938966b2144678dd0dc50bd03b8cc5:3
  https://secure.assets.tumblr.com/client/prod/app/vendor/index.js?_v=79938966b2144678dd0dc50bd03b8cc5:3
  https://secure.assets.tumblr.com/client/prod/app/vendor/index.js?_v=79938966b2144678dd0dc50bd03b8cc5:3
  https://secure.assets.tumblr.com/client/prod/app/vendor/index.js?_v=79938966b2144678dd0dc50bd03b8cc5:13
  https://secure.assets.tumblr.com/client/prod/app/vendor/index.js?_v=79938966b2144678dd0dc50bd03b8cc5:13
  https://secure.assets.tumblr.com/client/prod/app/vendor/index.js?_v=79938966b2144678dd0dc50bd03b8cc5:1 in i
  https://secure.assets.tumblr.com/client/prod/app/vendor/index.js?_v=79938966b2144678dd0dc50bd03b8cc5:1
  https://secure.assets.tumblr.com/client/prod/app/vendor/index.js?_v=79938966b2144678dd0dc50bd03b8cc5:1
  https://secure.assets.tumblr.com/client/prod/app/vendor/index.js?_v=79938966b2144678dd0dc50bd03b8cc5:1 in i
  https://secure.assets.tumblr.com/client/prod/app/context/loginandregister/index.js?_v=413274a83325123eea0075a954c00cf0:1 in o
  https://secure.assets.tumblr.com/client/prod/app/context/loginandregister/index.js?_v=413274a83325123eea0075a954c00cf0:1 in o
  https://secure.assets.tumblr.com/client/prod/app/context/loginandregister/index.js?_v=413274a83325123eea0075a954c00cf0:1
  https://secure.assets.tumblr.com/client/prod/app/context/loginandregister/index.js?_v=413274a83325123eea0075a954c00cf0:13
  https://secure.assets.tumblr.com/client/prod/app/context/loginandregister/index.js?_v=413274a83325123eea0075a954c00cf0:1 in o
  https://secure.assets.tumblr.com/client/prod/app/context/loginandregister/index.js?_v=413274a83325123eea0075a954c00cf0:1
  https://secure.assets.tumblr.com/client/prod/app/context/loginandregister/index.js?_v=413274a83325123eea0075a954c00cf0:1
  https://secure.assets.tumblr.com/client/prod/app/context/loginandregister/index.js?_v=413274a83325123eea0075a954c00cf0:16
  https://secure.assets.tumblr.com/client/prod/app/vendor/index.js?_v=79938966b2144678dd0dc50bd03b8cc5:2
  https://secure.assets.tumblr.com/client/prod/app/context/loginandregister/index.js?_v=413274a83325123eea0075a954c00cf0:17
  https://www.tumblr.com/:379
TypeError: 'undefined' is not an object (evaluating 'Tumblr.Events.on')

  https://secure.assets.tumblr.com/assets/scripts/dashboard.js?_v=091bd8e4d0961fb5f205b3cd04477764:256
  https://secure.assets.tumblr.com/assets/scripts/dashboard.js?_v=091bd8e4d0961fb5f205b3cd04477764:256
TypeError: 'undefined' is not an object (evaluating 'Tumblr.Prima.DOMEventor')

  https://secure.assets.tumblr.com/assets/scripts/dashboard.js?_v=091bd8e4d0961fb5f205b3cd04477764:68
  https://secure.assets.tumblr.com/assets/scripts/dashboard.js?_v=091bd8e4d0961fb5f205b3cd04477764:174
  https://secure.assets.tumblr.com/client/prod/app/vendor/index.js?_v=79938966b2144678dd0dc50bd03b8cc5:2
  https://secure.assets.tumblr.com/client/prod/app/vendor/index.js?_v=79938966b2144678dd0dc50bd03b8cc5:2
  https://secure.assets.tumblr.com/client/prod/app/vendor/index.js?_v=79938966b2144678dd0dc50bd03b8cc5:2
  https://www.tumblr.com/:312
  https://secure.assets.tumblr.com/client/prod/app/vendor/index.js?_v=79938966b2144678dd0dc50bd03b8cc5:20
  https://secure.assets.tumblr.com/client/prod/app/vendor/index.js?_v=79938966b2144678dd0dc50bd03b8cc5:20
  https://secure.assets.tumblr.com/client/prod/app/vendor/index.js?_v=79938966b2144678dd0dc50bd03b8cc5:20
  https://secure.assets.tumblr.com/client/prod/app/vendor/index.js?_v=79938966b2144678dd0dc50bd03b8cc5:19

And here's a screenshot, note the missing login/signup form which is supposed to be in the very center:

tumblr

Also seeing this for TypeError error. Not being caught by phantomjs onerror.

example script that is nosense.
document.xxxx.indexOf(234).jerry;

I see a TypeError in the logs but OnError is not catching it.

arderyp commented Jan 13, 2015

I too am experiencing this issue with a few select sites. They are internally hosted sites, so there is no use in me providing the extra examples. But, if it helps, I have duplicated the issues with my specific examples in 1.9.1, 1.9.2, 1.9.7, 1.9.8. These are the only versions I currently have set up to test.

Mine is a "ReferenceError: Cant find variable" thrown by a javascript call in the webpage/

Yup, same issue here.

It's not capturing the following error message as an error:

ReferenceError: Can't find variable: MutationObserver

It is however displaying this when you set a onConsoleMessage, but there is no way of knowing if it's actually an error or not.

I am getting page.onError

[ERROR - 2016-02-29T15:24:34.204Z] Session [8948d0-def8-11e5-a2ac-398cdabea00d] - page.onError - msg: SyntaxError: Parse error
[ERROR - 2016-02-29T15:24:34.204Z] Session [8948d0-def8-11e5-a2ac-398cdabea00d] - page.onError - stack:
[WARNING] JavaScript Console Errors:

  • SyntaxError: Parse error

i don't have any console error while running my app or tests.

I figured issue is coming from jQuery extend in my case which i am using for adding functions into shared object inside closure.

$.extend(shared,{a,b,c,d});
I am not getting this error while i am updated extend to this.
shared.a = a;
shared.b= b;
...

I'm using jQuery 1.7.2. Jasmine 2.0 for tests and phantom version 1.9.8.

Any help appreciated.

Never mind. It seems phantom js doesn't have prototype which was used by jQuery extend in version 1.9.8.
#10522

swiezak commented Mar 28, 2016

Hello,
I don't know if this is the right place, but I get the following error when I enter for example this url
http://www.ihg.com/destinations/us/en/united-states/texas/texas-city-hotels

[ERROR - 2016-03-28T20:49:42.576Z] Session [02315ce0-f526-11e5-94b3-bda7c3a7e7f9] - page.onError - msg: ReferenceError: Can't find variable: TEXT_FIND_HOTEL

  phantomjs://platform/console++.js:263 in error
[ERROR - 2016-03-28T20:49:42.576Z] Session [02315ce0-f526-11e5-94b3-bda7c3a7e7f9] - page.onError - stack:
  (anonymous function) (http://www.ihg.com/destinations/us/en/wro/scripts-city-page.js?BUILD_TIME=1458232780943:1)

  phantomjs://platform/console++.js:263 in error
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment