FEATURE: Ability to prevent resources from being loaded. #10230

Closed
morganrallen opened this Issue Sep 14, 2011 · 28 comments

Comments

morganrallen@gmail.com commented:

The desire here is to be abel to cancel a resource from being loaded during page.onResourceRequested something like

page.onResourceRequested = function(request) {
if(request.url.indexOf("run.js") > -1) {
return false; // or request.cancel() or request.abort();
}
}

Disclaimer:
This issue was migrated on 2013-03-15 from the project's former issue tracker on Google Code, Issue #230.
🌟   37 people had starred this issue at the time of migration.

Owner

ariya commented Sep 14, 2011

ariya.hi...@gmail.com commented:

 

 
Metadata Updates

  • Label(s) removed:
    • Type-Defect
  • Label(s) added:
    • Type-Enhancement

xavi.beu...@gmail.com commented:

this feature would be very helpful to minimize load times and start performing tasks earlier (i.e. avoiding Ads loading times, certaing .js , etc...)

It would be a fantastic feature

Owner

ariya commented Jan 8, 2012

ariya.hi...@gmail.com commented:

Issue 339 has been merged into this issue.

iwek commented Jun 7, 2012

iwokadzi...@gmail.com commented:

Hoping this gets added soon.

Owner

ariya commented Jun 18, 2012

olife...@gmail.com commented:

This would be very helpful!

Owner

ariya commented Aug 10, 2012

blademcc...@gmail.com commented:

whats the workaround for this?

fritzfs commented Aug 10, 2012

frit...@gmail.com commented:

At this moment there's only one way, modify the createRequest function in networkaccessmanager.cpp and rebuild phantomjs. Take a look at this example, http://stackoverflow.com/questions/4575245/how-to-tell-qwebpage-not-to-load-specific-type-of-resources.
Note: I'm not a phantomjs guru so perhaps someone else can comment this.

Owner

ariya commented Aug 16, 2012

blademcc...@gmail.com commented:

... what kind of donation to the project could get this as a high priority item? i might be able to arrange something.

Rebolon commented Sep 5, 2012

richard....@gmail.com commented:

I also think that it's a really good feature. Actually event like onResourceRequested is only usefull for logging (or other features). But it would be interesting ot disable some request (javascript that load advertising and that may break the testsuite...).

Owner

ariya commented Sep 10, 2012

nicefive...@gmail.com commented:

waiting for this feature!

Owner

ariya commented Oct 17, 2012

jonas.ge...@googlemail.com commented:

Yes, that would be nice.

I have created an small patch against PhantomJS for me and added an option --ignore-host='(google.com|facebook.com)'

So PhantomJS will not load any resources from hosts that matches the above regexp.

Owner

ariya commented Oct 20, 2012

seve...@gmail.com commented:

I would love this feature too. J

Owner

ariya commented Nov 29, 2012

jonas.ge...@googlemail.com commented:

My hacky patch for that:

https://github.com/hggh/phantomjs/compare/ignore-host.patch

If you apply that patch, you can use this:

--ignore-host='(google.com|twitter.com)'

That's perfect for Nagios/Icinga checks with https://github.com/hggh/phantomjs-nagios to ignore social media stuff or tracking.

Owner

ariya commented Dec 1, 2012

ariya.hi...@gmail.com commented:

Issue 395 has been merged into this issue.

Collaborator

JamesMGreene commented Dec 9, 2012

james.m....@gmail.com commented:

Issue 288 has been merged into this issue.

Owner

ariya commented Jan 5, 2013

ariya.hi...@gmail.com commented:

Allow to abort network requests.
ariya/phantomjs@09e929d

 
Metadata Updates

  • Label(s) added:
    • Component-Logic
    • Module-WebPage
  • Milestone updated: Release1.9 (was: ---)
  • Status updated: Fixed

ariya closed this Jan 5, 2013

Owner

ariya commented Jan 5, 2013

ariya.hi...@gmail.com commented:

For completeness, there is a new example loadurlwithoutcss.js which demonstrates this.

The gist:

page.onResourceRequested = function(requestData, request) {
if ((/http://.+?.css/gi).test(requestData['url']) || requestData['Content-Type'] == 'text/css') {
console.log('The url of the request is matching. Aborting: ' + requestData['url']);
request.abort();
}
};

Collaborator

JamesMGreene commented Jan 5, 2013

james.m....@gmail.com commented:

 

 
Metadata Updates

Collaborator

JamesMGreene commented Jan 5, 2013

james.m....@gmail.com commented:

For posterity, related discussion of new API: https://groups.google.com/d/topic/phantomjs/0OnJPkDAqSk/discussion

Collaborator

JamesMGreene commented Jan 10, 2013

james.m....@gmail.com commented:

Vitaliy:
Is it possible to CHANGE the URL at that point rather than just abort the request? I know that is desired, too, e.g. Issue #539 (read through the initial comment).

I also just looked at the first example for Qt 5.0 WebKit and saw it is doing just that (but with the QWebView class, which is higher level than what PhantomJS does): http://qt-project.org/doc/qt-5.0/qtwebkit/qtwebkit-index.html

morganrallen@gmail.com commented:

This was my original desired end result also but I was thinking along the
lines of aborting the original request and adding a new script tag for the
new resource wanted.

http://ithoughtyouweretherobot.com
Metal and Wire

http://nolonelyguineapigs.com/
Wandering and Rambling

http://morglog.org
Old and Neglected.

Owner

ariya commented Jan 10, 2013

ariya.hi...@gmail.com commented:

I don't see anything in http://qt-project.org/doc/qt-5.0/qtwebkit/qtwebkit-index.html which says something about changing the URL. You may have misunderstood the "delegate" term there, it's not about redirecting the URL but it's rather handling the functionality differently.

Collaborator

JamesMGreene commented Jan 10, 2013

james.m....@gmail.com commented:

Ah, yeah, that was indeed how I interpreted it. Drat....

In reality, though, in Qt, we can create and connect arbitrary signals, right? If not, it seems like we could easily generate a signal before we call [or at the start inside of?] NetworkAccessManager#createRequest and connect another on____ signal handler for that in the WebPage object. Am I totally off-base here? I recall Vitaliy mentioned something about not being able to appropriately stop/abort a request before it is issued... if that's the case, then we'd probably have to short-circuit such requests BEFORE the createRequest method is invoked.

Collaborator

JamesMGreene commented Jan 11, 2013

james.m....@gmail.com commented:

What if we implemented a custom QNetworkReply and set it to a terminal status in the createRequest method e.g. the first code sample on this page:
https://blogs.kde.org/2010/08/28/implementing-reusable-custom-qnetworkreply

I know an HTTP error status is not the same as aborting the request (reply) but I'd probably still prefer it for network performance over the current solution. Unless the reply object can be aborted AND returned in the createRequest method...?

Collaborator

JamesMGreene commented Feb 1, 2013

james.m....@gmail.com commented:

Vitaliy/Ariya, any thoughts on that?

Collaborator

Vitallium commented Feb 1, 2013

vitaliy....@gmail.com commented:

I'm still looking for solution, but no luck.

Collaborator

JamesMGreene commented Feb 2, 2013

james.m....@gmail.com commented:

Otherwise, seems like the method in the SO answer linked to earlier by Fritz (Comment #7) would work for both "aborting" (with an empty QUrl) or redirecting:
http://stackoverflow.com/questions/4575245/how-to-tell-qwebpage-not-to-load-specific-type-of-resources/4588113#4588113

ariya was assigned Mar 15, 2013

arlsr commented Dec 16, 2013

Usage

PhantomJS >= 1.9

// Abort all requests for URLs starting 'http://example.com/'

page.onResourceRequested = function(request, net) {
    if (request.url.indexOf('http://example.com/') === 0)
        net.abort();
};

https://github.com/ariya/phantomjs/wiki/API-Reference-WebPage

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment