More sophisticated adblocking

[The-Compiler]

There should be an adblocker.

edit: A host-based adblocker which is sufficient for most cases is implemented since a long time already, this issue is still open because it could be improved by implementing ABP-like filterlist support.

The eric IDE helpviewer does have one (in Python), which might be a good inspiration

[The-Compiler]

Before we have a real blocker we could use http://www.floppymoose.com/ in the default user-stylesheet. We probably should check the license though.

[The-Compiler]

Before we have a real blocker we could use http://www.floppymoose.com/ in the default user-stylesheet. We probably should check the license though.

[The-Compiler]

https://github.com/foxhead128/nimbus also does adblocking which might be another inspiration

[The-Compiler]

https://github.com/foxhead128/nimbus also does adblocking which might be another inspiration

[The-Compiler]

We should implement basic adblocking by downloading hosts-files and merging them.

Sources:

• http://www.malwaredomainlist.com/hostslist/hosts.txt
• http://someonewhocares.org/hosts/hosts
• http://winhelp2002.mvps.org/hosts.txt
• http://malwaredomains.lehigh.edu/files/justdomains
• http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&mimetype=plaintext
• http://hosts-file.net/download/hosts.zip (wait for confirmation if using is okay)

Algorithm for parsing/merging/etc:

• https://raw.githubusercontent.com/StevenBlack/hosts/master/updateHostsFile.py
• should be able to read zip files
• should be able to read files with just domains

User interface:

• Setting permissions->adblock-host-lists with a comma-separated list of URLs
• Command :update-adblock which downloads all lists (see Qt example) and merges them

[The-Compiler]

We should implement basic adblocking by downloading hosts-files and merging them.

Sources:

• http://www.malwaredomainlist.com/hostslist/hosts.txt
• http://someonewhocares.org/hosts/hosts
• http://winhelp2002.mvps.org/hosts.txt
• http://malwaredomains.lehigh.edu/files/justdomains
• http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&mimetype=plaintext
• http://hosts-file.net/download/hosts.zip (wait for confirmation if using is okay)

Algorithm for parsing/merging/etc:

• https://raw.githubusercontent.com/StevenBlack/hosts/master/updateHostsFile.py
• should be able to read zip files
• should be able to read files with just domains

User interface:

• Setting permissions->adblock-host-lists with a comma-separated list of URLs
• Command :update-adblock which downloads all lists (see Qt example) and merges them

[The-Compiler]

host-file adblocking has been implemented in 0134646 - leaving this open for more sophisticated blocking

[The-Compiler]

host-file adblocking has been implemented in 0134646 - leaving this open for more sophisticated blocking

[HalosGhost]

I would recommend µblock as a model to use. It aims to be much lighter and faster than ABP while actually extending some functionality.

[HalosGhost]

I would recommend µblock as a model to use. It aims to be much lighter and faster than ABP while actually extending some functionality.

[1-61803]

Is there any ad-blocking implementation as of now, internal or via plugin?

[1-61803]

Is there any ad-blocking implementation as of now, internal or via plugin?

[parchd-1]

Yes, there is host-file adblocking, as you can see above.
The plugin system isn't done yet, so other adblock systems aren't either. You can also see that in the issue linked to above ;).

[parchd-1]

Yes, there is host-file adblocking, as you can see above.
The plugin system isn't done yet, so other adblock systems aren't either. You can also see that in the issue linked to above ;).

[1-61803]

@parchd-1 I skimmed. I see the hosts file implementation from 2014 and #30 has adblock stroke through. What is the actual state of development of plugins that would allow better ad blocking but also plugins like ublock and greasemonkey?

[1-61803]

@parchd-1 I skimmed. I see the hosts file implementation from 2014 and #30 has adblock stroke through. What is the actual state of development of plugins that would allow better ad blocking but also plugins like ublock and greasemonkey?

[The-Compiler]

What is the actual state of development of plugins

There isn't anything other than the thoughts listed in #30 so far.

that would allow better ad blocking but also plugins like ublock

What from ublock are you missing, other than having adblockplus-like block lists?

and greasemonkey

Something similar to it should be possible with per-domain settings (#27) which is the current main focus - in around a week I'll launch qutebrowser's second crowdfunding, to work on this (and maybe start some work on plugins if there's time left) during my summer holidays. Subscribe to that issue if you want to get notified once it's up!

[The-Compiler]

What is the actual state of development of plugins

There isn't anything other than the thoughts listed in #30 so far.

that would allow better ad blocking but also plugins like ublock

What from ublock are you missing, other than having adblockplus-like block lists?

and greasemonkey

Something similar to it should be possible with per-domain settings (#27) which is the current main focus - in around a week I'll launch qutebrowser's second crowdfunding, to work on this (and maybe start some work on plugins if there's time left) during my summer holidays. Subscribe to that issue if you want to get notified once it's up!

[1-61803]

What from ublock are you missing, other than having adblockplus-like block lists?

I just have switched to ublock as ABP caused high CPU usage. Having the same lists makes rendering sites equally across hosts. Just a first impression after switching blocker.

I still have to make it work in my main older machine on 10.8. I encountered the same problem pointed out here. It should compile from source, it's python and qt, right?

Something similar to it should be possible with per-domain settings (#27) which is the current main focus - in around a week I'll launch qutebrowser's second crowdfunding

Subscribed. What's the best way to get regular (weekly but not daily) news? If I compile it in my 10.8 I'll be certainly donating to your project.

[1-61803]

What from ublock are you missing, other than having adblockplus-like block lists?

I just have switched to ublock as ABP caused high CPU usage. Having the same lists makes rendering sites equally across hosts. Just a first impression after switching blocker.

I still have to make it work in my main older machine on 10.8. I encountered the same problem pointed out here. It should compile from source, it's python and qt, right?

Something similar to it should be possible with per-domain settings (#27) which is the current main focus - in around a week I'll launch qutebrowser's second crowdfunding

Subscribed. What's the best way to get regular (weekly but not daily) news? If I compile it in my 10.8 I'll be certainly donating to your project.

[The-Compiler]

I still have to make it work in my main older machine on 10.8. I encountered the same problem pointed out here. It should compile from source, it's python and qt, right?

There's nothing in qutebrowser itself you'll need to compile, as it's Python. You'll need a working Python with PyQt though. If you can get a Python 3.5 or 3.6 installed (maybe via Homebrew or pyenv), you should be able to do something like pip install PyQt5 qutebrowser (optionally in a virtualenv). If you need help, open a separate issue though, please.

What's the best way to get regular (weekly but not daily) news?

There isn't anything like a regular newsletter. You can look at the changelog and github activity manually though.

I'd also recommend subscribing to the qutebrowser-announce mailinglist where important announcements like new releases are posted.

[The-Compiler]

I still have to make it work in my main older machine on 10.8. I encountered the same problem pointed out here. It should compile from source, it's python and qt, right?

There's nothing in qutebrowser itself you'll need to compile, as it's Python. You'll need a working Python with PyQt though. If you can get a Python 3.5 or 3.6 installed (maybe via Homebrew or pyenv), you should be able to do something like pip install PyQt5 qutebrowser (optionally in a virtualenv). If you need help, open a separate issue though, please.

What's the best way to get regular (weekly but not daily) news?

There isn't anything like a regular newsletter. You can look at the changelog and github activity manually though.

I'd also recommend subscribing to the qutebrowser-announce mailinglist where important announcements like new releases are posted.

[asmbits]

Is there any progress? It is the only reason I haven't switched to qutebrowser yet. I cannot imagine seeing ads while browsing google, youtube or any other site.

[asmbits]

Is there any progress? It is the only reason I haven't switched to qutebrowser yet. I cannot imagine seeing ads while browsing google, youtube or any other site.

[The-Compiler]

@asmbits I don't think anyone is working on this currently, as the current adblocker is working well enough overall and there are lots and lots of other more important issues 😉

[The-Compiler]

@asmbits I don't think anyone is working on this currently, as the current adblocker is working well enough overall and there are lots and lots of other more important issues 😉

[sagb]

On the sites where ads come not from standard CDNs the current adblock is useless. This major bug is very annoying.

[sagb]

On the sites where ads come not from standard CDNs the current adblock is useless. This major bug is very annoying.

[The-Compiler]

You seem to have an... interesting understanding of what "major bug" means. I'd be glad if someone picked this up, but it won't be me in the near future, I have lots of other stuff on my plate. Are you volunteering? 😉

[The-Compiler]

You seem to have an... interesting understanding of what "major bug" means. I'd be glad if someone picked this up, but it won't be me in the near future, I have lots of other stuff on my plate. Are you volunteering? 😉

[sagb]

Sorry, it's beyond my skills and available time.

[sagb]

Sorry, it's beyond my skills and available time.

[CosmoJG]

@The-Compiler Would it be alright if myself or someone else tried directly integrating some of uBlock's code? Or would you rather have something built from scratch?

I think direct integration would do wonders for preventing the classic open-source effort-reduplication problem.

[CosmoJG]

@The-Compiler Would it be alright if myself or someone else tried directly integrating some of uBlock's code? Or would you rather have something built from scratch?

I think direct integration would do wonders for preventing the classic open-source effort-reduplication problem.

[The-Compiler]

@CosmoJG I don't see how that'd be possible (as no WebExtension API exists), so the point is moot 😉

[The-Compiler]

@CosmoJG I don't see how that'd be possible (as no WebExtension API exists), so the point is moot 😉

[CosmoJG]

@The-Compiler Ahh, admittedly, I haven't put much thought into this, but I wasn't talking about including it as an extension. I meant I'd try and directly integrate the parts of uBlock Origin that make it different from Qutebrowser's built-in adblock. I'm operating on the assumption that there are code bits within Qutebrowser that perform similar functions to whatever WebExtension API calls uBlock makes.

Maybe when I have less work later in the week, I'll get off my ass (and stop talking out of it) and dig through Qutebrowser's core adblock to see what kinds of original improvements can be made. Granted, I've never written code for adblocking before and only have a basic understanding of how most of it works, but that doesn't mean I can't try!

[CosmoJG]

@The-Compiler Ahh, admittedly, I haven't put much thought into this, but I wasn't talking about including it as an extension. I meant I'd try and directly integrate the parts of uBlock Origin that make it different from Qutebrowser's built-in adblock. I'm operating on the assumption that there are code bits within Qutebrowser that perform similar functions to whatever WebExtension API calls uBlock makes.

Maybe when I have less work later in the week, I'll get off my ass (and stop talking out of it) and dig through Qutebrowser's core adblock to see what kinds of original improvements can be made. Granted, I've never written code for adblocking before and only have a basic understanding of how most of it works, but that doesn't mean I can't try!

[jgkamat]

I too have little idea how adblocking (and js in general) work, but I think a good starting point for this is to see how greasemonkey userscripts are injected into the page (and stylesheets as well). Once you get a script in the page, it becomes a js issue, which I have almost zero experience with :P

I don't think the current host-blocking adblock code is a good place to start (because this would probably need a completely different approach), but I think it's mostly located here.

[jgkamat]

I too have little idea how adblocking (and js in general) work, but I think a good starting point for this is to see how greasemonkey userscripts are injected into the page (and stylesheets as well). Once you get a script in the page, it becomes a js issue, which I have almost zero experience with :P

I don't think the current host-blocking adblock code is a good place to start (because this would probably need a completely different approach), but I think it's mostly located here.

[The-Compiler]

You won't be able to block network requests with JS injected in the page - there's stuff in the WebExtension API to do that which uBlock uses, but doing the same in qutebrowser won't be possible. In qutebrowser, JS injected into the page can only do whatever the page itself can do, and it has the same JS API a webpage can use (and not more).

Adblocking for qutebrowser really has to be implemented in Python - JavaScript won't work without having relevant APIs in QtWebEngine which probably won't be happening in the near future.

So, all you'll be able to take from uBlock is some general ideas and concepts (i.e. the "big picture"), and the filter list syntax (which is mostly coming from AdBlockPlus) - but not any actual code.

[The-Compiler]

You won't be able to block network requests with JS injected in the page - there's stuff in the WebExtension API to do that which uBlock uses, but doing the same in qutebrowser won't be possible. In qutebrowser, JS injected into the page can only do whatever the page itself can do, and it has the same JS API a webpage can use (and not more).

Adblocking for qutebrowser really has to be implemented in Python - JavaScript won't work without having relevant APIs in QtWebEngine which probably won't be happening in the near future.

So, all you'll be able to take from uBlock is some general ideas and concepts (i.e. the "big picture"), and the filter list syntax (which is mostly coming from AdBlockPlus) - but not any actual code.

[The-Compiler]

FWIW webmacs seems to use the brave/ad-block module from Python.

[The-Compiler]

FWIW webmacs seems to use the brave/ad-block module from Python.

[rr-]

The built-in filters seem to be less effective with each month, despite :adblock-update :(

[rr-]

The built-in filters seem to be less effective with each month, despite :adblock-update :(

[Yodzorah]

1. What about requests blocking via our two backends? Do they support domain patterns? And if not, what about dnsmasq?

2. uBlock, AdBlock, etc. suck. Sorry, but it's true. They work on similar rulesets for request blocking (with patterns, though), css and js and they are centered on just removing ads. They have full control over files, scripts, etc., they have full functionality for full control over the mentioned stuff, but instead of providing the control to user they just block stuff according to rulesets. It's just a pointless waste of resources. Look at ContentBlockHelper and uMatrix - they are much better. In qutebrowser we have JS, CSS and hosts blocklists. Also, we probably have most of the features of ContentBlockHelper/uMatrix in QtWebEngine, we just need to wait for per domain settings. I think, if there is any chance we can use more simple and powerful solutions instead of emulating familiar to many users but grotesque and ailing adblokers, we should think trice before choosing the second option.

3. Look at the ContentBlockHelper UI. I mean, not the design, it's awful, but the tree view of every loaded file for every requested website. What about something like that in qutebrowser? For example, we have ':set' mode, can we have the same for css, js, hosts, cookies, etc. settings in "per domain" mode? Not just with a sublist of settings but with the website tree?

[Yodzorah]

1. What about requests blocking via our two backends? Do they support domain patterns? And if not, what about dnsmasq?

2. uBlock, AdBlock, etc. suck. Sorry, but it's true. They work on similar rulesets for request blocking (with patterns, though), css and js and they are centered on just removing ads. They have full control over files, scripts, etc., they have full functionality for full control over the mentioned stuff, but instead of providing the control to user they just block stuff according to rulesets. It's just a pointless waste of resources. Look at ContentBlockHelper and uMatrix - they are much better. In qutebrowser we have JS, CSS and hosts blocklists. Also, we probably have most of the features of ContentBlockHelper/uMatrix in QtWebEngine, we just need to wait for per domain settings. I think, if there is any chance we can use more simple and powerful solutions instead of emulating familiar to many users but grotesque and ailing adblokers, we should think trice before choosing the second option.

3. Look at the ContentBlockHelper UI. I mean, not the design, it's awful, but the tree view of every loaded file for every requested website. What about something like that in qutebrowser? For example, we have ':set' mode, can we have the same for css, js, hosts, cookies, etc. settings in "per domain" mode? Not just with a sublist of settings but with the website tree?

[The-Compiler]

@Yodzorah

1. See #4188, though real adblock lists would also be patterns. Not sure what you're asking about dnsmasq.

2. You're free to have your opinion, but not everyone wants to spend time configuring rules on every website they visit. I'd like to have both in the long run.

3. That sounds quite similar to uMatrix, see #28 (and also #2626)

[The-Compiler]

@Yodzorah

1. See #4188, though real adblock lists would also be patterns. Not sure what you're asking about dnsmasq.

2. You're free to have your opinion, but not everyone wants to spend time configuring rules on every website they visit. I'd like to have both in the long run.

3. That sounds quite similar to uMatrix, see #28 (and also #2626)

[Yodzorah]

@The-Compiler,

2. My option doesn't require users to configure websites all the time in order to get rid from ads. It can do both with the functionality of usual adblockers, while the usual adblockers with functionality of usual adblockers can only block ads. I just don't think it's a good idea to have such highly specialized tool that, in fact, can do more really useful stuff with the same 'effort'. If one will add +10% of functionality to any popular adbloker, he will have far more powerfull and flexible tool and far more control over web. And if one will use usual 'simple' adblockers and suddenly understand he wants to block something else, he will ask for another highly specialized tool instead of something more flexible for both tasks. It's like installing separate extensions changing CSS for some websites instead of installing Stylish in Chrome.

Sorry for asking, but if qutebrowser is so centered on providing user full control, why prefer highly specialized solutions over flexible? I mean, not everyone even knows about user agent header, why such setting even exists in qutebrowser then?

[Yodzorah]

@The-Compiler,

2. My option doesn't require users to configure websites all the time in order to get rid from ads. It can do both with the functionality of usual adblockers, while the usual adblockers with functionality of usual adblockers can only block ads. I just don't think it's a good idea to have such highly specialized tool that, in fact, can do more really useful stuff with the same 'effort'. If one will add +10% of functionality to any popular adbloker, he will have far more powerfull and flexible tool and far more control over web. And if one will use usual 'simple' adblockers and suddenly understand he wants to block something else, he will ask for another highly specialized tool instead of something more flexible for both tasks. It's like installing separate extensions changing CSS for some websites instead of installing Stylish in Chrome.

Sorry for asking, but if qutebrowser is so centered on providing user full control, why prefer highly specialized solutions over flexible? I mean, not everyone even knows about user agent header, why such setting even exists in qutebrowser then?

[The-Compiler]

I'm not sure what you're trying to achieve with this discussion - like I said above I'd like to have both, as both solutions have their benefits and drawbacks. This is not the right place to discuss whether adblockers are a good thing or not.

[The-Compiler]

I'm not sure what you're trying to achieve with this discussion - like I said above I'd like to have both, as both solutions have their benefits and drawbacks. This is not the right place to discuss whether adblockers are a good thing or not.

[Yodzorah]

@The-Compiler, I was trying to convince you to not try usual adblockers until you will be sure there is no flexible solution) Maybe, it was not wise, but I should have to try at least - I'm using qutebrowser because of its flexibility, so if this won't be the case, I will lose even a half-comfortable browser and will be forced to migrate on Vivaldi, because with the same low level of flexibility it at least has a big pack of features.

[Yodzorah]

@The-Compiler, I was trying to convince you to not try usual adblockers until you will be sure there is no flexible solution) Maybe, it was not wise, but I should have to try at least - I'm using qutebrowser because of its flexibility, so if this won't be the case, I will lose even a half-comfortable browser and will be forced to migrate on Vivaldi, because with the same low level of flexibility it at least has a big pack of features.

[toofar]

There is no plan for qutebrowser to support the usual adblockers. See #2626 for a discussion about a flexible request blocking framework and #28 for a prettier UI over that.

[toofar]

There is no plan for qutebrowser to support the usual adblockers. See #2626 for a discussion about a flexible request blocking framework and #28 for a prettier UI over that.

[The-Compiler]

Note I've already linked those above, but whatever...

[The-Compiler]

Note I've already linked those above, but whatever...