Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

More sophisticated adblocking #29

Open
The-Compiler opened this Issue Oct 1, 2014 · 39 comments

Comments

Projects
None yet
@The-Compiler
Copy link
Collaborator

The-Compiler commented Oct 1, 2014

There should be an adblocker.

edit: A host-based adblocker which is sufficient for most cases is implemented since a long time already, this issue is still open because it could be improved by implementing ABP-like filterlist support.

The eric IDE helpviewer does have one (in Python), which might be a good inspiration

@The-Compiler The-Compiler self-assigned this Oct 1, 2014

@The-Compiler

This comment has been minimized.

Copy link
Collaborator Author

The-Compiler commented Oct 2, 2014

Before we have a real blocker we could use http://www.floppymoose.com/ in the default user-stylesheet. We probably should check the license though.

@The-Compiler

This comment has been minimized.

Copy link
Collaborator Author

The-Compiler commented Oct 2, 2014

https://github.com/foxhead128/nimbus also does adblocking which might be another inspiration

@The-Compiler

This comment was marked as outdated.

Copy link
Collaborator Author

The-Compiler commented Nov 13, 2014

We should implement basic adblocking by downloading hosts-files and merging them.

Sources:

Algorithm for parsing/merging/etc:

User interface:

  • Setting permissions->adblock-host-lists with a comma-separated list of URLs
  • Command :update-adblock which downloads all lists (see Qt example) and merges them
@The-Compiler

This comment has been minimized.

Copy link
Collaborator Author

The-Compiler commented Dec 3, 2014

host-file adblocking has been implemented in 0134646 - leaving this open for more sophisticated blocking

@HalosGhost

This comment has been minimized.

Copy link
Contributor

HalosGhost commented Dec 15, 2014

I would recommend µblock as a model to use. It aims to be much lighter and faster than ABP while actually extending some functionality.

@The-Compiler The-Compiler changed the title Adblock More sophisticated adblocking Oct 1, 2015

@The-Compiler The-Compiler removed their assignment Oct 1, 2015

@1-61803

This comment has been minimized.

Copy link

1-61803 commented Apr 6, 2017

Is there any ad-blocking implementation as of now, internal or via plugin?

@parchd-1

This comment has been minimized.

Copy link

parchd-1 commented Apr 6, 2017

Yes, there is host-file adblocking, as you can see above.
The plugin system isn't done yet, so other adblock systems aren't either. You can also see that in the issue linked to above ;).

@1-61803

This comment has been minimized.

Copy link

1-61803 commented Apr 6, 2017

@parchd-1 I skimmed. I see the hosts file implementation from 2014 and #30 has adblock stroke through. What is the actual state of development of plugins that would allow better ad blocking but also plugins like ublock and greasemonkey?

@The-Compiler

This comment has been minimized.

Copy link
Collaborator Author

The-Compiler commented Apr 7, 2017

What is the actual state of development of plugins

There isn't anything other than the thoughts listed in #30 so far.

that would allow better ad blocking but also plugins like ublock

What from ublock are you missing, other than having adblockplus-like block lists?

and greasemonkey

Something similar to it should be possible with per-domain settings (#27) which is the current main focus - in around a week I'll launch qutebrowser's second crowdfunding, to work on this (and maybe start some work on plugins if there's time left) during my summer holidays. Subscribe to that issue if you want to get notified once it's up!

@1-61803

This comment has been minimized.

Copy link

1-61803 commented Apr 7, 2017

What from ublock are you missing, other than having adblockplus-like block lists?

I just have switched to ublock as ABP caused high CPU usage. Having the same lists makes rendering sites equally across hosts. Just a first impression after switching blocker.

I still have to make it work in my main older machine on 10.8. I encountered the same problem pointed out here. It should compile from source, it's python and qt, right?

Something similar to it should be possible with per-domain settings (#27) which is the current main focus - in around a week I'll launch qutebrowser's second crowdfunding

Subscribed. What's the best way to get regular (weekly but not daily) news? If I compile it in my 10.8 I'll be certainly donating to your project.

@The-Compiler

This comment has been minimized.

Copy link
Collaborator Author

The-Compiler commented Apr 7, 2017

I still have to make it work in my main older machine on 10.8. I encountered the same problem pointed out here. It should compile from source, it's python and qt, right?

There's nothing in qutebrowser itself you'll need to compile, as it's Python. You'll need a working Python with PyQt though. If you can get a Python 3.5 or 3.6 installed (maybe via Homebrew or pyenv), you should be able to do something like pip install PyQt5 qutebrowser (optionally in a virtualenv). If you need help, open a separate issue though, please.

What's the best way to get regular (weekly but not daily) news?

There isn't anything like a regular newsletter. You can look at the changelog and github activity manually though.

I'd also recommend subscribing to the qutebrowser-announce mailinglist where important announcements like new releases are posted.

@ttygde

This comment was marked as off-topic.

Copy link

ttygde commented Feb 26, 2018

Is there any progress? It is the only reason I haven't switched to qutebrowser yet. I cannot imagine seeing ads while browsing google, youtube or any other site.

@The-Compiler

This comment was marked as off-topic.

Copy link
Collaborator Author

The-Compiler commented Feb 27, 2018

@asmbits I don't think anyone is working on this currently, as the current adblocker is working well enough overall and there are lots and lots of other more important issues 😉

@sagb

This comment was marked as off-topic.

Copy link

sagb commented Feb 27, 2018

On the sites where ads come not from standard CDNs the current adblock is useless. This major bug is very annoying.

@The-Compiler

This comment was marked as off-topic.

Copy link
Collaborator Author

The-Compiler commented Feb 27, 2018

You seem to have an... interesting understanding of what "major bug" means. I'd be glad if someone picked this up, but it won't be me in the near future, I have lots of other stuff on my plate. Are you volunteering? 😉

@sagb

This comment was marked as off-topic.

Copy link

sagb commented Feb 27, 2018

Sorry, it's beyond my skills and available time.

@cosmojg

This comment has been minimized.

Copy link

cosmojg commented Mar 22, 2018

@The-Compiler Would it be alright if myself or someone else tried directly integrating some of uBlock's code? Or would you rather have something built from scratch?

I think direct integration would do wonders for preventing the classic open-source effort-reduplication problem.

@The-Compiler

This comment has been minimized.

Copy link
Collaborator Author

The-Compiler commented Mar 22, 2018

@cosmojg I don't see how that'd be possible (as no WebExtension API exists), so the point is moot 😉

@cosmojg

This comment has been minimized.

Copy link

cosmojg commented Mar 26, 2018

@The-Compiler Ahh, admittedly, I haven't put much thought into this, but I wasn't talking about including it as an extension. I meant I'd try and directly integrate the parts of uBlock Origin that make it different from Qutebrowser's built-in adblock. I'm operating on the assumption that there are code bits within Qutebrowser that perform similar functions to whatever WebExtension API calls uBlock makes.

Maybe when I have less work later in the week, I'll get off my ass (and stop talking out of it) and dig through Qutebrowser's core adblock to see what kinds of original improvements can be made. Granted, I've never written code for adblocking before and only have a basic understanding of how most of it works, but that doesn't mean I can't try!

@jgkamat

This comment has been minimized.

Copy link
Collaborator

jgkamat commented Mar 26, 2018

I too have little idea how adblocking (and js in general) work, but I think a good starting point for this is to see how greasemonkey userscripts are injected into the page (and stylesheets as well). Once you get a script in the page, it becomes a js issue, which I have almost zero experience with :P

I don't think the current host-blocking adblock code is a good place to start (because this would probably need a completely different approach), but I think it's mostly located here.

@The-Compiler

This comment has been minimized.

Copy link
Collaborator Author

The-Compiler commented Mar 26, 2018

You won't be able to block network requests with JS injected in the page - there's stuff in the WebExtension API to do that which uBlock uses, but doing the same in qutebrowser won't be possible. In qutebrowser, JS injected into the page can only do whatever the page itself can do, and it has the same JS API a webpage can use (and not more).

Adblocking for qutebrowser really has to be implemented in Python - JavaScript won't work without having relevant APIs in QtWebEngine which probably won't be happening in the near future.

So, all you'll be able to take from uBlock is some general ideas and concepts (i.e. the "big picture"), and the filter list syntax (which is mostly coming from AdBlockPlus) - but not any actual code.

@The-Compiler

This comment has been minimized.

Copy link
Collaborator Author

The-Compiler commented Jun 27, 2018

FWIW webmacs seems to use the brave/ad-block module from Python.

@rr-

This comment has been minimized.

Copy link
Contributor

rr- commented Jul 31, 2018

The built-in filters seem to be less effective with each month, despite :adblock-update :(

@Yodzorah

This comment was marked as off-topic.

Copy link

Yodzorah commented Sep 7, 2018

  1. What about requests blocking via our two backends? Do they support domain patterns? And if not, what about dnsmasq?

  2. uBlock, AdBlock, etc. suck. Sorry, but it's true. They work on similar rulesets for request blocking (with patterns, though), css and js and they are centered on just removing ads. They have full control over files, scripts, etc., they have full functionality for full control over the mentioned stuff, but instead of providing the control to user they just block stuff according to rulesets. It's just a pointless waste of resources. Look at ContentBlockHelper and uMatrix - they are much better. In qutebrowser we have JS, CSS and hosts blocklists. Also, we probably have most of the features of ContentBlockHelper/uMatrix in QtWebEngine, we just need to wait for per domain settings. I think, if there is any chance we can use more simple and powerful solutions instead of emulating familiar to many users but grotesque and ailing adblokers, we should think trice before choosing the second option.

  3. Look at the ContentBlockHelper UI. I mean, not the design, it's awful, but the tree view of every loaded file for every requested website. What about something like that in qutebrowser? For example, we have ':set' mode, can we have the same for css, js, hosts, cookies, etc. settings in "per domain" mode? Not just with a sublist of settings but with the website tree?

@The-Compiler

This comment was marked as off-topic.

Copy link
Collaborator Author

The-Compiler commented Sep 7, 2018

@Yodzorah

  1. See #4188, though real adblock lists would also be patterns. Not sure what you're asking about dnsmasq.

  2. You're free to have your opinion, but not everyone wants to spend time configuring rules on every website they visit. I'd like to have both in the long run.

  3. That sounds quite similar to uMatrix, see #28 (and also #2626)

@Yodzorah

This comment was marked as off-topic.

Copy link

Yodzorah commented Sep 7, 2018

@The-Compiler,

  1. My option doesn't require users to configure websites all the time in order to get rid from ads. It can do both with the functionality of usual adblockers, while the usual adblockers with functionality of usual adblockers can only block ads. I just don't think it's a good idea to have such highly specialized tool that, in fact, can do more really useful stuff with the same 'effort'. If one will add +10% of functionality to any popular adbloker, he will have far more powerfull and flexible tool and far more control over web. And if one will use usual 'simple' adblockers and suddenly understand he wants to block something else, he will ask for another highly specialized tool instead of something more flexible for both tasks. It's like installing separate extensions changing CSS for some websites instead of installing Stylish in Chrome.

Sorry for asking, but if qutebrowser is so centered on providing user full control, why prefer highly specialized solutions over flexible? I mean, not everyone even knows about user agent header, why such setting even exists in qutebrowser then?

@The-Compiler

This comment was marked as off-topic.

Copy link
Collaborator Author

The-Compiler commented Sep 7, 2018

I'm not sure what you're trying to achieve with this discussion - like I said above I'd like to have both, as both solutions have their benefits and drawbacks. This is not the right place to discuss whether adblockers are a good thing or not.

@Yodzorah

This comment was marked as off-topic.

Copy link

Yodzorah commented Sep 7, 2018

@The-Compiler, I was trying to convince you to not try usual adblockers until you will be sure there is no flexible solution) Maybe, it was not wise, but I should have to try at least - I'm using qutebrowser because of its flexibility, so if this won't be the case, I will lose even a half-comfortable browser and will be forced to migrate on Vivaldi, because with the same low level of flexibility it at least has a big pack of features.

@toofar

This comment was marked as off-topic.

Copy link
Collaborator

toofar commented Sep 7, 2018

There is no plan for qutebrowser to support the usual adblockers. See #2626 for a discussion about a flexible request blocking framework and #28 for a prettier UI over that.

@The-Compiler

This comment was marked as off-topic.

Copy link
Collaborator Author

The-Compiler commented Sep 7, 2018

Note I've already linked those above, but whatever...

@mschilli87

This comment has been minimized.

Copy link
Contributor

mschilli87 commented Jan 25, 2019

Is there any way we will ever see this given Chromium might loose a lot of its content blocking support?
It's a pity these things happen just when we might loose support for any alternative engine.

@jgkamat

This comment has been minimized.

Copy link
Collaborator

jgkamat commented Jan 25, 2019

@jgkamat

This comment has been minimized.

Copy link
Collaborator

jgkamat commented Feb 3, 2019

@cosmojg

This comment has been minimized.

Copy link

cosmojg commented Feb 3, 2019

@jgkamat Jblock looks awesome! How far away are you from implementing this in Qutebrowser? How does it fare against webmacs's implementation of brave/ad-block?

@jgkamat

This comment has been minimized.

Copy link
Collaborator

jgkamat commented Feb 3, 2019

@jgkamat

This comment has been minimized.

Copy link
Collaborator

jgkamat commented Feb 7, 2019

@The-Compiler

This comment has been minimized.

Copy link
Collaborator Author

The-Compiler commented Feb 15, 2019

https://whotracks.me/blog/adblockers_performance_study.html has some interesting performance analysis of different adblocker engines.

All those engines being available implemented in JavaScript gave me a somewhat crazy idea: Could we use a QJSEngine to run a JavaScript adblocker and use its decisions (allow/block) from Python?

@The-Compiler

This comment has been minimized.

Copy link
Collaborator Author

The-Compiler commented Mar 18, 2019

From @jgkamat in IRC: https://testpages.adblockplus.org/ is quite useful to test an implementation.

@jgkamat

This comment has been minimized.

Copy link
Collaborator

jgkamat commented Apr 9, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.