content filtering using the built-in filtering engine

[thestinger]

No description provided.

[csagan5]

I made a summary here some time ago: https://github.com/bromite/bromite/wiki/AdBlocking

If I were to allocate development efforts nowadays I'd either build on top of Chromium's own one (never checked feasibility though) or adopt Brave's one.

[thestinger]

Yes, I like the look of their internal ad-blocking engine, which would need to be wired up to using other filter subscriptions. It also seems they'll be exposing it to addons for declarative content filtering, which will put some pressure on them to improve it too. They want to avoid having so many addons intercepting / modifying network requests and directly accessing / modifying the content of the page since there are so many malicious / spyware addons doing it and it's making them look bad.

[csagan5]

They want to avoid having so many addons intercepting / modifying network requests and directly accessing / modifying the content of the page since there are so many malicious / spyware addons doing it and it's making them look bad.

I disagree with this narrative; it is still mainly about having direct control (in terms of filtering or inspection) on user navigation preferences and URLs, IMO. The enterprise functionality implemented for this purpose will be used with the switch of a flag for any government that needs it.

[thestinger]

The current API used by extensions is very broken. It's not fail-safe but rather if it times out or the extension has an error it falls back to permitted the request. It has similarly broken behavior when multiple extensions are using it. It waits for a bit for the first response and then continues on. It's also being broadly used by spyware, since it's global rather than restricted to specific domains with permission being requested case-by-case:

https://arstechnica.com/information-technology/2019/07/dataspii-inside-the-debacle-that-dished-private-data-from-apple-tesla-blue-origin-and-4m-people/

If you set up a strict Content-Security-Policy with report-uri / report-to, you'll see that a substantial portion of users have malware / spyware extensions that are messing with your page content and injecting tracking, advertisements, themes creating vulnerabilities in the site, etc. They can and do remove the Content-Security-Policy header or make modifications to it, but there are enough extensions too stupid to do this that it can be monitoring to some extent from the reporting support.

It's important to have a fail-safe, declarative approach for reliability / robustness. The extension API is not truly viable for a use case involving security. Most people don't care about correctness or writing robust software but rather just having things mostly work and appear to be fine. That's the only reason such an awful API has become to broadly used and is tolerated.

The enterprise functionality implemented for this purpose will be used with the switch of a flag for any government that needs it.

The enterprise toggle is to allow enterprise deployments to continue using legacy approaches for substantially longer because they're considered incompetent and require far longer to move to newer technologies in practice. It's one of those internal toggles exposed primarily via the device management support for use in enterprise device management. The same thing ends up being done with other security transitions.

[thestinger]

Also, somewhat related is that there are ad-blockers like AdGuard doing TLS interception just like the ill-advised and horrifying enterprise and AntiVirus filtering deployments:

https://kb.adguard.com/en/general/https-filtering/https-filtering-known-issues

These interception implementations provide their own TLS stack and reimplement all the security features. They usually get a lot of this wrong and are missing security features (including AdGuard). It's also inherently less secure to some extent.

I feel it's very important to have a proper well integrated content filtering approach like Safari on iOS, i.e. declarative content filters selected by users with a robust, efficient internal filtering engine. I'm very happy that Chromium is moving in this direction regardless of how clueless media and privacy activists have perceived it. The Chromium security team are the ones pushing for it.

I think it would also be good to have real pinning support not bypassed by local root certificates, like in https://bugzilla.mozilla.org/show_bug.cgi?id=1567114.

[csagan5]

I'm very happy that Chromium is moving in this direction

The worthiness of this change will be judged in the capacity of extensions authors to achieve the same level of features as before.

how clueless media and privacy activists have perceived it.

The communication around this change from upstream has been spotty and leaving room for improvement (to be generous).

[thestinger]

The worthiness of this change will be judged in the capacity of extensions authors to achieve the same level of features as before.

Wiping out many of the worst malware / spyware extensions by only allowing case-by-case usage of inspection and requiring the user to authorize it for each site is going to be a major improvement.

Extensions functionality also needs to be judged based on whether it actually works correctly and robustly, especially for an extension supposedly improving privacy / security.

There are no extensions for Chromium on mobile so there's nothing being lost there and if I had the option to enable them, I wouldn't, similar to how Brave doesn't support using arbitrary extensions on the desktop but rather only a tiny set of whitelisted ones if I recall correctly.

[csagan5]

FYI in v76 the preference CONTENT_SETTINGS_TYPE_ADS becomes directly dependant on the safe browsing component, thus development of this issue might become more closely coupled to #35.

In Bromite my choice is still to rip out safe browsing, and I disagree with the upstream new direction to allow it to be more closely dependant on the safe browsing component.

[thestinger]

I would like to take an approach like Brave for Safe Browsing, although not using the Play Services implementation on mobile. The standard Safe Browsing retrieves a database based on a truncated 4 byte (32-bit) hash of the URL. This intentionally has many collisions and ends up downloading a database of many URLs. Brave proxies the connections through their own servers in order to provide a better privacy policy too. The existing implementation isn't privacy invasive unless the user opts in to reporting (and I've disabled doing that) or enables Scout which is still allowed but I want to remove that preference as it doesn't make sense to use it with a non-official Chrome browser and it just doesn't fit Vanadium.

Their Safe Browsing implementation for ad blocking is likely to figure out which sites should have it enabled, using this existing database system. I simply want to disable that part of it and apply the filters globally across sites, so that dependency may not be particularly relevant with this disabled. I'd also want to retrieve the filters from my own servers and maybe they should be signed. I'll need to look at how they implement it.

[csagan5]

@thestinger sure; to go back on the topic of content filtering: I have found some issues in v76, which I previously mistakenly attributed to the SafeBrowsing component, but they are instead due to the fact that NetworkService is enabled by default in v76.

For reference:

• https://docs.google.com/document/d/1wAHLw9h7gGuqJNCgG1mP1BmLtCGfZ2pys-PdZQ1vg7M/edit - Network Service in Chrome
• https://docs.google.com/document/d/1TZEuPvr2KAbP4_TZpuuwtEEArQsyAkc2HDu68l66YwU/edit?ts=598244df - WebRequest + Network Service

The net result is that some major work is needed to use Bromite's content filtering in v76 with Network Service enabled; for next release I will keep it disabled, until I can figure out what is needed to do content filtering with the service enabled.

This will eventually be necessary as upstream moves forward with Network Service; it is also (probably) a safer way to run network-related calls, although I cannot judge that.

My best idea so far is to implement a specific load flag to be set by the browser process when adblocking is enabled. Another option would be to use the built-in filtering engine as you state here in this issue; let me know if you manage to master the use and customisation of that engine, I would not want to switch the engine in Bromite but if it is not possible to plug it in the Network Service (via URLRequest) then I will have to.

[dm17]

I wonder if there is any news in this category? I've not found any... Perhaps there is an approach that would take less work that has been discovered since?

[JW10234]

Cromite has an AdBlock Plus patch that works very well, maybe it can be applied to Vanadium to block ads.

[thestinger]

Code from there cannot be used in Vanadium. It's GPLv3 which is incompatible and not legal to use within a WebView implementation.

[qoijjj]

There's also https://github.com/brave/adblock-rust which has come a long way, it seems.

It's MPLv2, which I think should be compatible?

[quh4gko8]

Implemented at #453

[ThatOneCalculator]

Awesome! As an end-user, what does this mean in terms of a timeline for full adblocking capabilities?

[thestinger]

The main limitation is that we're currently only using EasyList + EasyPrivacy. That can be extended with updates to the configuration app without Vanadium updates. Not all filter rules are supported by the standard content filtering engine so the other limitations come from that. There's not a specific timeline for any of this.