Skip to content
Browse files

Add HNTrie-based filter classes to store origin-only filters

Related issue:
- uBlockOrigin/uBlock-issues#528 (comment)

Following STrie-related work in above issue, I noticed that a large
number of filters in EasyList were filters which only had to match
against the document origin. For instance, among just the top 10
most populous buckets, there were four such buckets with over
hundreds of entries each:

- bits: 72, token: "http", 146 entries
- bits: 72, token: "https", 139 entries
- bits: 88, token: "http", 122 entries
- bits: 88, token: "https", 118 entries

These filters in these buckets have to be matched against all
the network requests.

In order to leverage HNTrie for these filters[1], they are now handled
in a special way so as to ensure they all end up in a single HNTrie
(per bucket), which means that instead of scanning hundreds of entries
per URL, there is now a single scan per bucket per URL for these
apply-everywhere filters.

Now, any filter which fulfill ALL the following condition will be
processed in a special manner internally:

- Is of the form `|https://` or `|http://` or `*`; and
- Does have a `domain=` option; and
- Does not have a negated domain in its `domain=` option; and
- Does not have `csp=` option; and
- Does not have a `redirect=` option

If a filter does not fulfill ALL the conditions above, no change
in behavior.

A filter which matches ALL of the above will be processed in a special

- The `domain=` option will be decomposed so as to create as many
  distinct filter as there is distinct value in the `domain=` option
- This also apply to the `badfilter` version of the filter, which
  means it now become possible to `badfilter` only one of the
  distinct filter without having to `badfilter` all of them.
- The logger will always report these special filters with only a
  single hostname in the `domain=` option.


[1] HNTrie is currently WASM-ed on Firefox.
  • Loading branch information...
gorhill committed Apr 19, 2019
1 parent fd9df4b commit 3f3a1543ea7fa51d700157a7f6bf0da08dd7a32b
Showing with 590 additions and 440 deletions.
  1. +2 −2 src/js/background.js
  2. +461 −322 src/js/static-net-filtering.js
  3. +58 −65 src/js/strie.js
  4. +69 −51 src/js/utils.js
@@ -137,8 +137,8 @@ const µBlock = (function() { // jshint ignore:line

// Read-only
systemSettings: {
compiledMagic: 8, // Increase when compiled format changes
selfieMagic: 9 // Increase when selfie format changes
compiledMagic: 10, // Increase when compiled format changes
selfieMagic: 10 // Increase when selfie format changes

restoreBackupSettings: {

0 comments on commit 3f3a154

Please sign in to comment.
You can’t perform that action at this time.