Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement cosmetic filtering #47

Merged
merged 36 commits into from Nov 22, 2019
Merged

Implement cosmetic filtering #47

merged 36 commits into from Nov 22, 2019

Conversation

@antonok-edm
Copy link
Collaborator

antonok-edm commented Aug 29, 2019

Closes #38.

@antonok-edm antonok-edm force-pushed the cosmetic-filters branch from ce04638 to fc98c6c Aug 29, 2019
@antonok-edm antonok-edm requested a review from AndriusA Aug 29, 2019
@antonok-edm antonok-edm mentioned this pull request Aug 29, 2019
@antonok-edm antonok-edm force-pushed the cosmetic-filters branch 8 times, most recently from a7026a0 to f453fb3 Oct 28, 2019
Copy link
Collaborator

AndriusA left a comment

Looks good - a few small nits that would be good to address, but not a blocker, marking approved

benches/bench_redirect_performance.rs Outdated Show resolved Hide resolved
src/blocker.rs Outdated Show resolved Hide resolved
src/cosmetic_filter_cache.rs Outdated Show resolved Hide resolved
src/cosmetic_filter_cache.rs Show resolved Hide resolved
src/engine.rs Outdated Show resolved Hide resolved
src/engine.rs Show resolved Hide resolved
src/filters/cosmetic.rs Outdated Show resolved Hide resolved
///
/// See the `matches_str` test helper function for an example of how to convert hostnames and
/// domains into the appropriate hash lists.
pub fn matches(&self, request_entities: &[Hash], request_hostnames: &[Hash]) -> bool {

This comment has been minimized.

Copy link
@AndriusA

AndriusA Nov 18, 2019

Collaborator

method makes sense, but not seeing where it gets used besides tests

This comment has been minimized.

Copy link
@antonok-edm

antonok-edm Nov 18, 2019

Author Collaborator

This was from before I started storing rules keyed by hostname.

It might still be useful for some kind of logging or dashboard? Should I just remove it altogether?

This comment has been minimized.

Copy link
@AndriusA

AndriusA Nov 19, 2019

Collaborator

let's remove it, if we want it again we can probably do better

add CosmeticFilter fields

add cosmetic mask fields
@antonok-edm antonok-edm force-pushed the cosmetic-filters branch from f453fb3 to f5df13f Nov 18, 2019
antonok-edm added 7 commits Jul 2, 2019
add tests for parsing cosmetic rules with simple selectors

implement is_simple_*selector methods

Finish cosmetic filter parsing implementation

The only thing missing from here is CSS rule validation. Generally, a
DOM is used for that purpose in Javascript-based implementations.
break rule parsing after '#' separator into separate function

remove support for `script:inject()`

It appears that `script:inject()` is no longer in use; the newer `+js()`
syntax is preferred.

Ref: https://github.com/gorhill/uBlock/wiki/Static-filter-syntax#scriptlet-injection
reduce code duplication in tests

add tests for hostname parsing

add tests for href selectors

The Cliqz implementation does not flag `href$=` selectors, so I've left
those unflagged as well.

add tests for entities

as well as fixes for associated logical errors

add tests for injected scripts

add style specifier tests

add tests for unicode and unsupported syntax

disallow unhide and script inject rules without hostname constraints

prevent generic style rules

prohibit double-negated cosmetic filters

remove href flag from cosmetic filters
Rather than pulling in an entire CSS styling library as a dependency,
this ensures that the syntax of supplied CSS selectors is valid, without
concern for the textual content of individual tokens. As new
pseudo-selectors are added to the W3C spec, this implementation should
still function as intended.

add tests for bad CSS selector inputs

protect against urls appearing in `:style()` filters

See uBlockOrigin/uBlock-issues#668
@antonok-edm antonok-edm force-pushed the cosmetic-filters branch 2 times, most recently from a3a20d0 to 3ee7cfe Nov 18, 2019
antonok-edm added 7 commits Jul 8, 2019
add tests for matching util functions

add tests for cosmetic filter matching

use iterators instead of loops during matching

document cosmetic filter rule matching functions

rename hash lists in cosmetic filter matching function

update matching algorithm for batch queries

simplify test matching implementation
add selector to key convertion tests

Currently, `escaped_characters` and `escape_codes` fail because there is
no handling for these at the moment.
add key to existing tests
store complex selector strings instead of entire rules

add function to allow retrieving rules for new classes and ids
antonok-edm added 21 commits Jul 18, 2019
…esheet

harden CosmeticFilterCache thread safety across FFI
add struct for keeping track of exceptions per-hostname

add type for returning information specific to a hostname
…t injects

tests for exceptions and base stylesheet

fix hidden_generic_rule implementation

add tests for hidden generic rules
add tests for style and script inject exceptions
tests for class and id stylesheets with exceptions
return a single injectable script instead of a Vec
@antonok-edm antonok-edm force-pushed the cosmetic-filters branch from 3ee7cfe to a2045a6 Nov 22, 2019
@AndriusA AndriusA merged commit b2aab08 into master Nov 22, 2019
0 of 4 checks passed
0 of 4 checks passed
Travis CI - Branch Build Errored
Details
Travis CI - Pull Request Build Errored
Details
continuous-integration/travis-ci/pr The Travis CI build is in progress
Details
continuous-integration/travis-ci/push The Travis CI build is in progress
Details
@AndriusA AndriusA deleted the cosmetic-filters branch Nov 22, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

2 participants
You can’t perform that action at this time.