Ad block engine used in the Brave browser for ABP filter syntax based lists like EasyList.
Clone or download
bbondy Merge pull request #151 from snyderp/return-exact-filter-from-cli
update the -D option on cli to return full original rule text
Latest commit fb69449 Dec 13, 2018
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
brave Make buildable for Android Jul 31, 2018
lib update the -D option on cli to return full original rule text Dec 12, 2018
lists updated URL of Finnish easylist addition to new, maintained version Dec 11, 2018
muon Fix undefined definitions of "FilterList" and "isBlockableProtocol" f… Jun 20, 2018
perf Use ARCH x86_64 for macOS Nov 21, 2018
sample Use ARCH x86_64 for macOS Nov 21, 2018
scripts update the -D option on cli to return full original rule text Dec 12, 2018
test remove unused filter serializing/deserializing methods, correct handl… Nov 21, 2018
vendor Add depot_tools submodule for future linting Dec 14, 2015
.gitignore moved protocol checking code to its own file, changed to a state mach… May 15, 2018
.gitmodules Add depot_tools submodule for future linting Dec 14, 2015
.npmignore Update npmignore Oct 1, 2016
.travis.yml Use latest node always on travis May 9, 2017
CHANGELOG.md Add HTML Filtering parsing Apr 19, 2017
LICENSE Initial commit Oct 13, 2015
Makefile fix path of node-gyp in Makefile Dec 4, 2017
README.md Update README.md Jan 10, 2017
ad_block_client.cc remove unused filter serializing/deserializing methods, correct handl… Nov 21, 2018
ad_block_client.h remove unused filter serializing/deserializing methods, correct handl… Nov 21, 2018
ad_block_client_wrap.cc remove unused filter serializing/deserializing methods, correct handl… Nov 21, 2018
ad_block_client_wrap.h Add functionality to generate manifest files from ad block lists May 30, 2018
addon.cc Implement overload for InitAll that npm run build needs Nov 13, 2017
bad_fingerprint.h Upgrade to Hash Set 2.0.0 Apr 7, 2018
bad_fingerprints.h Update bad fingerprints Apr 21, 2018
bad_fingerprints4.h Renaming files, classes and namespaces Jan 1, 2017
bad_fingerprints5.h Renaming files, classes and namespaces Jan 1, 2017
bad_fingerprints6.h Renaming files, classes and namespaces Jan 1, 2017
bad_fingerprints7.h Renaming files, classes and namespaces Jan 1, 2017
bad_fingerprints8.h Renaming files, classes and namespaces Jan 1, 2017
base.h Make it buildable in chromium and LB Jun 30, 2016
binding.gyp Make AdBlock compilable from within Brave Jun 15, 2018
cosmetic_filter.cc Renaming files, classes and namespaces Jan 1, 2017
cosmetic_filter.h Upgrade to Hash Set 2.0.0 Apr 7, 2018
data_file_version.h Set data file version to 4 in preparation for format changes Apr 7, 2018
filter.cc remove unused filter serializing/deserializing methods, correct handl… Nov 21, 2018
filter.h remove unused filter serializing/deserializing methods, correct handl… Nov 21, 2018
filter_list.cc Add component id's to ad block filter lists May 31, 2018
filter_list.h Add component id's to ad block filter lists May 31, 2018
include.js Lint JS Oct 25, 2016
main.cc Add EasyPrivacy Apr 7, 2018
no_fingerprint_domain.cc Add hash-sets for no fingerprint domain list checks Apr 19, 2018
no_fingerprint_domain.h Add hash-sets for no fingerprint domain list checks Apr 19, 2018
package-lock.json Add DEP to our own CppUnitLite with working macOS Nov 21, 2018
package.json Add DEP to our own CppUnitLite with working macOS Nov 21, 2018
perf.cc Add EasyPrivacy Apr 7, 2018
protocol.cc remove unused filter serializing/deserializing methods, correct handl… Nov 21, 2018
protocol.h moved protocol checking code to its own file, changed to a state mach… May 15, 2018

README.md

Build Status

Brave Ad Block

Native node module, and C++ library for Adblock Plus filter parsing for lists like EasyList.

It uses a bloom filter and Rabin-Karp algorithm to be super fast.

To include brave/ad-block in your project:

npm install --save ad-block

JS Sample

const {AdBlockClient, FilterOptions} = require('ad-block')
const client = new AdBlockClient()
client.parse('/public/ad/*$domain=slashdot.org')
client.parse('/public/ad3/*$script')
var b1 = client.matches('http://www.brianbondy.com/public/ad/some-ad', FilterOptions.script, 'slashdot.org')
var b2 = client.matches('http://www.brianbondy.com/public/ad/some-ad', FilterOptions.script, 'digg.com')
console.log('public/ad/* should match b1.  Actual: ', b1)
console.log('public/ad/* should not match b2.  Actual: ', b2)

C++ Sample

#include "ad_block_client.h"
#include <algorithm>
#include <iostream>
#include <fstream>
#include <sstream>
#include <iostream>
#include <string>

using namespace std;

string getFileContents(const char *filename)
{
  ifstream in(filename, ios::in);
  if (in) {
    ostringstream contents;
    contents << in.rdbuf();
    in.close();
    return(contents.str());
  }
  throw(errno);
}

void writeFile(const char *filename, const char *buffer, int length)
{
  ofstream outFile(filename, ios::out | ios::binary);
  if (outFile) {
    outFile.write(buffer, length);
    outFile.close();
    return;
  }
  throw(errno);
}


int main(int argc, char**argv) {
  std::string &&easyListTxt = getFileContents("./test/data/easylist.txt");
  const char *urlsToCheck[] = {
    // ||pagead2.googlesyndication.com^$~object-subrequest
    "http://pagead2.googlesyndication.com/pagead/show_ads.js",
    // Should be blocked by: ||googlesyndication.com/safeframe/$third-party
    "http://tpc.googlesyndication.com/safeframe/1-0-2/html/container.html",
    // Should be blocked by: ||googletagservices.com/tag/js/gpt_$third-party
    "http://www.googletagservices.com/tag/js/gpt_mobile.js",
    // Shouldn't be blocked
    "http://www.brianbondy.com"
  };

  // This is the site who's URLs are being checked, not the domain of the URL being checked.
  const char *currentPageDomain = "slashdot.org";

  // Parse easylist
  AdBlockClient client;
  client.parse(easyListTxt.c_str());

  // Do the checks
  std::for_each(urlsToCheck, urlsToCheck + sizeof(urlsToCheck) / sizeof(urlsToCheck[0]), [&client, currentPageDomain](std::string const &urlToCheck) {
    if (client.matches(urlToCheck.c_str(), FONoFilterOption, currentPageDomain)) {
      cout << urlToCheck << ": You should block this URL!" << endl;
    } else {
      cout << urlToCheck << ": You should NOT block this URL!" << endl;
    }
  });

  int size;
  // This buffer is allocate on the heap, you must call delete[] when you're done using it.
  char *buffer = client.serialize(size);
  writeFile("./ABPFilterParserData.dat", buffer, size);

  AdBlockClient client2;
  // Deserialize uses the buffer directly for subsequent matches, do not free until all matches are done.
  client2.deserialize(buffer);
  // Prints the same as client.matches would
  std::for_each(urlsToCheck, urlsToCheck + sizeof(urlsToCheck) / sizeof(urlsToCheck[0]), [&client2, currentPageDomain](std::string const &urlToCheck) {
    if (client2.matches(urlToCheck.c_str(), FONoFilterOption, currentPageDomain)) {
      cout << urlToCheck << ": You should block this URL!" << endl;
    } else {
      cout << urlToCheck << ": You should NOT block this URL!" << endl;
    }
  });
  delete[] buffer;
  return 0;
}

Util for checking URLs

  • Basic checking a URL: node scripts/check.js --host www.cnet.com --location https://s0.2mdn.net/instream/html5/ima3.js
  • Checking a URL with discovery: node scripts/check.js --host www.cnet.com --location "https://slashdot.org?t=1&ad_box_=2" --discover
  • Checking a URL against a particular adblock list: node scripts/check.js --uuid 03F91310-9244-40FA-BCF6-DA31B832F34D --host slashdot.org --location https://s.yimg.jp/images/ds/ult/toppage/rapidjp-1.0.0.js
  • Checking a URL from a loaded DAT file: node scripts/check.js --dat ./out/SafeBrowsingData.dat --host excellentmovies.net --location https://excellentmovies.net
  • Checking a list of URLs: node scripts/check.js --host www.cnet.com --list ./test/data/sitelist.txt
  • Checking a list of URLS with discovery: node scripts/check.js --host www.cnet.com --list ./test/data/sitelist.txt --discover

Developing brave/ad-block

  1. Clone the git repository from GitHub:

    git clone --recursive https://github.com/brave/ad-block

  2. Open the working directory:

    cd ad-block

  3. Install the Node (v5+) dependencies:

    npm install

Make the node module

make

Running sample (which also generates a .dat file for deserializing)

make sample

Running tests

make test

Clearing build files

make clean