Skip to content
Code used to build a Tracker Radar data set from raw crawl data.
JavaScript
Branch: master
Clone or download

Latest commit

Latest commit ffc5549 Mar 20, 2020

Files

Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
src Merge pull request #6 from duckduckgo/jd/timeouts Mar 20, 2020
.eslintrc.js lint config file Mar 2, 2020
.gitignore initial tracker radar detector Feb 27, 2020
LICENSE initial tracker radar detector Feb 27, 2020
README.md Update README.md Mar 3, 2020
config.json Update config.json Mar 2, 2020
package-lock.json initial tracker radar detector Feb 27, 2020
package.json initial tracker radar detector Feb 27, 2020
third-party.txt initial tracker radar detector Feb 27, 2020

README.md

DuckDuckGo Tracker Radar Detector

This is the code used to build a Tracker Radar data set using crawl data from the Tracker Radar Collector.

Getting Started

To generate a Tracker Radar data set follow these steps:

  1. Clone the Tracker Radar data repo

  2. Generate 3rd party request data using the Tracker Radar Collector

  3. Update the paths in config.json to point to your newly created crawler data files and the location of your Tracker Radar data repository

trackerDataLoc path to your Tracker Radar data repository
crawlerDataLoc path to your crawler data directory
performanceDataLoc path to your performance crawler data

Generating Tracker Radar data

  • Install dependencies

npm install

  • Build site performance summary (optional)

npm run build-performance

  • Update entity data (optional) note: requires some manual validation of the output data, see here for more info
npm run update-entities
npm run apply-entity-changes
  • Build Tracker Radar data files

npm run build

Contributing

Reporting bugs

  1. Check to see if the bug has not already been reported
  2. Create a bug report issue

New features

Right now all new feature development is handled internally.

Bug fixes

Most bug fixes are handled internally, but we will accept pull requests for bug fixes if you first:

  1. Create an issue describing the bug.
  2. Get approval from DDG staff before working on it. Since most bug fixes and feature development are handled internally, we want to make sure that your work doesn't conflict with any current projects

Questions or help with anything else DuckDuckGo related?

See DuckDuckGo Help Pages.

This software is licensed under the terms of the Apache License, Version 2.0 (see LICENSE).

You can’t perform that action at this time.