Skip to content
main
Switch branches/tags
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
 
 
 
 

shavar-list-creation

Build Status Coverage

This script fetches blocklist .json from urls (such as shavar-prod-lists) and generates safebrowsing-compatible digest list files to be served by shavar.

Requirements

  • python ≥ 3.6
  • (optional) virtualenv and/or virtualenvwrapper

Run

  1. (optional) Make a virtualenv for the project and activate it:

    virtualenv -p python3.8 shavar-list-creation
    source shavar-list-creation/bin/activate
    
  2. Install required libraries:

    pip install -r requirements-test.txt
    
  3. Copy the sample_shavar_list_creation.ini file to shavar_list_creation.ini:

    cp sample_shavar_list_creation.ini shavar_list_creation.ini
    
  4. Run the unit tests (currently under development):

    python -m pytest -v --cov=. --cov-branch
    
  5. Run the lists2safebrowsing.py script:

    ./lists2safebrowsing.py
    

Usage

This is run by a Jenkins deployment job every 30 minutes that:

  1. Checks out this repository
  2. Checks out the shavar-list-creation-config repository
  3. Copies stage.ini or prod.ini to shavar_list_creation.ini
  4. Runs python lists2safebrowsing.py, which uploads updated safebrowsing list files to S3 for shavar.

About

Script to transform the Disconnect block-list into Safebrowsing v2 format for Firefox Tracking Protection

Resources

License

Code of conduct

Stars

Watchers

Forks

Packages

No packages published

Languages