Configuration and instructions used to crawls top sites using a specially instrumented version of Firefox gathering data for Webcompat analysis.
Generate the seed list via a series of pre-crawls
Run an OpenWPM crawl in Google Cloud Platform
To update the OpenWPM Crawler and crawl-prep submodules to the latest commits in the remotely tracked branches:
git submodule update --remote