Skip to content
The tool Website Evidence Collector (WEC) automates the website evidence collection of storage and transfer of personal data. https://edps.europa.eu/press-publications/edps-inspection-software_en
JavaScript
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
assets
lib
.gitignore
CHANGELOG.md
FAQ.md
LICENSE.txt
README.md
package-lock.json
package.json
screencast-call.svg
screencast-installation.svg
website-evidence-collector.js
wec_logo.svg

README.md

Website Evidence Collector Logo

Website Evidence Collector

The tool Website Evidence Collector (WEC) automates the website evidence collection of storage and transfer of personal data. It is based on the browser Chromium/Chrome and its JavaScript software library for automation puppeteer.

Installation

Screencast Installation

  1. The Website Evidence Collector is a set of scripts written in JavaScript for execution by Node.js. Install Node.js and the Node.js package manager (NPM).
  • Windows or Mac: Follow the guide on https://nodejs.org/en/.
  • Linux: use the Linux package manager to install Node.js, e.g. zypper in nodejs10 (check version) or apt install nodejs.
  1. Install the Website Evidence Collector from
  • the tarball archive file (*.tar.gz) downloaded from the EDPS Website: npm install --global ./website-evidence-collector-*.tar.gz (with * to be replaced by the current release version),
  • Github with npm install --global https://github.com/EU-EDPS/website-evidence-collector/tarball/latest, or
  • Github with npm install --global https://github.com/EU-EDPS/website-evidence-collector/tarball/master to get a potentially broken testing version, which includes the latest changes.

The tool can be uninstalled with npm uninstall --global website-evidence-collector.

Run Website Evidence Collection

Screencast Call

To start the collection for e.g. https://example.com, open the terminal and run website-evidence-collector https://example.com. The folder output contains the gathered evidence.

Notice on the Processing of Personal Data: This tool carries out automated processing of data of websites for the purpose of identifying their processing of personal data. If you run the tool to visit web pages containing personal data, this tool will download, display, and store these personal data in the form of text files and screenshots, and you will therefore process personal data.

Examples with Command Line Options

Simple Output on the Terminal only

website-evidence-collector --no-output --quiet --yaml https://example.com

Ignore Certificate Errors during Collection

website-evidence-collector -y -q https://untrusted-root.badssl.com -- --ignore-certificate-errors

All command line arguments after -- (the second in case of npm) are applied to launch Chromium.

Reference: https://peter.sh/experiments/chromium-command-line-switches/#ignore-certificate-errors

Use pretty-printed Live Logs

First, install pino-pretty with npm install -g pino-pretty.

On POSIX compliant systems such as Linux or Mac, the output can be filtered to get prettier output:

website-evidence-collector --output https://example.com | pino-pretty --timestampKey timestamp --messageKey type --ignore stack,raw,origin

The formatting and provided information for pretty printing is configured in the script section of the package.json.

Frequently Asked Questions

Please find a collection of frequently asked questions with answers in FAQ.md

Setup of the Development Environment

  1. Install the dependencies according to the Installation Guide point 1.
  2. Install the version control system Git (https://git-scm.com/).
  3. Download the Website Evidence Collector a. from the EDPS Website and unpack the received folder with e.g. 7zip, or b. from Github with git clone https://github.com/EU-EDPS/website-evidence-collector.
  4. Open the terminal and navigate to the folder website-evidence-collector.
  5. Install the dependencies using npm install
  6. Consider to use npm link to make the command website-evidence-collector outside of the project folder.

TODO List

Resources for Developers

Use of Hooks for Restructuring Source Code:

Contributors

  • Robert Riemann (European Data Protection Supervisor, initial author)

License

This work, excluding filter lists, is distributed under the European Union Public Licence (the ‘EUPL’). Please find the terms in the file LICENSE.txt.

Filter lists in the assets/ directory are authored by the EasyList authors (https://easylist.to/) and are for your convenience distributed together with this work under their respective license as indicated in their file headers.

You can’t perform that action at this time.