.github/workflows: integrate Jafar QA checks

Part of #356

.github/workflows/qa.yml

@@ -0,0 +1,17 @@
+name: qa
+on: [push]
+jobs:
+  test:
+    runs-on: "${{ matrix.os }}"
+    strategy:
+      fail-fast: false
+      matrix:
+        os: [ubuntu-latest]
+        go: ["1.14"]
+        experiment: ["telegram"]
+    steps:
+      - uses: actions/setup-go@v1
+        with:
+          go-version: ${{ matrix.go }}
+      - uses: actions/checkout@v2
+      - run: ./QA/docker/start.bash ./QA/docker/${{ matrix.experiment }}.sh

.gitignore

@@ -1,5 +1,6 @@
 /.vscode
 /asn.mmdb
+/badproxy.pem
 /ca-bundle.pem
 /country.mmdb
 /jafar
@@ -8,3 +9,4 @@
 /miniooni.exe
 /oonipsiphon/
 /report.jsonl
+/telegram.jsonl

QA/README.md

@@ -0,0 +1,53 @@
+# Quality Assurance scripts
+
+This directory contains quality assurance scripts that use Jafar to
+ensure that OONI implementations behave. These scripts take as unique
+command line argument the path to a binary with a OONI Probe v2.x
+compatible command line interface.
+
+Tools with this CLI are:
+
+1. `github.com/ooni/probe-legacy`
+2. `github.com/measurement-kit/measurement-kit/src/measurement_kit`
+3. `github.com/ooni/probe-engine/cmd/miniooni`
+
+## Run QA on a Linux system
+
+These scripts assume you're on a Linux system with `iptables`, `bash`,
+`python3`, and possibly a bunch of other tools installed.
+
+To start the QA script, run this command:
+
+```bash
+sudo ./QA/$nettest/$nettest.py $ooni_exe
+```
+
+where `$nettest` is the nettest name (e.g. `telegram`) and `$ooni_exe`
+is the OONI Probe v2.x compatible binary to test.
+
+The Python script needs to run as root. Note however that sudo will also
+be used to run `$ooni_exe` with the privileges of the `$SUDO_USER` that
+called `sudo ./QA/$nettest/$nettest.py ...`.
+
+## Run QA using a docker container
+
+Run test in a suitable Docker container using:
+
+```
+./QA/docker/start.sh ./QA/docker/$nettest.sh
+```
+
+Note that this will run a `--privileged` docker container.
+This will eventually run the Python script you would run on Linux.
+
+For now, the docker scripts only perform QA of `miniooni`.
+
+## Diagnosing issues
+
+The Python script that performs the QA runs a specific OONI test under
+different failure conditions and stops at the first unexpected value found
+in the resulting JSONL report. You can infer what went wrong by reading
+the output of the `$ooni_exe` command itself, which should be above the point
+where the Python script stopped, as well as by inspecting the JSONL file on
+disk. By convention such file is named `$nettest.jsonl` and only contains
+the result of the last run of `$nettest`.

QA/docker/start.bash

@@ -0,0 +1,5 @@
+#!/bin/sh
+set -ex
+DOCKER=${DOCKER:-docker}
+$DOCKER build -t jafar-qa ./cmd/jafar/
+$DOCKER run --privileged -v`pwd`:/jafar -w/jafar jafar-qa "$@"

QA/docker/telegram.sh

@@ -0,0 +1,5 @@
+#!/bin/sh
+set -ex
+go build -v ./cmd/miniooni
+go build -v ./cmd/jafar
+sudo ./QA/telegram/telegram.py ./miniooni