QA and Testing

QA and Testing

Local checks

The fastest local loop, run from the overlay root on the Gentoo test VM:

pkgcheck scan --config scripts/pkgcheck.conf   # full overlay scan with the overlay's settings
pkgdev manifest                                # manifest integrity
python3 scripts/simple-qa-check.py             # wraps pkgcheck/pkgdev, degrades gracefully if missing

Plain pkgcheck scan with no --config ignores scripts/pkgcheck.conf (it's not the standard metadata/pkgcheck.conf location pkgcheck auto-discovers) and will scan with default settings — including the RequiredUseDefaults/RedundantVersion checks this overlay deliberately disables (see below). simple-qa-check.py finds scripts/pkgcheck.conf on its own (it searches its own directory, the parent directory, and the cwd), so it doesn't need --config passed explicitly.

simple-qa-check.py auto-detects available tools (pkgcheck, pkgdev, emerge) and produces the same HTML/Markdown report format CI does — useful to preview what a PR comment will look like.

For a closer match to CI (a real Docker container, not just whatever's on the VM):

python3 scripts/test-qa-pipeline.py               # Docker-based, mirrors the workflow
python3 scripts/test-qa-pipeline.py --interactive  # drop into a shell in the container
python3 scripts/test-qa-pipeline.py --fallback     # fall back to simple-qa-check.py if no Docker

Manifest / metadata maintenance

./scripts/digests_and_cache.sh   # runs `ebuild <file> manifest` for every ebuild, repo-wide

Despite the name, it currently only regenerates Manifest files (ebuild <file> manifest per package directory) — the egencache metadata-cache step is present in the script but commented out.

Run this after bulk changes that touch many packages' Manifests (e.g. after a "drop old version" cleanup pass — see Release Workflow).

What CI runs

qa-check.yml: builds a Docker image from fsvm88/gentoo-docker-pkgdev-pkgcheck:latest, syncs the Gentoo tree, configures this overlay, then runs simple-qa-check.py inside the container. Runs on push/PR to main, weekly (Sunday 06:00 UTC), and on manual dispatch. Posts a summary comment on PRs and uploads full reports as a 30-day artifact.

deploy-pages.yml: on successful main runs, publishes the reports to GitHub Pages — https://fsvm88.github.io/cosmic-overlay/qa-reports/.

pkgcheck config

scripts/pkgcheck.conf:

[DEFAULT]
exit = error,warning
keywords = -RequiredUseDefaults,-RedundantVersion
arches = amd64,arm64

RequiredUseDefaults/RedundantVersion are disabled deliberately — they fire on patterns that are fine for an overlay but not for the main tree. If you hit a new pkgcheck warning that's similarly not applicable here, add it to keywords in this file rather than silencing it ad hoc in CI.

What pkgcheck won't catch

pkgcheck scan is necessary but not sufficient — it doesn't build anything, so wrong variable references inside eclass functions or subtly broken src_install logic can pass a scan cleanly. Pair it with python3 scripts/simple-qa-check.py for the static side, and actually building for the rest — emerge -uDN @world --quiet --keep-going on the test VM (see Release Workflow), or ebuild ... prepare/compile/install for a single package when narrowing down a specific failure (see Bumping Ebuilds).