v1.1.1

This is the first public release since 1.0.1 (August 2023). Versions 1.0.2 to 1.1.1 were only used internally, so their changes are included here.

⚠️ Security fix: please upgrade

CVE-XXXX-XXXXX: remote code execution through the "redo" and "tuning" features. All versions up to and including 1.0.1 are affected.

• The flaw: an unauthenticated client could register results with crafted parameters, then ask the server to re-run them. The values were put unquoted in a shell script, so the server ran the client's commands. The flaw is in the server (backend); the qaboard CLI package is not affected.
• The fix: values are now quoted, and the options passed to LSF are validated. Re-running results, tuning and other actions require a logged-in user, and jobs run as that user.

A review of the server for the same kind of problem fixed these too:

• Client-chosen folders: the server only writes, deletes or runs code in the storage folders listed in QABOARD_STORAGE_ROOTS, and never in its own folders. Results stored elsewhere are refused.
• Symbolic links: deleting results no longer deletes the contents of folders they link to. Files listed in manifests must be inside their own folder.
• Missing logins: changing or deleting a run, stopping or deleting batches, exports, restoring artifacts and the GitLab proxies now require a login.
• Shell injection when stopping batches, through their stored runner options.
• Git argument injection through commit ids when restoring artifacts.
• Path traversal in the custom test-groups files.
• Token leaks: GITLAB_ACCESS_TOKEN and GITHUB_ACCESS_TOKEN could be sent to a host chosen by the client. They are now only sent to configured hosts.
• Webhooks: they could make the server clone a repository to any path. They can now be authenticated with QABOARD_WEBHOOK_SECRET.
• Dask schedulers: stopping Dask batches could connect to any scheduler address. Only those in QABOARD_DASK_SCHEDULERS are used now.
• Session key: a hardcoded default Flask SECRET_KEY allowed forging logins. If none is configured, a random key is now generated.

Thanks to the reporter, and to Samsung PSIRT, for the report and the coordination.

The remaining known issues and their mitigations are listed in SECURITY.md. The main one: by design, the qa CLI still sends results to the server without authentication.

Upgrading

• Set QABOARD_STORAGE_ROOTS if your results are not under /mnt/qaboard. List the folders separated by :, e.g. /mnt/qaboard:/mnt/datasets. Without it, results stored elsewhere are refused, and can't be re-run or deleted from the web app. See environment variables.
• Set QABOARD_WEBHOOK_SECRET (recommended), and the same secret in your GitLab and GitHub webhooks. For GitHub Enterprise, also list its hostname in QABOARD_GITHUB_HOSTS.
• Scripts that call actions like redo or delete now need an API token. See Using the API from scripts.
• Re-running outputs or tuning on LSF goes through QA_RUNNERS_LSF_BRIDGE, with the {user} and {bsub_command} placeholders. Set it to use these features with LSF. See LSF integration.
• Set SECRET_KEY explicitly if you run several backend replicas, with the same value everywhere. If it's not set, a key is generated and saved in the backend's data volume.
• Mount your results storage in a site overlay. The base docker-compose.yml no longer mounts it; start from deployments/example.yml.

Highlights

For people looking at results

• A "What's new" panel with the release notes, and a new user guide.
• A new logs viewer: live updates, search, failure reasons, and LSF's own reports (exit reason, run time, memory used).
• On the commit page:
  • a controls panel to pick visualizations, metrics and sorting;
  • search runs by ID;
  • faster loading for big batches.
• In the image viewer:
  • a synced crosshair, zoom and pan;
  • copy images to the clipboard;
  • TIFF versions of PNG outputs, and comparisons between formats;
  • images with identical pixels are no longer flagged as different.
• Synced video comparisons.
• Copy a run's configuration in tuning format, and save custom batches with Ctrl-S.
• Milestones have owners.

For project integration (qa CLI, qaboard.yaml)

• Limit how many runs of a batch use LSF at once (runners.lsf.concurrency, --lsf-concurrency).
• A new Dask runner for batches of many short runs (pip install "qaboard[dask]").
• Pass arguments to your code with --, and give several batch names to one qa batch.
• qa batch stops before sending more than 1,000 runs at once.
• Site packages provide defaults for all projects (a base qaboard.yaml).
• Projects spread over many repositories can share one templated configuration.
• API tokens: the CLI reads them from $QA_TOKEN or ~/.qaboard/secrets.yaml.
• QABOARD_API_VERIFY controls TLS verification.
• Packaged with pyproject.toml.

For administrators

• Single sign-on with SAML, an option to require a login, and sign-in restricted to given users, groups, projects or namespaces.
• Zero-downtime deploys with deployments/deploy.py, and a Helm chart for Kubernetes.
• GitHub repositories, alongside GitLab.
• A health endpoint, and a backend tuned for heavy load.
• The backend runs on Debian trixie and Python 3.13.
• On an existing database, create the idx_outputs_batch_user_storage index by hand to speed up storage quota queries. New databases get it automatically.

The full release notes, month by month, are at https://samsung.github.io/qaboard/release-notes.