GitHub Actions Automation

GitHub Actions Automation

In short

Three workflows maintain the repo: a ShellCheck gate that lints yabs.sh on every push, and two near-identical build pipelines that watch upstream fio and iperf3 for new releases, cross-compile static binaries for all five architectures, scan them on VirusTotal, and publish them as GitHub Release assets. Releases are only created when every binary comes back clean.

For everyone

What this means as a user:

• The binaries YABS downloads were built by public, inspectable CI — not compiled on someone's laptop.
• New upstream fio/iperf3 versions are picked up automatically (daily checks), so binaries stay reasonably fresh without manual work.
• If a VirusTotal engine flags a binary, the release is skipped entirely — you'd rather wait than run something a scanner dislikes.

ci.yml — the ShellCheck gate

on: push: branches: [master]
jobs:
  shellcheck:
    steps:
      - uses: actions/checkout@v4
      - uses: ludeeus/action-shellcheck@2.0.0
        env: { SHELLCHECK_OPTS: -e SC1091 }

Every push to master runs ShellCheck over the script. -e SC1091 suppresses the "can't follow source" family of warnings (the script doesn't source files, but the exclusion is retained). Added Nov 2023 (#77); the script has been clean since a Dec 2024 fix-up pass. It is a gate, not a formatter — any warning fails the CI run until the line is rewritten or the finding is deliberately suppressed.

To run the same check locally (what CI effectively does):

docker run --rm -v "$PWD/yabs.sh:/y.sh:ro" koalaman/shellcheck:stable -S warning /y.sh

build-fio.yml / build-iperf3.yml — the binary pipelines

The two workflows are structurally identical; differences are called out below.

Triggers

• Schedule (cron): once a day — fio at 23 5 * * * (05:23 UTC), iperf3 at 47 5 * * * (05:47 UTC). Purpose: catch new upstream releases without anyone watching.
• workflow_dispatch with a force boolean: manual rebuild even when a release for the latest upstream version already exists — the mechanism for adding a new architecture to an existing release.

Both run on ubuntu-latest (GitHub-hosted; the project briefly used a self-hosted runner in Dec 2025, but the Sep 2026 workflow rewrite moved back to ubuntu-latest). permissions: contents: write — needed to create releases.

fio vs iperf3 workflows: what actually differs

Aspect	build-fio.yml	build-iperf3.yml
Upstream watched	axboe/fio	esnet/iperf
Tag scheme	upstream tag as-is (fio-3.43)	iperf3- + upstream tag (iperf3-3.22)
Cron	23 5 * * * (05:23 UTC)	47 5 * * * (05:47 UTC)
arm32 leg	builds normally	continue-on-error: true (historically flaky)
Source attached	fio-<ver>.tar.gz (GPL-2.0)	iperf-<ver>.tar.gz (BSD-3-Clause)
Missing-arm handling	n/a	x64 must exist to release; body notes the missing arm asset
Build step	libaio → fio configure/make	iperf3 configure/make (no libaio)

Job graph

flowchart TD
    A["check-release<br/>----------------<br/>query upstream latest tag via GitHub API<br/>gh release view -> already published?<br/>should-build = no release OR force"] -->|should-build == false| Z((done — no build))
    A -->|should-build == true| B["build (matrix x5)<br/>----------------<br/>x64 x86 aarch64 arm riscv64<br/>Docker: holy-build-box + musl cross<br/>upload artifact (1 day)"]
    B --> C["virustotal-scan<br/>----------------<br/>upload each binary to VT API v3<br/>poll up to 10x30s<br/>all-clean = no malicious, no suspicious,<br/>no upload failure/timeout"]
    C -->|all-clean == false| E
    C -->|all-clean == true| D["create-release<br/>----------------<br/>softprops/action-gh-release@v2<br/>binaries + checksums + upstream source<br/>VT table in body"]
    B -->|failure| E["notify-failure<br/>----------------<br/>step summary: version + reason<br/>(build failed vs VT flagged)"]
    D --> E2((done))

Loading

check-release

Queries the upstream repo's latest release tag via the GitHub API (axboe/fio → e.g. fio-3.43, used as-is for the tag name; esnet/iperf → e.g. 3.22, published as iperf3-3.22), then gh release view to check whether the YABS repo already has that release. Outputs latest-version and should-build (true when the release is missing, or when force=true). An empty API response fails the job hard — a failed lookup must never be mistaken for "version exists".

build (matrix)

Five legs — one per ARCH/CROSS/HOST triple (see the table in Precompiled Binaries). fail-fast: false, so one arch's failure doesn't kill the others. Each leg:

1. Create compilation script — heredocs the same steps as bin/cross-compile.sh (toolchain download with toolchains-release primary / musl.cc fallback; libaio static; fio configure+make, or iperf3 configure+make; libcheck).
2. Compile — docker run phusion/holy-build-box-64:latest with ARCH/CROSS/HOST/VERSION env vars.
3. Verify binary — ls -la + file in the log.
4. Upload artifact — retention-days: 1 (artifacts only need to survive to the release job).

iperf3 difference: the whole job has continue-on-error: ${{ matrix.arch == 'arm' }} — 32-bit ARM iperf3 builds have been historically flaky, so that leg may fail without failing the workflow.

Artifacts are short-lived on purpose: binaries get retention-days: 1 (they only need to survive until create-release picks them up), while the VirusTotal scan markdown gets retention-days: 30 for later audit.

virustotal-scan

• Downloads all build artifacts, uploads each binary to the VirusTotal v3 files API (POST https://www.virustotal.com/api/v3/files, multipart form, x-apikey header from the VIRUSTOTAL_API_KEY secret). The response's data.id is the analysis handle.
• Polls GET /api/v3/analyses/<id> until status == "completed", up to 10 × 30 s (~5 min per binary, sequential — by the time the first finishes the rest are usually done).
• Reads data.attributes.stats.{malicious,suspicious,undetected} and builds a markdown table — Binary | Status | Malicious | Suspicious | Undetected | VirusTotal URL — where the URL is the file's hash page (virustotal.com/gui/file/<sha256>). Saved to the step summary, a 30-day artifact, and passed to the release job via a heredoc-delimited output.
• all-clean is false if any binary has malicious > 0 or suspicious > 0, or if an upload failed or timed out — the gate is deliberately "any flag, any engine".

Known weakness: fresh, never-before-seen static musl binaries occasionally trip a single engine's ML heuristics (typically Microsoft's) at random per-arch. One flag anywhere blocks the whole release; the false positive usually clears within days as the file hash gains reputation. A relaxation has been designed but not implemented: block only on ≥2 malicious or any suspicious, and publish single-flag builds as a draft release carrying the VT table for review.

create-release

Runs only when should-build and all-clean:

1. Collect binaries + generate <tool>-checksums.sha256.
2. Download the upstream source tarball and attach it — fio is GPL-2.0 (source offer required), iperf3 is BSD-3-Clause (notice retention).
3. softprops/action-gh-release@v2 creates the release: tag (e.g. fio-3.43, iperf3-3.22), the VT scan table embedded in the body, checksum instructions, license notes.
4. iperf3 difference: artifact download is continue-on-error, an iperf3_x64 presence check gates the release, and if iperf3_arm is missing the body gets a note ("ARM 32-bit binary compilation failed (known issue with iperf3 versions >3.15)").
5. Overwrite semantics: on a force rebuild, same-name assets are overwritten (overwrite_files defaults to true) and the body is regenerated — this is how fio_riscv64/iperf3_riscv64 were retro-added to fio-3.42/iperf3-3.21-era releases in Oct 2026.

notify-failure

if: always() && should-build && (failure() || all-clean == 'false') — writes a step summary naming the version and the failure class ("VirusTotal scan detected issues" — with the table — vs "Build compilation failed") plus a link to the run logs. It's a summary page, not an external alert.

Anatomy of a published release

A successful create-release produces a release tagged fio-<ver> or iperf3-<ver> containing:

• fio_x64 / fio_x86 / fio_aarch64 / fio_arm / fio_riscv64 (or iperf3_* equivalents) — the assets yabs.sh downloads at runtime.
• <tool>-checksums.sha256 — sha256sum output over every binary.
• The upstream source tarball — fio-<ver>.tar.gz or iperf-<ver>.tar.gz.
• A body with: architecture key, "scanned by VirusTotal and verified clean", the per-binary VT results table, checksum instructions, and license notes (GPL-2.0 for fio / BSD-3-Clause for iperf3, with the maintainer's unofficial-build disclaimer).

Secrets

Secret	Used for
VIRUSTOTAL_API_KEY	VT API v3 uploads/polls (repo secret)
GITHUB_TOKEN	Release creation (automatic; contents: write)

Recipes

A narrated successful run (fio, scheduled)

1. 05:23 UTC — cron fires build-fio.yml.
2. check-release: GitHub API says upstream is fio-3.44; gh release view fio-3.44 fails → should-build=true.
3. build: 5 parallel legs spin up Holy Build Box containers; ~minutes later fio_x64, fio_x86, fio_aarch64, fio_arm, fio_riscv64 exist as artifacts; libcheck has already verified staticity inside each container.
4. virustotal-scan: each binary is POSTed to www.virustotal.com/api/v3/files; analyses polled; all come back 0 malicious / 0 suspicious → all-clean=true, table saved.
5. create-release: upstream fio-3.44.tar.gz fetched, checksums generated, release fio-3.44 published with the VT table in the body.
6. notify-failure: skipped (nothing failed).
7. Next yabs.sh run anywhere greps fio-3.44 out of the releases Atom feed and downloads it — usually within a day of upstream tagging.

Most scheduled days stop at step 2: upstream hasn't cut a release, should-build=false, everything downstream skips, total runtime seconds.

Force a rebuild (e.g. add an arch to an existing release)

1. Actions → Build fio Static Binaries (or iperf3) → Run workflow → set force = true.
2. The pipeline rebuilds all matrix arches and re-scans everything; the release's same-name assets are overwritten and the body replaced.
3. If any leg fails or VT flags anything, nothing is published — re-run once the flake clears.

Reproduce a build locally

The workflows embed the same compile steps as bin/cross-compile.sh, so you can reproduce a leg by hand (x86_64 host, Docker, no secrets needed — the VT/release jobs are GitHub-only):

docker run -t -i --rm -v `pwd`:/io \
  --env ARCH=aarch64 --env CROSS=aarch64-linux-musl --env HOST=aarch64-linux-gnu \
  phusion/holy-build-box-64:latest bash /io/cross-compile.sh

Outputs fio_aarch64 / iperf3_aarch64 in the mounted directory. Run it on a copy in a scratch directory to keep the repo clean; a full fio + iperf3 build takes several minutes per architecture.

Add a new architecture

1. Add a matrix leg (arch, cross, host) to both build-fio.yml and build-iperf3.yml.
2. Make sure the matching musl toolchain tarball (<CROSS>-cross.tgz) exists on the toolchains release (mirror from musl.cc if new).
3. Add the uname -m → ARCH mapping in yabs.sh (the ARCH name must equal the asset suffix fio_<ARCH>/iperf3_<ARCH>), plus any arch-specific fallbacks (fio file size, cpuinfo parsing, etc.).
4. Dispatch both workflows with force=true so the new asset lands on the existing release.

Failure modes to expect

Failure	Signature	Handling
musl.cc blocks the runner	toolchain download 403/fails	primary URL is the toolchains release; musl.cc is only a fallback
Upstream API flake	check-release exits 1 ("Failed to determine latest version")	curl has --retry 5; rerun tomorrow
arm32 iperf3 leg dies	build failure on arm only	continue-on-error; release notes flag the missing asset
VT upload/timeout	Upload Failed/Timed out row	counts as not-clean; release skipped; re-run later
Single-engine false positive	1 malicious engine on a fresh binary	blocks release; typically clears in days as the hash gains reputation
gh release view on a deleted tag	treated as missing → rebuild	correct behaviour

See also

• Precompiled Binaries — what gets built and the flag-level "why"
• Contributing — running ShellCheck/actionlint before a PR
• Supported Platforms — the arch matrix these workflows target