Repository navigation
Checkout audit
Four targets on the checkout itself, not on what it ships. make consistency-audit holds the statements the metadata makes against each other; make build-verify asks whether the build output committed to the repository comes from the source beside it; make changelog-lint asks whether the App Store finds this release in the changelog; make php-compat asks whether the app runs on the PHP versions its servers run it on.
Tip
TL;DR — make dev-init once per machine, then make consistency-audit (a second, metadata only), make build-verify (minutes: it installs and builds the checkout twice, outside your app), make changelog-lint (a second; make tag runs it before its confirmation) and make php-compat (after make build; it runs one build image per PHP version it checks, pulled on first use). Each prints a verdict with findings; ARGS="--details" adds the tables where a target has them, ARGS=--json the machine-readable form, make help-<target> the full option list.
- What consistency-audit checks
- What build-verify does
- What changelog-lint checks
- What php-compat checks
- Options
- Where the analysers live
- FAQ
An app states the same fact in several files, and each file is valid on its own. A linter checks one file against a rule and cannot see any of that; the audit holds the files against each other and reports where they disagree:
- the version in
appinfo/info.xmlagainst the one inpackage.json - the PHP floor in
info.xmlagainstcomposer.json. A lower floor ininfo.xmlis a[!]finding: the server enables the app on a PHP below the composer floor, where the packages resolved for that floor may not run. A lower floor incomposer.jsonis a note that names the version to raise it to, since the server does not enable the app belowmin-versionandmake buildresolvesvendor/on that floor or above - every PHP extension that
composer.jsonor a package incomposer.lockrequires against the<lib>entries ininfo.xml, unless every Nextcloud server has it - the Node version in
package.jsonagainst.nvmrcand against the installed workflows
The second half is what a release needs and a checkout can be missing:
- the
<repository>and<bugs>links, the only way from a published release back to the project - a
<nextcloud>range that names servers that exist and reaches no further than the App Store publishing rules allow at the time of the release: the newest released major plus one - an app name in
<name>without Nextcloud in it, which those rules demand as well - an
engines.nodethat admits a Node line the release can be built on. ncmake picks the build Node from the Node.js release schedule, and the audit picks it the same way. A range that admits only end-of-life lines is a[!]finding that names the line the release is built on, the date it reached end of life and the oldest line still maintained. A range that admits no line released so far is one as well, with the first line it admits and its start date, since make then falls back to the newest LTS line, which the range does not admit. So is a range that admits no LTS line, with the date the first line it admits enters LTS: a contributor on the current LTS cannot build the app - a lockfile, and one rather than two
- a licence stated the same way everywhere
- a linter that a script calls while no configuration for it is in the tree
- a directory the code reads at run time that the shipped file set leaves out: the PHP that ships reaches it through
__DIR__, or the frontend throughlinkToorgenerateFilePathwith the app id. The build and the packing pass without it, and the app fails later, in the one code path that reads it. The report names the first place that reads each such directory and thekeep_extraline inncmake.mkthat ships it. In an ncmake app it is a[!]finding; in any other app it is a note on what adopting ncmake would need, since such an app packs its own file set
It also names the build file make reads: of GNUmakefile, makefile and Makefile, only the first one present. It warns when that file hides an ncmake stub or copy, so that no ncmake target exists, and points a stub still committed as Makefile or makefile to git mv <name> GNUmakefile.
Each of those is reported once, with the reason it matters, and never as a score. What the checkout does not state is named as not stated instead of being guessed at. The one exception is the oldest Nextcloud major still maintained, which is written into the analyser, and the report names the month that is from. The values the audit takes from the network are read when it runs: the newest released major, the highest major among the release tags vX.Y.Z of nextcloud/server (a beta or rc tag is no release), the major nextcloud/server develops on master, from its version.php, and, for an app with a package.json, the Node.js release schedule. A <nextcloud max-version> beyond master names servers nobody has run the app on; one beyond the newest release plus one breaks the publishing rules, which happens while the next major is branched and not yet released. Either is a [!] finding. Without the network a list of majors and a snapshot of the schedule in the analyser stand in, and the report says where each value came from and on which day the schedule was judged.
It reads metadata only: no build, no install, no dependency tree, no bytes. That makes it useful on a fresh clone, and on an app that never heard of ncmake.
It copies the tracked files into a scratch tree outside the app, installs the locked dependencies there, runs the build — twice — and compares what came out against what the checkout carries.
What it compares is where the build writes. Where the bundler config names that directory, output.path in a webpack or rspack config or build.outDir in a vite config, that directory counts, together with js/ or css/ only where git ignores them. A committed js/ next to a config that writes to dist/ is then the app's source. The config is read as text, not run, so a path counts only where the config writes it out literally, as a string or as path.resolve(__dirname, 'dist'). A computed one, or a config that leaves the output to a shared base config such as @nextcloud/webpack-vue-config, falls back on js/ and css/. ARGS=--build=<dir> overrides both.
Source files can sit in an output directory as well, next to what the build writes there. They are told apart by what refers to them: a relative import, a name the bundler config maps to a directory through resolve.alias, a require.context in webpack, an import.meta.glob in vite, or a name the app loads the file by in its PHP code, its templates or info.xml.
Twice, because a single build cannot tell the two answers apart. A file that differs after one build was either built from another source or is written differently every time, and only a second build separates them: what both builds wrote byte for byte the same way is what this source produces, so a difference against the checkout is the checkout's. A file the two builds disagree on is named and gets no verdict, because no build reproduces it. The second build runs in another directory, so output that depends on where the build ran counts as not reproducible too. ARGS=--once builds once and states the open question instead of answering it.
A difference both builds reproduce means the output in the checkout was built from another source, and make build is the remedy. Where git ignores the output and make build has just run on this working tree, that cannot be the reason: ARGS=--fresh says so, and the same difference then reads as a build that depends on something a fresh clone does not have, or on the directory it ran in. That output is a build of this source in another directory as well: where it matches the first build on every file name and every byte but build metadata, it stands in for the second build, which then does not run.
Hashed file names are paired by content, so app-3f9c1a2b.mjs and app-8b20de41.mjs are read as one file under two names, and a reference that carries the hash counts as build metadata rather than a difference. A source map is compared as the map it is: what it was compiled from decides, the directory it records its sources under does not.
An initialized submodule is source like the app's own, so its tracked files are copied too. One the checkout has not initialized cannot be, and when the build then fails, the report names the submodule and says the build could not be verified, rather than blame the app for source the rebuild never had. git submodule update --init fetches it.
The scratch tree is a git repository with the HEAD and the tags of the checkout, and git lists exactly the files copied into it, so a build that runs git ls-files or git describe gets the answers a fresh clone gives. The build output stays out of the tree, so git describe --dirty reports it as dirty.
In a container the analyser runs on lib_node_image with the newest LTS line like every other one, so an engines.node that admits only Node 14 or 16 cannot keep it from loading. The build steps run on the Node line the app declares all the same: the directory of that line from nodejs.org, the one make build runs on, is mounted read-only beside it, and its bin goes first on PATH for the build steps alone. For a node_image of your own, its /usr/local, where the Node images keep node, npm and corepack, is copied out of a container that is created from that image and never started, and mounted in its place. An app on an end-of-life Node line is rebuilt like any other, and the runtime line of the report names the Node the build ran on.
The checkout is read and never written to — the build output in it stays where it is, nothing is installed into it, and the scratch tree is removed again.
Important
This is the one analyser that installs and builds, so it needs the network and takes minutes rather than a second. Where it runs it needs git and npm: in a container lib_node_image brings git and the mounted Node line the npm, with RUNTIME=bare the host brings both.
The App Store takes the changelog of a release from the tarball and nowhere else. It reads exactly two kinds of file there:
| File | Read as |
|---|---|
CHANGELOG.md |
English |
CHANGELOG.<code>.md |
one per language of the store's list, such as CHANGELOG.de.md or CHANGELOG.pt-br.md
|
Each is decoded as UTF-8, and what the store shows is the section under the heading that names the version in appinfo/info.xml: a line that begins with ## 1.2.3, ## [1.2.3] or ## v1.2.3. A version with a - is a pre-release and is read from ## [Unreleased], as a nightly is. Whatever the store does not find, the release is published without, and nothing reports it.
A tagged pre-release is a release of its own all the same, and changelog-lint holds it to the section that names it, ## [1.2.3-rc.1], as make changelog writes it. The store shows that section on the page of the stable 1.2.3, below that release's own section. On the page of the pre-release itself it shows ## [Unreleased]; the lint reports that with [i] and does not demand it.
changelog-lint repeats that reading on the checkout, with the rules of the App Store's release parser:
-
[!]noCHANGELOG.md -
[!]a name that differs only in its spelling (changelog.md,CHANGELOG.MD,CHANGELOG.De.md,CHANGELOG.pt_BR.md), together with thegit mvthat fixes it -
[!]a language code the store does not know;en-auanden-gbare left out of its list -
[!]a file that is not valid UTF-8 -
[!]a changelog without a section for this version, or with an empty one; a byte order mark that hides the heading on the first line is named as the reason -
[i]a pre-release, whose own page on the store shows the Unreleased section instead of its section -
[i]a changelog under a name the store never reads (CHANGES.md,HISTORY.md,CHANGELOG) -
[i]aCHANGELOG.en.md, which the store reads as English afterCHANGELOG.mdand in its place -
[ok]the number of lines the store shows, per language
The store's language list is written into the analyser, as the Nextcloud majors are into consistency-audit, and the report names the Django release it is taken from. ncmake ships CHANGELOG.md and every CHANGELOG.*.md, each with its .license sidecar (see Building and packaging). make changelog writes the English section; a translation is written by hand.
Unlike consistency-audit and build-verify, changelog-lint exits 1 on a [!] finding. make tag runs it before its confirmation and refuses on that exit code; make tag NO_CHANGELOG_LINT=1 tags regardless (see Releasing). Elsewhere the exit code stops a chain of commands, or fails a CI job, which holds a merge back only where branch protection makes it a required check.
A Nextcloud server states the PHP versions it runs on in its lib/versioncheck.php, and the app states the servers it runs on in appinfo/info.xml. Between the two lies every PHP the app meets in the field. make build resolves the dependencies on the lowest of them, so a package that requires a PHP below the highest one installs without complaint, and a file that does not parse on another PHP does not show either, until a server runs it. php-compat checks two of them:
| PHP | What it is |
|---|---|
| the floor | the PHP make build resolves against: <php min-version>, raised to the floor of the newest end-of-life Nextcloud major (see How ncmake understands your app) |
| the ceiling | the newest PHP minor that the server of <nextcloud max-version> runs on, read from its stable branch, or from master for a major without one; a higher <php max-version> raises it, but no further than the ceiling of master |
PHP=all checks every minor in between as well.
A <php max-version> above the ceiling of master runs once more, apart from these, on the PHP ahead of master: in ghcr.io/ernolf/ncmake-php:<minor>-next. NEXT=1 runs ghcr.io/ernolf/ncmake-php:next, the newest of those minors, for an app that does not declare it.
Each runs in the build image of that minor, ghcr.io/ernolf/ncmake-php:<minor>, which carries the extensions a Nextcloud server offers, and does two things there:
-
composer check-platform-reqs --no-dev: whether every installed package, and the app's owncomposer.json, accepts this PHP -
php -loverappinfo/,lib/,templates/andvendor/: whether every PHP file that ships parses on this PHP, and what this PHP reports as deprecated in it. The lint loads no extension, since a parse calls none, runs as many PHP processes as the runtime has processors and, from PHP 8.3 on, lints 64 files per process
The findings:
-
[!]a<php max-version>below the ceiling: a server on a newer PHP refuses to enable the app -
[!]a<php max-version>above the ceiling of master: no server runs that PHP yet, so nobody has checked the app on it, and once a server allows it, it enables the app there. Like a<nextcloud max-version>above master inconsistency-audit, the declaration is the finding, not a run that could not be checked -
[!]a package, or the app, that requires a PHP the check ran on does not meet -
[!]a shipped PHP file that does not parse -
[!]a PHP version that could not be checked, an image that could not be pulled for instance -
[i]a deprecation or warning while parsing, grouped by package;--detailslists every file -
[i]an extension the build image lacks, which depends on how a server builds its PHP rather than on the app -
[i]a file of a dev package invendor/, whichmake builddoes not ship -
[i]novendor/and nocomposer.lock, so composer had nothing to check -
[ok]the PHP versions checked, and how many files parse on them
The run ahead of master prepares the app; it does not show that a server runs it. Its findings carry a php-next- id and are named after the exact PHP, for example 8.6.0RC3, and a [!] among them is reported as [i], so they never change the exit code. A php-next finding states whether that PHP has a release yet, and php-max-above-master says the same, or that the -next image could not be run. A -next image whose PHP master already allows is reported as php-next-none and left out.
It reads what is installed, so run it after make build: then vendor/ holds what ships. Without vendor/, composer reads composer.lock, and only the files of the app are linted. The output of each run stays in build/php-compat/<minor>, that of the run ahead of master in build/php-compat/next. It exits 1 on a [!] finding, so a CI job that runs it after make build fails on it. With RUNTIME=bare it runs on the PHP of the host, once per minor, and says that it did; the run ahead of master needs its image, so it is left out, and the output says so.
ARGS is passed to the analyser; a bare word in it is the checkout to read (default: the current directory). For consistency-audit:
ARGS |
What it does |
|---|---|
| (none) | this app |
‑‑details |
adds the tables behind the findings |
"‑‑details ‑‑top=0" |
and no row limit in those tables |
‑‑json |
the whole result, findings included, and nothing else |
‑‑help |
the analyser's own option list |
For build-verify:
ARGS |
What it does |
|---|---|
| (none) | this app, where its bundler config writes, else js/ and css/
|
‑‑build=dist |
another output directory |
‑‑details |
adds the per-file tables, with the byte counts of both builds |
‑‑once |
one build instead of two |
‑‑fresh |
the output was built by make build right before, on this working tree |
‑‑keep |
keeps the scratch tree and the log each build step writes |
‑‑json |
the whole result, findings included, and nothing else |
‑‑help |
the analyser's own option list |
For changelog-lint:
ARGS |
What it does |
|---|---|
| (none) | this app, this version |
‑‑nightly |
the section a nightly is read from |
‑‑json |
the whole result, findings included, and nothing else |
‑‑help |
the analyser's own option list |
For php-compat:
| What it does | |
|---|---|
| (none) | the floor and the ceiling |
PHP=all |
every minor in between too |
NEXT=1 |
and the newest PHP ahead of master |
ARGS=‑‑details |
every file behind the findings |
ARGS=‑‑json |
the whole result, findings included, and nothing else |
ARGS=‑‑help |
the analyser's own option list |
make consistency-audit
make build-verify ARGS="--details"
make changelog-lint
make build-verify ARGS=--json > verify.json
make build && make build-verify ARGS=--fresh
make consistency-audit JSON=audit.json
make build && make php-compat PHP=all
make build && make php-compat NEXT=1Each of the four targets takes JSON=<file>, which writes the JSON of ARGS=--json to <file> as well, from the same run, and the report is printed as without it. The file can lie outside the app; it is replaced once the run is complete, and a run that stops on an error writes none.
--keep and the build logs only survive with RUNTIME=bare and Node on the host: in a container the scratch tree lives in the container's own /tmp and goes with it.
Only the app itself is mounted into the container, so auditing a checkout outside it needs RUNTIME=bare too.
make dev-init fetches mk/audit.mk along with the other developer modules, so someone who only clones your app to build it never sees these targets.
The analysers are dependency-free ESM files and share the report core with the Bundle analysis, so findings read the same way everywhere. They live in the per-machine ncmake cache, are fetched on first use and refreshed on the same TTL and ETag terms as the modules, and are mounted read-only at /ncmake into the throwaway Node container. The fetch happens only when one of the targets is on the command line, so no other target in any app pays for it. php-compat runs its PHP versions in the build images of ncmake and hands only their output to the analyser.
Why does build-verify flag a file in js/ when js/ is git-ignored? Because make dist packs the working tree: a stale build in it ships. The finding is about the bytes a release would carry, not about what git tracks.
What do I do with a differing file? make dist-clean && make build, then commit what the build wrote. Run build-verify again and the file is identical.
A file is named as not reproducible. Is that my bug? Not necessarily: a build that writes a timestamp or an unstable order into its output differs from itself. The verifier names such a file and passes no verdict on it, because nothing can be concluded from a build that does not reproduce.
Can CI run them? consistency-audit and changelog-lint cost a second, and all consistency-audit reads from the network is the list of release tags of nextcloud/server, the version.php of its master and, for an app with a package.json, the Node.js release schedule, which makes them cheap anywhere. build-verify installs and builds twice, so it belongs in a job that is allowed to take minutes. consistency-audit and build-verify are reporting targets, not gates: use ARGS=--json for a machine-readable result, or JSON=<file> to get it beside the report from the same run. After a make build in the same job, ARGS=--fresh tells build-verify so, and where that output matches its first build, the second build does not run. changelog-lint exits 1 on a [!] finding, so its job fails where the App Store would publish the release without its changelog; made a required check, that failure holds the merge back. php-compat needs the network for versioncheck.php and the images, and runs one image per PHP version; it exits 1 on a [!] finding as well, so it fits a job after make build.
