Qualification

Run and preserve qualification gates

The qualification script is a collection mechanism for a source revision, not a universal certificate. It should connect a concrete command run to logs, test XML, lint findings and the APK identity. Re-running it is useful when source or build inputs change. Re-running it repeatedly without changes is less useful than completing the runtime checks that automated unit tests cannot cover.

Start from a clean, identified checkout

Resolve the source revision and check the working tree before the gate. A fresh clone makes the phrase fresh-clone gate meaningful. If you run in a dirty checkout, preserve the diff and describe the result accordingly. Do not label a local tree as the public commit when uncommitted implementation changes affected the build. Evidence that accurately describes a dirty run is still useful, but it answers a different question.

git rev-parse HEAD
git status --short
bash scripts/qualify.sh
bash gradlew -p tools/receipt-checker test installDist

Run the checker separately because it is a separate project. Record each exit status. A shell wrapper that continues after failures can make an overall command appear successful, so inspect the script's behavior and captured outputs. The expected historical counts are useful for detecting a discrepancy; they are not numbers to force onto a new report. If the count changes, reconcile the test inventory rather than editing the summary.

Read the result files

JUnit XML identifies executed suites, failures and skipped cases. Lint output identifies static Android findings. A live node-agent integration log describes a different environment from a pure JVM test. An instrumented-test APK being assembled is not an instrumented test being run on Android. Write those distinctions into the qualification note. Readers should never have to guess whether a green build includes Keystore behavior on a device.

A full gate can pass while the wallet journey remains unqualified. That is expected because wallet authorization requires a runtime with a compatible wallet and user approval. Likewise, receipt tests can verify synthetic records without producing a receipt from a real chain operation. Automated tests should protect logic; runtime evidence should establish behavior in the environment that actually uses that logic.

Preserve useful files, not everything

Commit the relevant XML, logs, manifest and summary under a stable evidence directory. Review files before adding them so tokens, local secrets, unrelated downloads and personal documents do not enter the repository. APK distribution is usually better handled as a release asset or authenticated artifact than as repeated large binaries in source history. If an APK is stored elsewhere, preserve its full hash and location.

The evidence commit may come after the tested source revision. State that explicitly. A later evidence-only merge should not require pretending the binary was built from that merge. If you rebuild at the later revision, you have a new build identity and should keep that run separate. This is a straightforward provenance rule, not an obstacle to shipping.

Define acceptance before collecting results

A suitable qualification summary lists command, source, environment, exit status, suite totals, lint status and exclusions. A suitable runtime summary lists the action and the observation that would establish PASS. This separation makes it possible to close a gate without inflating another. The exercise for a maintainer is to reproduce one suite, compare its XML with the historical evidence and explain any difference in plain language before updating public numbers.

---

Wiki home · Qualification updates · Sources and credits