Skip to content

CI reliability overhaul: kill phantom failures, unblock releases, gate real signal (0283) - #414

Merged
crs48 merged 13 commits into
mainfrom
claude/0283-ci-failure-patterns-and-pipeline-health
Jul 8, 2026
Merged

CI reliability overhaul: kill phantom failures, unblock releases, gate real signal (0283)#414
crs48 merged 13 commits into
mainfrom
claude/0283-ci-failure-patterns-and-pipeline-health

Conversation

@crs48

@crs48 crs48 commented Jul 8, 2026

Copy link
Copy Markdown
Owner

Implements docs/explorations/0283 — CI failure patterns and pipeline health.

Over the last ~500 Actions runs, ~75% of failures came from four causes unrelated to code correctness, while the one real breakage (Electron Release) went unnoticed in the noise.

Tier 1 — stop the bleeding

  • mobile-e2e.yml: quoted the two run: lines whose : made the file unparseable — GitHub emitted a phantom failed run on every push (48/48 failures, ~40% of all failure volume).
  • audiotee Package.swift: deployment floor .macOS(.v14).macOS("14.2") (AudioHardwareCreateProcessTap is 14.2+). Every Electron Release macOS build since 0279 failed on this; verified locally with swift build -c release. Runtime capability selection already gates on darwin ≥ 23.4 before probing the helper.
  • Fallow: removed from PR CI entirely (user decision — stronger than the exploration's report-only option). Weekly schedule + manual dispatch keep the hard --fail-on-issues gate; turbo cache re-enabled. It was the slowest job (~17 min) and failed 44% of PR runs on advisory notes.
  • visual-capture: job-level continue-on-error so the self-described informational check can't red a PR.

Tier 2 — prevent the classes

  • actionlint 1.7.12 in the required lint job + guarded pre-commit pass (shellcheck severity floor warning in both). Fixed what it flags: untrusted head.ref inlined in ai-changeset.yml (moved to env), unquoted command substitutions in schema-check.yml, unused loop vars.
  • .husky/prepare-commit-msg: dependency-free DCO auto-sign-off (survives --no-verify; merges/squashes untouched). DCO failed 76% of recent runs.
  • scripts/check-storybook-shim.mjs (pnpm check:storybook-shim): walks the story import graph with storybook's own alias map and fails the lint job in seconds when the plugins shim misses a name — the drift that broke Visual UI Capture in 0279, 0280, and feat(web): labs toggle + workspace editing affordances (0282) #412. Backfills ImporterContribution.
  • plugins-registry.yml: daily rebuild now refreshes a standing plugins-registry/data PR (signed off, skip-changelog) instead of pushing to protected main, which repo rules rejected every time.

Tier 3 — ratchets

  • Native Helpers workflow: path-filtered macos-14 swift build on PRs touching apps/electron/native/**, so availability drift surfaces pre-release.
  • scripts/changelog/assert-fragment.mjs Stop-hook nudge (mirrors the changesets hook): one-shot turn-end reminder when app/package source changed without a changelog fragment.

Deferred (calendar-gated, unchecked in the doc)

  • Adding DCO to required checks after ≥1 week of green.
  • Two-week failure-rate re-measurement.

Expected outcome: failed-run rate drops from ~21% to ~2-3%, and a red ✗ regains meaning.

🤖 Generated with Claude Code

xNet Test and others added 10 commits July 8, 2026 10:46
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: xNet Test <test@xnet.dev>
mobile-e2e.yml had an unparseable plain scalar (': ' inside the run: echo
lines), so GitHub created a phantom failed run on every push to every
branch (48/48 failures). Quote both lines.

xnet-audiotee declared .macOS(.v14) while calling
AudioHardwareCreateProcessTap (14.2+), failing every Electron Release
macOS build since 0279 merged. Raise the deployment floor to 14.2;
runtime capability selection already gates on darwin >= 23.4 (macOS
14.4) in core-audio-tap.ts before probing the helper, so no runtime
change is needed. Verified locally with swift build -c release.

Signed-off-by: xNet Test <test@xnet.dev>
Fallow was the pipeline's slowest job (~17 min avg), failed 44% of PR
runs on advisory notes despite never being a required check, and its
concurrency group cancelled 6 of the last 27 runs mid-flight. Remove the
pull_request trigger (user decision — stronger than the report-only
option in exploration 0283): the weekly scheduled run and manual
dispatch keep the hard --fail-on-issues gate and SARIF upload. Also
re-enable the turbo cache and simplify the now-single-base ref plumbing.

Signed-off-by: xNet Test <test@xnet.dev>
…a PR

visual-capture.yml documents itself as informational but a Build
Storybook failure still failed the capture job — a red X on 29% of
recent runs. Add job-level continue-on-error to honour the contract.

The real signal it carried (the storybook plugins shim drifting from
what the story bundle imports out of @xnetjs/plugins — broke three times
across 0279/0280/#412) moves into the required lint job:
check-storybook-shim.mjs walks the story import graph with the same
alias map storybook uses and fails in seconds when a name is missing
from the shim. Backfills the one current gap (ImporterContribution,
type-only so the build never noticed).

Signed-off-by: xNet Test <test@xnet.dev>
One unparseable workflow file produced 48 phantom failed runs (0283);
nothing validated .github/workflows. Add actionlint 1.7.12 to the
required lint job and a guarded pre-commit pass for staged workflow
files (advisory when the binary is absent). shellcheck severity floor
is 'warning' in both so local and CI agree.

Fix what it flags at that level: ai-changeset.yml inlined the
attacker-controlled PR head ref into a script (moved to an env var),
schema-check.yml had unquoted command substitutions inside JSON echoes,
and three retry loops used an unused loop variable.

Signed-off-by: xNet Test <test@xnet.dev>
The dco.yml check failed on 22 of 29 recent runs because agent-driven
commits never carry Signed-off-by (0283). prepare-commit-msg adds the
trailer from the configured git identity — dependency-free so it works
in worktrees without node_modules, idempotent, and untouched by
--no-verify (git only skips pre-commit/commit-msg). The CI check stays
as the backstop.

Signed-off-by: xNet Test <test@xnet.dev>
The daily rebuild pushed directly to main, which the repo ruleset
rejects ('changes must be made through a pull request') — the cron
failed every day the index changed (0283). Refresh a standing
plugins-registry/data PR via peter-evans/create-pull-request, labelled
skip-changelog and signed off; merging stays deliberate, same as the
changesets release PR (and with the same no-CI-on-GITHUB_TOKEN-PRs
admin-merge caveat).

Signed-off-by: xNet Test <test@xnet.dev>
Native Helpers workflow (path-filtered, macos-14) compiles the audiotee
Swift helper on PRs that touch apps/electron/native/**, so macOS
availability drift surfaces before release time instead of after three
failed release runs (0283).

assert-fragment.mjs mirrors the changesets Stop hook: when the branch
changed app/package source without a changelog fragment, nudge once at
turn-end (the required changelog-section check would otherwise reject
the PR with feedback arriving only at PR time). Deliberately one-shot
and skippable — skip-changelog is a legitimate answer.

Signed-off-by: xNet Test <test@xnet.dev>
Signed-off-by: xNet Test <test@xnet.dev>
Signed-off-by: xNet Test <test@xnet.dev>
@crs48
crs48 temporarily deployed to pr-414 July 8, 2026 18:05 — with GitHub Actions Inactive
The dispatch-only stub never built @xnetjs/* deps, so build:web died on
TS2307 the first time the (now-parseable) workflow actually ran.

Signed-off-by: xNet Test <test@xnet.dev>
@crs48
crs48 temporarily deployed to pr-414 July 8, 2026 18:09 — with GitHub Actions Inactive
@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

🖼️ UI changes in this PR

No UI changes detected in this PR.

Signed-off-by: xNet Test <test@xnet.dev>
github-actions Bot added a commit that referenced this pull request Jul 8, 2026
@crs48
crs48 temporarily deployed to pr-414 July 8, 2026 18:15 — with GitHub Actions Inactive
Signed-off-by: xNet Test <test@xnet.dev>
@crs48
crs48 temporarily deployed to pr-414 July 8, 2026 18:21 — with GitHub Actions Inactive
@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Preview removed for PR #414.

github-actions Bot added a commit that referenced this pull request Jul 8, 2026
@crs48
crs48 merged commit f5f256c into main Jul 8, 2026
17 checks passed
@crs48
crs48 deleted the claude/0283-ci-failure-patterns-and-pipeline-health branch July 8, 2026 18:29
github-actions Bot added a commit that referenced this pull request Jul 8, 2026
crs48 added a commit that referenced this pull request Jul 8, 2026
#416)

Follow-up to #414 (exploration 0283). The 14.2 deployment-floor fix got
audiotee compiling again on the Electron Release macOS jobs — which
exposed the next line of the same step: SwiftPM symlinks
`.build/release` to the arch directory, so the normalising `cp` copied
the binary onto its own inode and exited 1 on both arches.

Skip the copy when source and destination are the same file (`[ "$SRC"
-ef "$DST" ]`).

Validated via branch dispatch (run 28967007377): both `build-macos` jobs
now pass `Build xnet-audiotee helper`. Also checks off the Electron
Release and Plugins Registry validation boxes in the 0283 doc (registry
validated by dispatch → PR #415, merged).

🤖 Generated with [Claude Code](https://claude.com/claude-code)
crs48 added a commit that referenced this pull request Jul 8, 2026
## Summary
The dispatch-only `mobile-e2e.yml` workflow (exploration 0238, un-broken
in #414) failed at **Add Android project**: `apps/mobile` had no
Capacitor dependencies and every script shelled out to `npx --yes
@capacitor/cli@latest`, but Capacitor 8's `cap add android` requires
`@capacitor/android` installed in the project.

- Pin `@capacitor/cli`, `@capacitor/core`, `@capacitor/android`,
`@capacitor/ios` at **8.4.1** as devDependencies of `xnet-mobile-shell`
(private, no changeset needed).
- Switch all scripts from npx-latest to the pinned CLI (`pnpm exec cap
…`) — reproducible and no surprise majors.
- Update the README prerequisites section, which documented the npx
approach.

## Verification
`gh workflow run mobile-e2e.yml --ref claude/upbeat-napier-8b894c -f
platform=android` → [run
28968475239](https://github.com/crs48/xNet/actions/runs/28968475239)
**success**: every step green through the `Run Maestro flows` TODO,
including `[success] android platform added!` and Capacitor sync.
Emulator/Maestro wiring remains the documented TODO.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant