π€ Generated by the Daily AI Assistant
Part of #181.
Problem
.github/workflows/ci.yaml shows uneven negative-path test coverage across the 15 composite actions:
| Action |
Negative-path test? |
Notes |
aggregate-job-checks |
β
|
test-β¦-failure, test-β¦-unknown (uses continue-on-error + outcome assertion) |
setup-agent-skills |
β
|
test-β¦-missing-input, test-β¦-malformed-line |
update-agent-skills |
β
|
test-β¦-missing-dir |
approve-pr |
β |
only the happy-path |
create-issues-from-todos |
β |
only the happy-path |
enable-auto-merge-on-pr |
β |
only the happy-path (dry-run) |
login-to-ghcr |
β |
only the happy-path |
run-dotnet-tests |
β |
only the matrix happy-path |
setup-go-toolchain |
β |
happy-path + private-modules variant; no failure mode |
setup-ksail-cli |
β |
matrix happy-path |
sync-github-labels |
β |
only the happy-path |
upload-coverage |
β |
smoke test only |
upsert-issue |
β
(sort of) |
create + close β covers two states, not error paths |
The well-covered ones use a consistent pattern (continue-on-error: true + outcome assertion) that's directly transferable. Regressions on error handling (bad inputs, missing files, missing config, malformed YAML, wrong arguments) wouldn't be caught today.
Proposed direction
For each action in the β rows above, add one negative-path test that exercises a clear, mechanically-reachable failure mode. Examples:
sync-github-labels β missing/invalid config-file.
login-to-ghcr β missing/empty github-token (verify clean failure, not a crash).
enable-auto-merge-on-pr β invalid merge-method (e.g. merge-method: "rebasee").
upload-coverage β missing file:; or unknown language:.
setup-go-toolchain β invalid go-version: string (e.g. "banana").
upsert-issue β missing both body and body-file (the action's documented error path).
setup-ksail-cli β no obvious negative path; may be skippable, document why in the PR body if so.
run-dotnet-tests β non-existent working-directory.
approve-pr / create-issues-from-todos β gated by if: on devantler-only PRs and use real GitHub App auth, harder to write a public negative test without leaking PAT scope; investigate and document in the PR if not feasible.
Each test follows the existing pattern (use continue-on-error: true on the action invocation, then a follow-up bash step that asserts steps.<id>.outcome == 'failure'). One PR per action or one consolidated PR β the PR author's judgement; consolidated is preferred for review efficiency unless the test-count gets unwieldy.
Acceptance criteria
Out of scope
- Extracting a shared "negative-path harness" reusable workflow β if scaffolding duplicates >2β3 times, capture as a follow-up against
devantler-tech/reusable-workflows per the monorepo AGENTS.md Holistic review rule, but don't pre-emptively extract.
- Re-running
actionlint policy in CI (already part of zizmor / standard checks).
Part of #181.
Problem
.github/workflows/ci.yamlshows uneven negative-path test coverage across the 15 composite actions:aggregate-job-checkstest-β¦-failure,test-β¦-unknown(usescontinue-on-error+ outcome assertion)setup-agent-skillstest-β¦-missing-input,test-β¦-malformed-lineupdate-agent-skillstest-β¦-missing-dirapprove-prcreate-issues-from-todosenable-auto-merge-on-prlogin-to-ghcrrun-dotnet-testssetup-go-toolchainsetup-ksail-clisync-github-labelsupload-coverageupsert-issuecreate+closeβ covers two states, not error pathsThe well-covered ones use a consistent pattern (
continue-on-error: true+outcomeassertion) that's directly transferable. Regressions on error handling (bad inputs, missing files, missing config, malformed YAML, wrong arguments) wouldn't be caught today.Proposed direction
For each action in the β rows above, add one negative-path test that exercises a clear, mechanically-reachable failure mode. Examples:
sync-github-labelsβ missing/invalidconfig-file.login-to-ghcrβ missing/emptygithub-token(verify clean failure, not a crash).enable-auto-merge-on-prβ invalidmerge-method(e.g.merge-method: "rebasee").upload-coverageβ missingfile:; or unknownlanguage:.setup-go-toolchainβ invalidgo-version:string (e.g."banana").upsert-issueβ missing bothbodyandbody-file(the action's documented error path).setup-ksail-cliβ no obvious negative path; may be skippable, document why in the PR body if so.run-dotnet-testsβ non-existentworking-directory.approve-pr/create-issues-from-todosβ gated byif:on devantler-only PRs and use real GitHub App auth, harder to write a public negative test without leaking PAT scope; investigate and document in the PR if not feasible.Each test follows the existing pattern (use
continue-on-error: trueon the action invocation, then a follow-upbashstep that assertssteps.<id>.outcome == 'failure'). One PR per action or one consolidated PR β the PR author's judgement; consolidated is preferred for review efficiency unless the test-count gets unwieldy.Acceptance criteria
sync-github-labels,login-to-ghcr,enable-auto-merge-on-pr,upload-coverage,setup-go-toolchain,upsert-issue,run-dotnet-tests.setup-ksail-cli,approve-pr,create-issues-from-todos) have a one-paragraph justification in the PR body for why they were skipped.continue-on-error: trueβoutcomeassertion).zizmorpasses.Out of scope
devantler-tech/reusable-workflowsper the monorepoAGENTS.mdHolistic review rule, but don't pre-emptively extract.actionlintpolicy in CI (already part of zizmor / standard checks).