Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions .changeset/fix-workflows-skip-generated.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
---
"fix-workflows": minor
---

fix-workflows: stop reformatting generated workflows

The YAML fixer already skipped `*.lock.yml`, but the separate `oxfmt` step in
`action.yml` (and its `cli.ts` equivalent) globbed `.github/workflows/**/*.yml`
with no exclusions, so it reformatted the compiled agentic workflows the fixer had
deliberately left alone. The "this file needs updating" annotation that follows
then points a maintainer at compiler output, and applying it desyncs a committed
lock from `gh aw compile`.

Both invocations now exclude the generator-owned workflows via negated globs. One
list (`GENERATED_WORKFLOWS`) drives all three sites: the fixer calls
`isGeneratedWorkflow`, `cli.ts` derives its globs from
`generatedWorkflowSkipGlobs()`, and `action.yml`, whose bash step cannot import
either, has its hand-copied literals pinned to that function by a test. The
negations are recursive, matching the positive glob they subtract from.

`agentics-maintenance.yml` joins `*.lock.yml` in that set: gh-aw regenerates it
unconditionally and it is not named `*.lock.yml`, so nothing was skipping it; it
took 6 checkout-followed-by-setup violations and 12 `runs-on` rewrites on a file no
human can fix.

Negated globs rather than `ignorePatterns` in `.oxfmtrc.json`, because that config
lives in the action's own directory rather than the repo being formatted and its
patterns do not resolve against the working directory (verified: the lock was still
rewritten).

Observed on Khan/agent-settings#48, which is the first repo to run this action over
a directory containing compiled agentic workflows.
14 changes: 13 additions & 1 deletion actions/fix-workflows/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,12 +36,24 @@ runs:
core.setFailed(err.message)
}

# Without the negations this step reformats compiler output that the YAML
# fixer above correctly skips, which desyncs a committed lock from `gh aw
# compile` and makes the next step annotate a generated file with a fix
# nobody should apply. `ignorePatterns` in .oxfmtrc.json cannot do this: the
# config lives in the action's directory, not the repo being formatted, and
# its patterns do not resolve against the working directory.
#
# A bash step cannot import the TypeScript, so these literals are a copy of
# `generatedWorkflowSkipGlobs()` in index.ts; generated-workflows.test.ts
# fails if they drift from it.
- name: Format YAML files with oxfmt
shell: bash
run: |
npx --yes oxfmt@0.44.0 --write \
--config "${{ github.action_path }}/.oxfmtrc.json" \
".github/workflows/**/*.yml"
".github/workflows/**/*.yml" \
'!.github/workflows/**/*.lock.yml' \
'!.github/workflows/**/agentics-maintenance.yml'

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thought (non-blocking): The stated harm is the annotation, but the annotation step itself is left unguarded. Per the PR's own framing, the real damage is the Show fix instructions step blindly annotating everything git diff reports under .github/workflows/ (confirmed in action.yml: it loops over git diff --name-only with no filter). This fix closes the two known mutators but leaves the harmful-hint layer trusting them forever; filtering generated files out of the warning loop would make the stated failure impossible regardless of which upstream step touches them.

A sketch, not a committable replacement:

In the `Show fix instructions` loop, skip (or specially word the warning for) files matching the generated set — or, per the other finding, files carrying the `DO NOT EDIT.` marker — so the desync-inviting hint can never point at compiler output.


- name: Show fix instructions
shell: bash
Expand Down
17 changes: 14 additions & 3 deletions actions/fix-workflows/cli.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,10 @@
import {execSync} from "node:child_process";
import * as path from "node:path";

import fixWorkflows, {DEFAULT_SETUP_ACTION} from "./index";
import fixWorkflows, {
DEFAULT_SETUP_ACTION,
generatedWorkflowSkipGlobs,
} from "./index";

function parseArgs(argv: string[]): {
fixRunsOn: boolean;
Expand Down Expand Up @@ -44,13 +47,21 @@ const core = {

fixWorkflows({core, fixRunsOn, setupAction})
.then(() => {
// Format workflow files with oxfmt after fixing lint violations.
// Format workflow files with oxfmt after fixing lint violations, skipping
// the generator-owned ones for the same reason the fixer does (see
// GENERATED_WORKFLOWS in index.ts). Negated globs rather than
// .oxfmtrc.json ignorePatterns: the config lives beside this script, not in
// the repo being formatted, and its patterns do not resolve against the
// working directory.
const configPath = path.join(__dirname, ".oxfmtrc.json");
const skip = generatedWorkflowSkipGlobs()
.map((glob) => JSON.stringify(glob))
.join(" ");
console.log("Formatting workflow files with oxfmt..."); // eslint-disable-line no-console
execSync(
`npx --yes oxfmt@0.44.0 --write --config ${JSON.stringify(
configPath,
)} ".github/workflows/**/*.yml"`,
)} ".github/workflows/**/*.yml" ${skip}`,
{stdio: "inherit"},
);
})
Expand Down
81 changes: 81 additions & 0 deletions actions/fix-workflows/generated-workflows.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
import * as fs from "fs";
import {describe, expect, it} from "vitest";

import {generatedWorkflowSkipGlobs, isGeneratedWorkflow} from "./index.ts";

/**
* Which `.github/workflows` entries a generator owns, split from index.test.ts to
* keep that file inside its 1000-line cap.
*
* `GENERATED_WORKFLOWS` is the single source of truth for a set that three places
* act on: the fixer calls `isGeneratedWorkflow`, cli.ts derives its oxfmt globs
* from `generatedWorkflowSkipGlobs()`, and action.yml's bash step -- which cannot
* import either -- hand-copies those globs. The last of those is what this file's
* second describe block pins, because the three diverged once already: the YAML
* fixer skipped `*.lock.yml` while the oxfmt step reformatted it.
*/

const actionYml = fs.readFileSync(
new URL("./action.yml", import.meta.url),
"utf-8",
);

/**
* The oxfmt step's `run:` body alone, so that a quoted `!...` added to some other
* step cannot pass for one of the skip globs.
*/
function oxfmtStep(): string {
const afterName = actionYml.split(
"- name: Format YAML files with oxfmt",
)[1];
if (afterName == null) {
throw new Error(
"action.yml has no oxfmt step; rename the test with it",
);
}
return afterName.split("\n - name:")[0]!;
}

describe("isGeneratedWorkflow", () => {
// These files are compiler output. Fixing them is worse than useless: the fix
// is overwritten on the next `gh aw compile`, and the "this file needs
// updating" annotation invites a maintainer to desync a committed lock from
// its compiler.
it("skips compiled agentic workflow locks", () => {
expect(isGeneratedWorkflow("review.lock.yml")).toBe(true);
expect(isGeneratedWorkflow("autofix.lock.yml")).toBe(true);
});

it("skips gh-aw's maintenance workflow, which is not named *.lock.yml", () => {
expect(isGeneratedWorkflow("agentics-maintenance.yml")).toBe(true);
});

it("still checks hand-written workflows, including lookalikes", () => {
expect(isGeneratedWorkflow("node-ci.yml")).toBe(false);
expect(isGeneratedWorkflow("validate-workflows.yml")).toBe(false);
// Not a lock: the suffix test is `.lock.yml`, not `lock` anywhere.
expect(isGeneratedWorkflow("lock-threads.yml")).toBe(false);
expect(isGeneratedWorkflow("agentics-maintenance-notes.yml")).toBe(
false,
);
});
});

describe("action.yml's copy of the skip globs", () => {
// The only site that cannot derive its skip set from GENERATED_WORKFLOWS, so
// it is the only one that can silently fall behind a fourth entry. This test
// is the substitute for the import it cannot do.
it("matches generatedWorkflowSkipGlobs()", () => {
const literals = [...oxfmtStep().matchAll(/'(![^']+)'/g)].map(
(match) => match[1],
);
expect(literals).toEqual(generatedWorkflowSkipGlobs());
});

// Negating patterns the positive glob never selects would leave the locks
// unformatted for the wrong reason, and would quietly stop the negations
// being load-bearing if that glob were narrowed.
it("still formats workflows recursively, so the negations subtract from that set", () => {
expect(oxfmtStep()).toContain('".github/workflows/**/*.yml"');
});
});
63 changes: 62 additions & 1 deletion actions/fix-workflows/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -178,6 +178,67 @@ export function stepIgnoresSetup(step: YAMLMap): boolean {
// File discovery
// ---------------------------------------------------------------------------

/**
* Workflow files this action must not touch because a generator owns them: the
* fix would be overwritten on the next `gh aw compile`, and worse, a maintainer
* who follows our own "this file needs updating" hint desyncs the committed file
* from its compiler.
*
* `*.lock.yml` is the compiled agentic workflow; `agentics-maintenance.yml` is
* gh-aw's scheduled housekeeping workflow, which is regenerated unconditionally
* and (unhelpfully) is not named `*.lock.yml`.
*
* Matching by name rather than by content is deliberate, and the cheap
* content check is not the upgrade it looks like. Both files gh-aw emits do
* carry a `DO NOT EDIT.` marker in their first line, so the *fixer* could
* sniff for it and stop needing this list. The oxfmt step cannot: it selects
* files by glob, and a glob cannot read a header. Teaching only the fixer to
* sniff would put the two back out of step -- a marked-but-unlisted file would
* be skipped by the fixer and still reformatted by the formatter, which is
* exactly the bug this list exists to close. A content rule has to arrive at
* both sites at once (the formatter taking an explicit file list computed here
* instead of a glob), so it is left for when a third generated workflow makes
* that refactor worth it.
*/
export const GENERATED_WORKFLOWS = {
Comment thread
khan-actions-bot marked this conversation as resolved.
Comment thread
khan-actions-bot marked this conversation as resolved.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (non-blocking): The 'a glob cannot read a header' premise doesn't hold for a bash step, and it's what forces this whole name-list apparatus. The comment defers the content-marker rule because 'the oxfmt step selects files by glob, and a glob cannot read a header' — but the oxfmt step is shell: bash, which can trivially compute a marker-filtered file list (e.g. grep -L 'DO NOT EDIT.' .github/workflows/*.yml fed to oxfmt as explicit paths; verified both lock files and gh-aw output generally carry that marker — on line 3, incidentally, not 'their first line' as the comment states). That would let the fixer and both formatter invocations share one content rule today, deleting GENERATED_WORKFLOWS, generatedWorkflowSkipGlobs(), the hand-copied literals, and the 81-line test that string-parses action.yml to pin them. The deferred refactor looks cheaper than the drift-prevention machinery being added to avoid it.

A sketch, not a committable replacement:

Have the fixer skip files whose first few lines contain gh-aw's `DO NOT EDIT.` marker, and have both oxfmt invocations format an explicit file list filtered the same way (bash: `grep -L`; cli.ts: reuse the fixer's predicate), instead of maintaining a name/suffix list with a cross-file drift test.

suffixes: [".lock.yml"],
names: ["agentics-maintenance.yml"],
};

/** Whether a `.github/workflows` entry is generator-owned (see above). */
export function isGeneratedWorkflow(name: string): boolean {
return (
GENERATED_WORKFLOWS.suffixes.some((suffix) => name.endsWith(suffix)) ||
GENERATED_WORKFLOWS.names.includes(name)
);
}

/**
* `GENERATED_WORKFLOWS` as negated oxfmt globs, for the formatting step that
* runs after the fixer.
*
* The formatter selects files by glob rather than by `isGeneratedWorkflow`, so
* the skip set has to be expressible as globs too; deriving them here keeps
* cli.ts from carrying a second copy of the set. action.yml still spells them
* out (a composite action's bash step cannot import this module), and
* generated-workflows.test.ts asserts its literals against this function so
* that copy cannot drift.
*
* The `**` mirrors the recursive positive glob both call sites pass. It also
* matches files directly in `.github/workflows`, so one pattern covers both
* depths (verified against the pinned oxfmt@0.44.0).
*/
export function generatedWorkflowSkipGlobs(): string[] {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (non-blocking): generatedWorkflowSkipGlobs()'s exact returned globs are never asserted. The isGeneratedWorkflow predicate is well covered, but generatedWorkflowSkipGlobs() is only checked via expect(literals).toEqual(generatedWorkflowSkipGlobs()), which verifies action.yml matches the function, not that the function produces correct globs. A direct assertion of the exact strings would regression-lock the negated-glob format that the whole fix depends on.

Suggested change
export function generatedWorkflowSkipGlobs(): string[] {
it("produces recursive negated globs that subtract the generated set", () => {
expect(generatedWorkflowSkipGlobs()).toEqual([
"!.github/workflows/**/*.lock.yml",
"!.github/workflows/**/agentics-maintenance.yml",
]);
});

return [
...GENERATED_WORKFLOWS.suffixes.map(
(suffix) => `!.github/workflows/**/*${suffix}`,
),
...GENERATED_WORKFLOWS.names.map(
(name) => `!.github/workflows/**/${name}`,
),
];
}

function getFilesToCheck(): string[] {
const files: string[] = [];

Expand All @@ -186,7 +247,7 @@ function getFilesToCheck(): string[] {
if (
entry.isFile() &&
(entry.name.endsWith(".yml") || entry.name.endsWith(".yaml")) &&
!entry.name.endsWith(".lock.yml")
!isGeneratedWorkflow(entry.name)
) {
files.push(path.join(".github", "workflows", entry.name));
}
Expand Down
Loading