Skip to content

CI toil-reduction sweep: registry determinism + codex auto-refresh + local CI dry-run (3 beads)#261

Merged
boshu2 merged 3 commits into
mainfrom
crank/ci-toil-reduction
May 7, 2026
Merged

CI toil-reduction sweep: registry determinism + codex auto-refresh + local CI dry-run (3 beads)#261
boshu2 merged 3 commits into
mainfrom
crank/ci-toil-reduction

Conversation

@boshu2
Copy link
Copy Markdown
Owner

@boshu2 boshu2 commented May 7, 2026

Summary

Closes 3 beads (soc-k47k, soc-7qq9, soc-ws40) filed during the 2026-05-07 retrospective on CI/push-gate toil. See .agents/learnings/2026-05-07-ci-push-gate-toil-pattern.md for the data: 9 push iterations across 4 PRs, ~45min wall-clock + thousands of orchestrator tokens spent diagnosing identical-looking failures with different root causes.

Three compounding fixes shipped together to amortize the CI cycle cost (the very pattern the learning argues for).

What changed

soc-k47k — registry-check determinism

soc-7qq9 — PostToolUse codex auto-refresh

  • New hooks/postedit-codex-refresh.sh fires after Edit/Write of any skills/<name>/SKILL.md, references/, scripts/, or schemas/ file.
  • Auto-runs scripts/refresh-codex-artifacts.sh --scope head so codex hashes stay current; operator never sees the parity warning.
  • Registered in hooks/hooks.json (matchers Edit + Write, 10s timeout). Embedded copy synced via make sync-hooks.
  • Disable via AGENTOPS_HOOKS_DISABLED=1 or AGENTOPS_CODEX_AUTOREFRESH_DISABLED=1.
  • Best-effort: failures log to stderr but always exit 0 (informational hook — never blocks the editor).

soc-ws40 — local CI-equivalent dry-run

  • New scripts/test-ci-deterministic-gates.sh runs the 4 deterministic CI gates in <30s: registry-check, skill-lint, heal-skill --strict, codex artifact metadata.
  • Non-fail-fast: reports all failures together so you can fix them in one diagnostic round instead of N push iterations.
  • Currently runs in <30s on a clean tree (vs. ~5min for one CI canary cycle).

Test coverage

  • 4 smoke tests for postedit-codex-refresh.sh in tests/hooks/test-hooks.sh (non-skill path no-op, both disable env vars, orphan path).
  • scripts/check-test-fixture-parity.sh passes (was failing without the new tests).

Test plan

  • bash scripts/test-ci-deterministic-gates.sh → 4/4 PASS
  • bash tests/hooks/test-hooks.sh (post-edit-codex-refresh suite) → 4/4 PASS
  • bash scripts/check-test-fixture-parity.sh → exit 0
  • cd cli && go test ./... → 11858 passing
  • Pre-push gate (fast) passed
  • Live verification: edit any skill/SKILL.md and observe skills-codex/<name>/.agentops-generated.json auto-refresh

How this composes

With #1 + #2 + #3 wired:

Together: the CI surprises that drove this session's toil should be catchable locally before push. Future PR CI cycles should compress from N iterations to 1.

Closes

  • soc-k47k (registry-check determinism via git ls-files)
  • soc-7qq9 (PostToolUse codex auto-refresh)
  • soc-ws40 (scripts/test-ci-deterministic-gates.sh local dry-run)

boshu2 added 2 commits May 7, 2026 19:18
…fresh + local CI dry-run (soc-k47k, soc-7qq9, soc-ws40)

Three compounding fixes filed during the 2026-05-07 retrospective on
CI/push-gate toil. See `.agents/learnings/2026-05-07-ci-push-gate-toil-pattern.md`
for the data: 9 push iterations across 4 PRs, ~45min wall-clock + thousands
of orchestrator tokens spent diagnosing identical-looking failures with
different root causes.

soc-k47k — registry-check determinism:
- scripts/generate-registry.sh build_knowledge_stores() now walks
  `git ls-files .agents/` instead of filesystem `find`. CI's clean
  checkout and a session-built local checkout produce identical output
  because both consult the git index (deterministic).
- Removed the `[[ -d .agents ]] || return` early-exit so behavior is
  uniform regardless of working-tree state.
- Eliminates the dominant single source of failed-push retries this
  session (5 iterations across PRs #255 and #260 alone).

soc-7qq9 — PostToolUse codex auto-refresh:
- New hooks/postedit-codex-refresh.sh fires after Edit/Write of any
  skills/<name>/SKILL.md, references/, scripts/, or schemas/ file.
  Auto-runs `scripts/refresh-codex-artifacts.sh --scope head` so codex
  hashes stay current; operator never sees the parity warning.
- Registered in hooks/hooks.json (matchers Edit + Write, 10s timeout).
- Embedded copy synced via `make sync-hooks`.
- Disable via AGENTOPS_HOOKS_DISABLED=1 or
  AGENTOPS_CODEX_AUTOREFRESH_DISABLED=1.
- Best-effort: failures log to stderr but always exit 0.

soc-ws40 — local CI-equivalent dry-run:
- New scripts/test-ci-deterministic-gates.sh runs the 4 deterministic
  CI gates that have historically surprised PRs at push time: registry-
  check, skill-lint, heal-skill --strict, codex artifact metadata.
- Non-fail-fast: reports all failures together so you can fix them in
  one diagnostic round instead of N push iterations.
- Currently runs in <30s on a clean tree (vs. ~5min for one CI canary
  cycle).

Together these compound: with #1 + #2 + #3 wired, the CI surprises that
drove this session's toil should be catchable locally before push.

go test ./... → 11858 passed in 50 packages.
bash scripts/test-ci-deterministic-gates.sh → 4/4 pass.
Required by scripts/check-test-fixture-parity.sh — every hooks/*.sh must
have at least one test reference. 4 smoke tests:
- non-skill path → silent exit 0
- AGENTOPS_HOOKS_DISABLED=1 short-circuits
- AGENTOPS_CODEX_AUTOREFRESH_DISABLED=1 short-circuits
- skill path outside any git repo → no-op
@boshu2 boshu2 merged commit 4cf012b into main May 7, 2026
47 of 48 checks passed
@boshu2 boshu2 deleted the crank/ci-toil-reduction branch May 7, 2026 23:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant