docs(skills): reduce automation noise and consolidate drafting improvements - #487
docs(skills): reduce automation noise and consolidate drafting improvements#487rachaelrenk wants to merge 3 commits into
Conversation
…ements Recurring docs agents were producing more PRs and Slack messages than the team could absorb. Three systemic causes, plus a batch of GitBook-era migration artifacts that left several skills unable to run as written. Shared conventions (skill-authoring-guidelines.md): - Add "One standing PR per automation": stable branch and title, look before creating, add to the existing PR rather than opening another. - Invert "Slack notifications" to actionable-only. The old rule required posting on every run and was the direct cause of the channel noise. Its silent-failure rationale is preserved by requiring a run log instead. - Rewrite "Log availability" so outer loops read the log branch rather than main, and never merge the standing log PR as a workflow step. Cron correctness: - `0 17 1-7 * 1` is not "first Monday". Cron ORs day-of-month with day-of-week, so it fired ~11 times a month and produced four conflicting PRs in six days. Replace with `0 17 * * 1` plus an in-skill first-week guard in improve-drafting-skills, improve-aeo-crosslink-skill, and improve-404-monitor-skill. PR reuse applied to: improve-drafting-skills, weekly-404-monitor, afdocs-fix, sync-error-docs, sync_terminology, sync-openapi-spec, improve-aeo-crosslink-skill, improve-404-monitor-skill. update-changelog keeps one PR per release (correct) but now detects stacked release PRs. Slack volume: aeo_crosslink_audit no longer posts on no-change runs; weekly-404-monitor gates on threshold and folds its Phase 2 results into a single message instead of two; afdocs-audit posts only on regression or a blocked audit, backed by a new run log for the baseline. Migration artifacts: a find-and-replace during the GitBook-to-Astro move substituted descriptions into file paths. sync-error-docs referenced `astro.config.mjs (sidebar config)` and `vercel.json (redirects)` as real paths, had an invalid grep, and still called the GitBook API - it could not have succeeded. Also corrected the sidebar location to src/sidebar.ts, dropped the GITBOOK_TOKEN dependency, and fixed dead `.warp/references/terminology.md` paths in four skills. Consolidates PRs #450, #454, #468, and #484, which all edited draft_docs/SKILL.md and conflicted with each other. Overlapping patterns were merged rather than stacked, and PR #468's frontmatter-description edits were dropped as already superseded on main. Co-Authored-By: Warp Agent <agent@warp.dev>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
I'm starting a first review of this pull request. You can view the conversation on Warp. I completed the review and no human review was requested for this pull request. Comment Powered by Oz |
There was a problem hiding this comment.
Overview
This PR consolidates recurring docs automation guidance, adds standing-PR conventions, reduces Slack notifications, and updates GitBook-era path and redirect instructions across docs skills.
Concerns
sync-error-docsnow documents error redirect handling that does not match the currentvercel.jsonshape: existing error redirects are maintained in both slash and no-slash source variants, and the new catch-all verification checks for the wrong source string.- The actionable-only Slack policy is applied to outer-loop skipped/no-action runs that do not write their own run log, which contradicts the new hard requirement that quiet runs must have a per-run log record.
Verdict
Found: 0 critical, 3 important, 0 suggestions
Request changes
Comment /oz-review on this pull request to retrigger a review (up to 3 times on the same pull request).
Powered by Oz
Two conflicts, both in areas this PR deliberately rewrites.
`skill-authoring-guidelines.md` — Slack notifications section. Main reworded
the old post-on-every-run rule; this PR inverts it. Kept the inversion, and
folded main's added case ("runs that exited early because a source signal was
unavailable") into the actionable list, where it belongs as a failure rather
than a no-op.
`aeo_crosslink_audit/SKILL.md` — step 8. Main renamed `SLACK_BOT_TOKEN` to
`BUZZ_SLACK_TOKEN`; this PR rewrote the same paragraph for actionable-only
posting. Kept both: the new posting rule with main's token name. Taking either
side wholesale would have lost one of the two changes.
Also preserved from main across the auto-merged files: the `oz-dev` to `oz`
rename. Cloud sandboxes only ship `oz`, so a skill calling `oz-dev` silently
loses its run link. Verified no `oz-dev` invocations survive; the three
remaining mentions are main's prose warning against it.
Review feedback (oz-for-oss):
- Error redirects must be added in both slash forms. Verified: all 17 error
codes in vercel.json carry both a trailing-slash and a no-trailing-slash
source (34 entries, none with only one). The skill documented one form, so a
new code would have gotten half the coverage of every existing one. Fixed in
sync-error-docs and its redirect-patterns reference, including the matching
catch-all pair.
- Silent no-action runs contradicted this PR's own rule that silence requires a
per-run log. Correct catch: the outer loops have no run log, and
improve-drafting-skills claimed its signal log covers guard-skipped runs,
which is false since the guard exits before the collector. Rather than adding
three more standing log PRs to monthly agents — which fights this PR's goal —
the rule now accepts either a run log entry or an explicit outcome line in run
output, with the run log required for any skill an outer loop reads. Corrected
the false claim in improve-drafting-skills.
- The third comment claimed the catch-all source is `/errors/:code/` so the
documented grep would report it missing. Not accurate: both `/errors/:code`
and `/errors/:code/` exist, so the grep matches. The underlying observation
about both variants was still worth acting on and is covered above.
Aligned the three outer-loop skills to `BUZZ_SLACK_TOKEN` for consistency with
main's new guidance that the token must match the destination channel's bot.
Both secrets exist in the Oz store, so the old name would authenticate but can
fail with `channel_not_found`.
Co-Authored-By: Warp Agent <agent@warp.dev>
…-skills The schedule was deployed as `0 15 1 * *` (the 1st of each month) rather than the `0 17 * * 1` + first-week-guard combination the skill documented. Both are correct and both fire exactly once a month, but the docs and the deployed schedule disagreed. Documented the deployed expression. Restricting only day-of-month is unambiguous because day-of-week stays `*`, so there is no ORing hazard. The tradeoff is noted: the 1st can land on a weekend, delaying review. Kept the first-week guard as a safety net and explained why, since it no longer trips on its own: it is what would narrow a day-of-week expression back to the first Monday, and it contains the blast radius if the day-of-month/day-of-week ORing mistake is ever reintroduced. Reworded the guard's skip message, which still referenced 'first Monday'. Co-Authored-By: Warp Agent <agent@warp.dev>
|
/oz-review |
|
I'm re-reviewing this pull request in response to a review request. You can view the conversation on Warp. I completed the review and no human review was requested for this pull request. Comment Powered by Oz |
There was a problem hiding this comment.
Overview
This PR consolidates recurring docs automation guidance, reduces no-op Slack noise, fixes stale migration-era paths, and standardizes standing PR behavior across several skills. The changes are broadly coherent, but one outer-loop fallback still allows analysis to proceed from an incomplete log source after the branch fetch fails, which undercuts the core log-branch guarantee.
Concerns
.agents/skills/improve-aeo-crosslink-skill/SKILL.mdshould not silently fall back to the current checkout when the log branch is unavailable; that can analyze stale data and then skip notification.
Verdict
Found: 0 critical, 1 important, 0 suggestions
Request changes
Comment /oz-review on this pull request to retrigger a review (up to 3 times on the same pull request).
Powered by Oz
|
|
||
| **Do not merge the standing log PR.** An earlier version of this skill attempted the merge as its first step. That coupled the analysis to a repo write the agent may not have permission to perform, and turned an unmerged PR into a hard failure rather than a non-event. Merging is human housekeeping; see "Log availability" in `.agents/references/skill-authoring-guidelines.md`. | ||
|
|
||
| If the branch does not exist or the fetch fails, fall back to reading `.agents/logs/aeo_crosslink_audit_runs.md` from the current checkout and note in the run output that the history may be incomplete. Do not abort the run. |
There was a problem hiding this comment.
Summary
Recurring docs agents were generating more PRs and Slack messages than the team could absorb. This fixes the three systemic causes, repairs a batch of GitBook-era migration artifacts found along the way, and consolidates four conflicting stacked PRs into this one.
The trigger was
improve-drafting-skillsopening a new PR and posting to#growth-docstwo days running. The root cause turned out not to be the skill's logic.Root cause: the cron was wrong
The schedule used
0 17 1-7 * 1, documented as "first Monday of each month." That is not what cron does. When an expression restricts both day-of-month and day-of-week, the two fields are ORed, so it fired on every day of the 1st–7th and every Monday — roughly 11 times a month. Confirmed by the schedule's own data:last_ranThursday 2026-08-06,next_runFriday 2026-08-07.Standard cron cannot express "first Monday," so the fix is
0 17 * * 1plus an in-skill first-week guard. The same broken expression was also present, undeployed, inimprove-aeo-crosslink-skillandimprove-404-monitor-skill; both are fixed here before they could reproduce it.Changes
Shared conventions —
.agents/references/skill-authoring-guidelines.mdTwo existing rules in this file were the direct cause of the problems, not drift. Skills were following them correctly.
main, which meant analyzing a truncated history whose staleness depended on review cadence. Now they read the log branch, and no agent merges the standing log PR as a workflow step.PR reuse
Applied to
improve-drafting-skills,weekly-404-monitor,afdocs-fix,sync-error-docs,sync_terminology,sync-openapi-spec,improve-aeo-crosslink-skill, andimprove-404-monitor-skill.aeo_crosslink_auditalready had the guard and now points at the shared reference.update-changelogis deliberately excluded from the standing-PR rule — one PR per release is correct there. It instead gained stacking detection, since consecutive release PRs all insert at the top of the same changelog file and would otherwise conflict or merge out of order.Slack volume
aeo_crosslink_audit- no longer posts on no-change runs.weekly-404-monitor- gates on the reporting threshold, and folds its Phase 2 redirect results into the single Phase 1 message rather than posting twice per run.afdocs-audit- posts only on a score regression or a firewall-blocked audit. This needed a baseline, so it gets a new run log (.agents/logs/afdocs_audit_runs.md) that compares against the most recent valid entry, so a blocked run never registers as a regression.GitBook migration artifacts
A find-and-replace during the migration substituted descriptions into file paths, leaving instructions pointing at files that do not exist.
sync-error-docswas the worst: it treatedastro.config.mjs (sidebar config)andvercel.json (redirects)as literal paths in eight places, contained agrepthat would be a shell syntax error, and still called the GitBook API viaGITBOOK_TOKEN. It could not have succeeded as written. Rewritten against verified reality — sidebar issrc/sidebar.tswith bare slug strings, redirects are JSON invercel.json, and the/errors/:codecatch-all already covers the prefix case so no per-code GitBook call is needed..warp/references/terminology.mdis an empty leftover directory; the real file is.agents/references/terminology.md. Four skills pointed at the dead path, which meantsync_terminologywouldgit adda nonexistent file. Fixed insync_terminology,answer_question,triage-issue-local, andcreate_pr.draft_docs,check_for_broken_links,create_pr, andreview-docs-pr.Consolidated stacked PRs
Supersedes #450, #454, #468, and #484. All four edited
draft_docs/SKILL.mdand conflicted with each other. Overlapping patterns were merged into single coherent rules rather than appended as four blocks — "Related pages required" appeared in both #454 and #468, and #450's screenshot guidance was folded into the existing screenshot rule instead of competing with it. #468's frontmatter-description edits were dropped entirely:mainhas since moved past them.Verification
style_lint.py --changed— 0 issues (no content files touched).check_links.py --internal-only— 363 files, 3449 internal links, 0 broken.SKILL.mdfiles.git diff --checkclean.check_pr_body.pyacross a simulated two-run body.Notes for reviewers
Two things worth a second opinion:
check_pr_body.pycompatibility. It asserts each required heading appears exactly once and rejects duplicate headings. A standing PR whose runs each append their own## Patterns addressedwould fail on the second run and silently block every future update. The prescribed body format therefore keeps headings fixed and appends dated bullets underneath. Worth confirming that reads well after several months of accumulation.update-changelogexemption. I judged one-PR-per-release correct there rather than forcing the standing-PR rule. If you would rather it also use a standing PR, that is a small change.Unverified claims
None — the sidebar location, redirect format and catch-all,
platformerrors.gopath, error page directory, and archived state ofwarpdotdev/gitbookwere each verified directly against the repos before being written down.Follow-ups (Oz-side, not in this PR)
Schedule changes cannot be made through a repo change. Full list is in the plan; the load-bearing ones:
LziMazxr8KQWQGL4lz14NCcron from0 17 1-7 * 1to0 17 * * 1.warpdotdev/gitbookrepo (~17 runs/week against a dead repo).check_for_broken_linksschedule onwarpdotdev/docsto replace the three deleted overlapping ones.Warp Docs / gitbookenvironmentsync-error-docsschedule — only after this merges and a dry run passes.Update
Merged latest
main(conflicts resolved) and addressed the review:vercel.jsoncarry both forms, so documenting one would have left new codes half-covered.improve-drafting-skillswrongly claimed its signal log covered guard-skipped runs. Silence now requires a durable record of the outcome — a run log entry (still required where an outer loop reads the history) or an explicit outcome line for a pre-work short-circuit./errors/:codeand/errors/:code/exist. Explained on the thread.BUZZ_SLACK_TOKEN, matching every other#growth-docsposter.Co-Authored-By: Warp Agent agent@warp.dev