Skip to content

1385: Automate GitHub Project Status updates on PR merge across yalesites-project, atomic, and component-library-twig - #2

Closed
dblanken-yale wants to merge 3 commits into
mainfrom
1385-project-status-sync
Closed

1385: Automate GitHub Project Status updates on PR merge across yalesites-project, atomic, and component-library-twig#2
dblanken-yale wants to merge 3 commits into
mainfrom
1385-project-status-sync

Conversation

@dblanken-yale

@dblanken-yale dblanken-yale commented Aug 16, 2026

Copy link
Copy Markdown
Contributor

1385: Automate GitHub Project Status updates on PR merge across yalesites-project, atomic, and component-library-twig

Description of work

This repo holds the shared half of the automation: the reusable workflow, its logic, its tests, and its documentation. The three code repos each get a 12-line caller that points here.

  • Merge this PR first. The caller workflows reference yalesites-org/.github/.github/workflows/project-status-sync.yml@main, which 404s until this lands on main.
  • .github/workflows/project-status-sync.yml — a workflow_call reusable workflow that sets a YaleSites-Internal ticket's Status on the YaleSites Board (project #6) as its PR moves:
    • needs review label added → In review
    • merged to developReady for Release (in dev)
    • merged to main/masterDone
  • scripts/project-status-sync/status-rules.js — the decision rules, written as pure functions so they can actually be tested: which ticket a PR belongs to, which status an event produces, whether that status is a forward move, and which PRs a release promotion carried.
  • scripts/project-status-sync/sync.js — the GitHub API side (Projects v2 GraphQL, commit-range walk), run by actions/github-script.
  • docs/project-status-sync.md — the developer-facing documentation the ticket asks for: what triggers it, what it sets, how to add it to a new repo, and how to work on it.
  • .github/workflows/test-scripts.yml — runs node --test on PRs touching scripts/.

Notable decisions, all open to being reversed in review:

  • Hosted here rather than in YaleSites-Internal. @miketullo95's comment correctly identified that YaleSites-Internal/.github/workflows/02-pr-status-monitor.yml already implements most of this. I ported its logic but not its location: YaleSites-Internal is private with Actions access_level: none, and opening it up to three public repos would — per GitHub's own warning — let outside collaborators on those repos indirectly reach the private repo and read its workflow logs. This repo is public, so there is no access gating, and @main here is a stable ref (hosting in yalesites-project would mean @develop, which moves on every platform merge).
  • 02-pr-status-monitor.yml is left alone. It is provably inert — its const dryRun = '${{ github.event.inputs.dry_run }}' !== 'false' evaluates to true on every non-manual run — so it cannot double-fire against this. Retiring it is worth its own ticket; the new docs page says so.
  • Release promotions fan out. Without this, "merge to main → Done" would ship inert: in all three repos the promotion to production is a batch PR carrying no single ticket (developmain "RC Update" in atomic/CLT, v*/release/*master in yalesites-project). A merge to main from one of those walks the commit range and marks every carried ticket Done. A hotfix branched straight to master is not a promotion and takes the single-ticket path.
  • Statuses only ever move forward (Backlog < Ready For Work < To Do < Blocked < In progress < In review < Ready for Release (in dev) < Done), so a late needs review label cannot drag a shipped ticket backwards. This is what keeps it from fighting the manual pass code review / pass functional review labels. Blocked sits before In progress on purpose so In review may override it.
  • No ticket comments. 02 commented on the ticket for every merge; that would be notification spam on every needs review label. The board Status change is the deliverable.

This ships dormant until an admin creates the token. PROJECT_TOKEN (a PAT with repo + project + read:org — the same secret name 07-label-to-project-fields.yml already uses successfully) exists today only as a repo secret on YaleSites-Internal. It needs to be an org-level secret shared with yalesites-project, atomic, and component-library-twig. Until then the workflow logs a warning and exits 0 rather than putting a red X on every merge in three repos.

Security review — three things I want a second opinion on

A dedicated security pass was run over this diff. Two findings are fixed in the code, one is fixed in the docs, and three are deliberate judgment calls I would rather you overrule than have me decide alone:

  1. A PR author picks the ticket number. The number comes from the branch, title, or body, with no check that the ticket relates to the PR — so a collaborator with only triage rights could retitle a PR, add needs review, and advance an arbitrary board item. I left this as-is: the damage is capped by the forward-only rule (advance only, Status field only, never regress), everyone with access to these repos is Yale staff, and the alternative — matching on branch name only — would drop legitimate PRs whose branch is not ticket-numbered. Say the word and I will restrict it.
  2. yalesites-org/.github main should be branch-protected. The callers pin @main and the workflow runs sync.js from main with the token in scope, so direct push access here is code execution in three repos. I did not change repo settings on my own — this needs doing before the token is created.
  3. Actions are pinned by major tag (@v4, @v7), matching every other workflow in these repos. Given a PAT is in scope, pinning actions/github-script@v7 to a SHA would be the safer choice; I kept the tag for consistency rather than diverge unilaterally.

Fixed already: a core.warning was echoing private YaleSites-Internal issue titles into public run annotations (removed, along with the now-unused GraphQL field); persist-credentials: false added to the test workflow, which runs PR-authored test files; the unused pull-requests: read permission dropped. The docs now also recommend a GitHub App token or a machine-account fine-grained PAT (Projects: write + Issues: read on YaleSites-Internal) instead of the broad classic repo PAT the ticket suggested — a classic repo PAT is read and write on every private repo its owner can reach, which is much more than this needs in three public repos.

The review found no workflow injection: every attacker-controlled value reaches JavaScript only through context.payload, the one ${{ }} in a run: block is github.repository, and the caller repo is never checked out, so PR-authored code never executes in the token-bearing job. Fork PRs cannot obtain the token.

Verification. Tests were written first and confirmed red before the implementation existed. node --test → 43 tests, 43 pass. actionlint → exit 0 on every workflow file. The parser fixtures are copied verbatim from real merged PRs in all three repos, including the awkward ones (bump-atomic-1810 and RC Update, which have no ticket; hotfix/2230-hotfix-1, whose "2230" is a version, not a ticket; and a stacked PR whose base is another feature branch). The three things unit tests cannot cover were checked read-only against the live API: the Projects v2 query returns the expected field and option IDs for a real ticket; the not-a-real-ticket case returns an errors array that Octokit throws on, which is now caught explicitly rather than falling through a null check that would never have been reached; and the fan-out commit range on a real RC PR (atomic#503) yields exactly the PRs it should.

Functional testing steps:

Nothing here can be exercised until the org secret exists, so testing is in two parts.

Now — review the logic without running it:

  • git clone this branch and run node --test — 43 tests should pass.
  • Run actionlint at the repo root — exit 0.
  • Read docs/project-status-sync.md and confirm the described behavior is the behavior the team actually wants, particularly the release fan-out and the forward-only rule.

After the org secret PROJECT_TOKEN is created and shared with the three repos:

  • Set dry_run: true in one caller workflow, open a throwaway PR from a branch named 1385-dry-run-check, add the needs review label, and confirm the Actions log says it would set Status to "In review" — and that the board did not change.
  • Remove dry_run, repeat, and confirm ticket 1385 moves to In review on the board.
  • Merge a ticketed PR to develop and confirm its ticket moves to Ready for Release (in dev).
  • Add needs review to a PR whose ticket is already "Ready for Release (in dev)" and confirm the board does not move backwards.
  • Merge a PR with no ticket (a dependency bump) and confirm the run is green with a warning, not a failure.

References yalesites-org/YaleSites-Internal#1385

Other work completed in

This is one work-unit across four repos, all on branch 1385-project-status-sync:

Scoping: which repos can use this — answering @dblanken-yale

Fair challenge, and the first cut was too loose. This workflow lives in a public repo, so GitHub will let any repository call it — every other repo in the org, and repos outside the org too. The token was the only thing in the way, and that depends on the org secret's visibility being set right and staying right.

Fixed in 12016ef with two locks:

  1. An explicit allowed_repos allow list, defaulting to exactly yalesites-project, atomic, component-library-twig, tokens, and YaleSites-Internal. It is checked before anything is read or written, so a call from anywhere else logs a warning and stops. tokens and YaleSites-Internal are on the list but have no caller workflow yet — they can opt in later without touching the shared logic.
  2. Documented that the org secret must be scoped to "selected repositories", not "all repositories". That is the primary control; the allow list is the backstop for if it is ever loosened.

Adding a repo now takes two deliberate steps — edit the allow list and share the secret. Six new unit tests cover it, including a call from yalesites-org/ysph and from a look-alike someone-else/atomic.

Also fixed in 12016ef: only yalesites-project may mark a ticket Done

The first cut treated any merge to main/master as Done, including atomic's and component-library-twig's developmain "RC Update". Checking the board says that is wrong: every ticket carried by the 2026-08-14 component-library-twig RC — 1529, 1532, 1536, 1537 — still sits at Ready for Release (in dev), because a companion RC is an intermediate release step, not "shipped". Left alone this would have mass-marked tickets Done a week or more early, in bulk, through the release fan-out. Done now comes only from the production_repo (default yalesites-project).

Done is now decided by a board sweep, not a commit walk (256f778)

@dblanken-yale walked the RC scenario and found a real gap in the first cut. Measured against the actual v2.23.0 release (PR #1276, v2230master): of the 12 tickets that release shipped, only 9 would have been marked Done. Three — 1239, 1266, 1311 — would have been stranded at Ready for Release (in dev) permanently, because their code lived only in atomic or component-library-twig. Companion work reaches yalesites-project as a single bump-atomic-XXXX commit with no ticket reference, so a commit-range walk of yalesites-project can never see those tickets. All three had to be moved by hand on the real board — exactly the work this ticket exists to remove.

So the commit-range fan-out is gone, replaced by a sweep. On a yalesites-project merge to master:

  1. Take every ticket at Ready for Release (in dev) — only that column.
  2. Find its pull requests via GitHub cross-references (our References yalesites-org/YaleSites-Internal#1234 line registers one even though it is not a closing keyword).
  3. Keep the merged ones in a release repo.
  4. Mark Done once every one of those has reached its repo's production branch — master for yalesites-project, main for atomic, component-library-twig, and tokens.

Four deliberate properties, each backed by a real ticket:

  • Only the repos a ticket actually touched need work. 1239 shipped with only a CLT PR, 1266 with only an atomic PR. Requiring PRs everywhere would strand them.
  • But every repo it did touch must have shipped. If the CLT half made the RC and the yalesites-project half merged to develop afterwards, the ticket is half out and stays put.
  • Unmerged PRs are ignored. 1266 and 1311 both have abandoned yalesites-project PRs and shipped anyway.
  • Only release_branches repos count. 1349 is cross-referenced from yalesites-claude-plugins, which has no release process; left in it would hold the ticket open forever.

A ticket not at Ready for Release (in dev) is never touched, even if a PR for it just shipped — work often lands on a ticket deliberately held open for more work, and completing it would be wrong.

Verified read-only against the live API before writing any of it: cross-references find all 9 v2.23.0 tickets including 1265, whose branch (drupal-10-6-10-update) has no ticket number at all; and compare/{branch}...{sha} cleanly separates shipped from not — behind/identical give ahead_by: 0, while CLT#692 (merged to develop after the Aug-14 RC) gives diverged, ahead_by: 2. The predicate tests ahead_by === 0, and a vanished merge commit 404s and is treated as not-shipped rather than throwing.

Removing the commit walk also removes the one place a transient 502 could redden a release PR. Suite is now 62 tests.

One open question for review: a ticket whose work is only in tokens becomes Done at the next yalesites-project release, since that is what triggers the sweep. That seems right — nothing is shipped until the platform ships — but flagging it in case you disagree.

Keeps a YaleSites-Internal ticket's Status on the YaleSites Board in step
with its pull request, so the board stops depending on someone remembering
to move a card.

The `needs review` label sets the ticket to In review, a merge to develop
sets Ready for Release (in dev), and a merge to main or master sets Done.
A merge to main from develop, a v* branch, or release/* is a release
promotion, so it walks the commit range and marks every ticket it carried.

Ticket matching is by convention (branch name, PR title, or a qualified
YaleSites-Internal reference in the body), because our PR bodies use
"References" rather than a closing keyword and so GitHub never records the
link itself. A bare "Fixes #123" is deliberately not matched: that number
means an issue in the pull request's own repo, and those numbers overlap
YaleSites-Internal's almost exactly.

Statuses only ever move forward, so a late label cannot drag a shipped
ticket backwards, and nothing here fights the manual review labels.

Anything unexpected is logged as a warning and the run exits cleanly.
Board bookkeeping is never worth a red X on a merge, and without the
PROJECT_TOKEN secret the workflow is dormant rather than broken.

References yalesites-org/YaleSites-Internal#1385
Two scoping corrections.

This workflow lives in a public repository, so GitHub lets any repository
call it, in this organization or outside it. The token was the only thing
standing in the way, and that relies on the org secret's visibility being
set correctly and staying that way. Add an explicit allowed_repos list,
defaulting to the five YaleSites repos, checked before anything is read or
written. The documentation now also says to scope the secret to selected
repositories rather than all of them.

Only yalesites-project may mark a ticket Done. In atomic and
component-library-twig a merge to main is the RC promotion, an
intermediate release step rather than "shipped", and the board already
records it that way: every ticket carried by the 2026-08-14
component-library-twig RC still sits at Ready for Release (in dev).
Treating those merges as Done would have marked tickets shipped a week or
more early, and would have done it in bulk through the release fan-out.

References yalesites-org/YaleSites-Internal#1385
The commit-range fan-out could only ever see tickets whose code landed in
yalesites-project. Companion work arrives there as a single
bump-atomic-XXXX commit carrying no ticket reference, so tickets whose
code lived only in atomic or component-library-twig were never marked
Done. Measured against the real v2.23.0 release: of the 12 tickets it
shipped, 9 would have been found and 3 (1239, 1266, 1311) stranded at
Ready for Release (in dev) forever. All three had to be moved by hand.

Replace it with a board sweep. When yalesites-project merges to master,
take every ticket sitting at Ready for Release (in dev), find its pull
requests through GitHub's cross-references, and mark it Done once every
merged pull request it has in a release repo has reached that repo's
production branch.

Only that column is considered. Work often lands on a ticket that is
deliberately still open because more is coming, and a shipped pull request
must not complete it.

A ticket needs work only in the repos it actually touched, but every repo
it did touch must have shipped, so a ticket whose component-library-twig
half made the RC and whose yalesites-project half merged to develop
afterwards stays put. Unmerged pull requests are ignored; 1266 and 1311
both have abandoned pull requests and shipped anyway. Only repos named in
release_branches count, so a reference from a repo with no release process
cannot hold a ticket open.

This also removes the commit-range walk, which was the one place a
transient error could redden a release pull request.

References yalesites-org/YaleSites-Internal#1385
@dblanken-yale

Copy link
Copy Markdown
Contributor Author

Closing for now; I think we are going to try a different method, but will keep this branch here in case we need it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant