Skip to content

Add guardrails to prevent reference-impl-sync agent from skipping deterministic scripts#148

Merged
edburns merged 1 commit intomainfrom
edburns/remind-agent-to-stick-to-prompt
May 4, 2026
Merged

Add guardrails to prevent reference-impl-sync agent from skipping deterministic scripts#148
edburns merged 1 commit intomainfrom
edburns/remind-agent-to-stick-to-prompt

Conversation

@edburns
Copy link
Copy Markdown
Collaborator

@edburns edburns commented May 4, 2026

.github/workflows/reference-impl-sync.md

  • Replace vague "Follow the prompt" instruction in the issue body template with explicit mandates: read the prompt file in full, do not clone the reference implementation manually, do not update .lastmerge manually, do not skip the finish script.

.github/workflows/build-test.yml

  • Add "Validate reference-impl-sync completeness" step to the java-sdk job. Only runs on PRs labeled reference-impl-sync.
  • Check 1: .lastmerge must appear in the PR diff (proves finish script ran).
  • Check 2: if scripts/codegen/ changed, src/generated/java/ must also have changed (proves codegen ran when the schema version was bumped).

…erministic scripts

.github/workflows/reference-impl-sync.md
  - Replace vague "Follow the prompt" instruction in the issue body template
    with explicit mandates: read the prompt file in full, do not clone the
    reference implementation manually, do not update .lastmerge manually,
    do not skip the finish script.

.github/workflows/build-test.yml
  - Add "Validate reference-impl-sync completeness" step to the java-sdk job.
    Only runs on PRs labeled reference-impl-sync.
  - Check 1: .lastmerge must appear in the PR diff (proves finish script ran).
  - Check 2: if scripts/codegen/ changed, src/generated/java/ must also have
    changed (proves codegen ran when the schema version was bumped).
Copilot AI review requested due to automatic review settings May 4, 2026 20:48
@edburns edburns merged commit 7fa76cf into main May 4, 2026
9 checks passed
@edburns edburns deleted the edburns/remind-agent-to-stick-to-prompt branch May 4, 2026 20:52
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds workflow-level guardrails to make “reference implementation sync” PRs more deterministic and harder for automation to partially complete (e.g., skipping finish scripts or codegen).

Changes:

  • Tighten the reference-impl-sync issue template instructions to explicitly forbid manual steps and skipping the finish script.
  • Add a CI validation step (only for PRs labeled reference-impl-sync) to enforce that .lastmerge and generated code changes are present when expected.
Show a summary per file
File Description
.github/workflows/reference-impl-sync.md Strengthens the issue body “Instructions” section to mandate using the prompt + scripts and prohibit manual cloning/.lastmerge edits.
.github/workflows/build-test.yml Adds a labeled-PR-only step to validate that the finish script and (when applicable) codegen ran by checking for expected diff contents.

Copilot's findings

  • Files reviewed: 2/2 changed files
  • Comments generated: 3

CHANGED=$(git diff --name-only origin/main...HEAD)

# 1. .lastmerge must be updated (proves finish script ran)
if echo "$CHANGED" | grep -q '^\\.lastmerge$'; then
github.event_name == 'pull_request' &&
contains(github.event.pull_request.labels.*.name, 'reference-impl-sync')
run: |
git fetch origin main --depth=1

❌❌Do NOT clone the reference implementation manually — the start script does this.❌❌
❌❌Do NOT update .lastmerge manually — the finish script does this.❌❌
❌❌Do NOT skip the finish script — it syncs codegen versions and updates .lastmerge.❌❌
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants