ci(spec-sync): summarize the drift PR description with an LLM - #124
Merged
Conversation
After the mechanical + AI-wiring commits land, a best-effort step feeds the PR diff (surface-first, capped) to the Anthropic API and appends a "## What changed" section under the static process/safety preamble, so the body reflects what the drift actually changed instead of only the boilerplate. continue-on-error, so it never reds the run; the step can only edit the PR body via SPEC_SYNC_TOKEN, so an injected spec description can't escalate past text. Added to both the V1 and V2 jobs. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Adds best-effort Anthropic-generated summaries to V1 and V2 spec-sync PR descriptions.
Changes:
- Collects and truncates relevant PR diffs.
- Generates concise public-surface summaries.
- Appends summaries while preserving the static safety preamble.
…ot notes - Move the inline summarize logic (duplicated across the V1 and V2 jobs) into scripts/spec-sync/summarize-pr.sh; both jobs now call it. The only per-job difference — the V2 /v2/workflow* exclusion — is passed via SUMMARY_SCOPE_NOTE. - Bound the Anthropic call with --connect-timeout/--max-time: continue-on-error does not rescue a hung socket (it would burn the 60-min job timeout). - V2: exclude /v2/workflow* from the summary so a workflow-only drift can't advertise SDK endpoints the PR never wired. - Guard the body read and append-only edit so a transient failure never clobbers the static preamble. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The summary is built from untrusted spec descriptions and appended to the PR body, which also carries the reserved `<!-- spec-sync-slack-thread: <ts> -->` marker that thread-ts.sh trusts to route Slack lifecycle notifications. A prompt-injected marker in the model output could hijack that routing (or become the only marker if the root-save step failed). Strip HTML-comment delimiters from the summary before appending. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
After the mechanical + AI-wiring commits land, a best-effort step feeds the PR diff to the Anthropic API and appends a
## What changedsection under the existing static preamble (which stays authoritative: gates + human-review).continue-on-error, so a summary failure never reds the run; only edits the PR body viaSPEC_SYNC_TOKEN. Added to both V1 and V2 jobs. Native GitHub Copilot PR summaries are manual-only, hence doing it in the workflow.