You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Set DEFAULT_ANTHROPIC_MODEL to claude-sonnet-5 in anthropic-config.mjs
Remove temperature: 0 from Anthropic API calls in semver-analysis.mjs and dependabot-anthropic-gate.mjs — Sonnet 5 returns HTTP 400 for non-default sampling parameters
2. Improve semver classification
Always consult Claude — remove the patch early-exit when build output is unchanged
Include source diff — send a unified git diff alongside the build output diff so CI-only and source-only changes are classified from actual code changes, not heuristics
Update the system prompt to instruct the judge to use source diff when build artifacts are identical
Add resolveSemverAnthropicModel() defaulting to claude-sonnet-5 for
semver-analysis.mjs. Sonnet 5 rejects non-default sampling parameters,
so drop temperature: 0 from the semver API request. Dependabot gate
stays on the shared claude-sonnet-4-6 default until migrated separately.
Co-authored-by: Jonathan Kingston <jonathanKingston@users.noreply.github.com>
Use claude-sonnet-5 as the shared default in anthropic-config.mjs and
drop the semver-specific resolver. Remove temperature: 0 from
dependabot-anthropic-gate.mjs — Sonnet 5 rejects non-default sampling
parameters.
Co-authored-by: Jonathan Kingston <jonathanKingston@users.noreply.github.com>
Cursor Agent can help with this pull request. Just @cursor in comments and I'll start working on changes in this branch. Learn more about Cursor Agents
The reason will be displayed to describe this comment to others. Learn more.
Stale comment
Injected PR Evaluation: Web Compatibility & Security
PR: Migrate CI LLM judges to Claude Sonnet 5 (jkt/auto/semver-sonnet-5-63fb)
This PR updates three CI-only scripts under .github/scripts/:
anthropic-config.mjs — default model claude-sonnet-4-6 → claude-sonnet-5
dependabot-anthropic-gate.mjs — removes temperature: 0 from two Anthropic API calls
semver-analysis.mjs — removes temperature: 0 from one Anthropic API call
No files under injected/, messaging/, or special-pages/ runtime bundles are modified.
Web Compatibility Assessment
No findings. This change does not touch page-world injection, API overrides, DOM manipulation, messaging transports, or platform entry points. Zero web compatibility impact.
Security Assessment
No injected-runtime findings.
Info — CI automation determinism (.github/scripts/dependabot-anthropic-gate.mjs, .github/scripts/semver-analysis.mjs): Removing temperature: 0 may increase variability in Dependabot auto-merge gate and semver-label LLM decisions. This is an operational/consistency consideration for CI automation, not a page-world security vulnerability. The existing trust boundaries (Cursor check-run triple validation, authenticated-app author filtering, tool-choice constrained outputs, fail-closed gate logic) are unchanged.
Info — model ID (anthropic-config.mjs): claude-sonnet-5 matches Anthropic's documented drop-in replacement for claude-sonnet-4-6.
Risk Level
Low Risk — CI-only LLM judge configuration; no injected runtime, messaging, or captured-globals changes.
Recommendations
Verify first CI run with the new model on dependabot-auto-merge.yml and semver-label.yml workflows to confirm the API accepts requests without temperature (Sonnet 5 uses adaptive thinking; parameter support may differ).
Monitor gate behavior for a few Dependabot PRs after merge — if auto-merge/semver decisions become noisier, consider whether Sonnet 5 exposes an equivalent determinism knob (e.g. effort level) rather than temperature.
No injected-code tests or rollout actions required.
The reason will be displayed to describe this comment to others. Learn more.
Stale comment
Injected PR Evaluation: Web Compatibility & Security
Reviewed synchronize diff c5a328e → 126f601 (3 files under .github/scripts/ only). No injected/, messaging/, or special-pages/ runtime code is touched.
Web Compatibility Assessment
File
Lines
Severity
Finding
—
—
info
No findings. Changes are confined to CI LLM judge configuration. No browser API overrides, wrapper utilities, DOM interaction, platform entry points, or page-world injection code is modified. Zero impact on third-party site compatibility.
Security Assessment
File
Lines
Severity
Finding
—
—
info
No injected-runtime findings. No changes to captured-globals.js, message bridge trust boundaries, messaging transports, nativeData handling, or origin validation.
.github/scripts/dependabot-anthropic-gate.mjs
882–994
info
Dependabot merge gate trust model is unchanged: Cursor check triple-validation (name + app slug + details host), TRUSTED_AUTOMATION_AUTHORS allowlist, forced tool_choice with schema-validated structured output, and fail-closed evidence requirements remain intact.
.github/scripts/anthropic-config.mjs
7
info
Model ID claude-sonnet-5 is Anthropic's documented drop-in replacement for claude-sonnet-4-6. Override via ANTHROPIC_MODEL env var still supported.
Removing temperature: 0 is required for Sonnet 5 — non-default sampling parameters return HTTP 400 per Anthropic's migration guide. This is an API compatibility fix, not a security relaxation.
Risk Level
Low Risk — CI-only LLM judge configuration with no injected runtime code changes and no modification to existing security trust boundaries.
Recommendations
(info) Merge after CI is green. No injected-runtime blockers.
(info) Monitor the first dependabot-auto-merge.yml and semver-label.yml runs on Sonnet 5 — adaptive thinking is on by default and max_tokens is a hard cap on thinking + response. Current budgets (400/512/800) should be sufficient for these short structured-judge calls, but watch for truncated tool-use responses.
(info) Expect slightly less deterministic LLM judge output now that temperature: 0 is removed (Sonnet 5 constraint). The dependabot gate mitigates this via forced tool_choice + schema validation; semver analysis still parses free-form JSON and may see minor classification variance across runs — acceptable for this use case.
Remove the patch early-exit when build output is unchanged. Generate a
git source diff in semver-label.yml and include it alongside the build
diff so Claude can classify CI-only and source-only changes. Add unit
tests for prompt formatting.
Co-authored-by: Jonathan Kingston <jonathanKingston@users.noreply.github.com>
The reason will be displayed to describe this comment to others. Learn more.
Stale comment
Web Compatibility Assessment
No findings. This PR touches only .github/scripts/ and .github/workflows/ — CI automation for LLM judges and semver labeling. There are zero changes to injected/, messaging/, special-pages/ runtime bundles, wrapper-utils.js, captured-globals.js, or any page-world API overrides. Web compatibility categories (API surface fidelity, prototype chain integrity, DOM interaction, timing, platform-specific breakage, third-party script compatibility) are not applicable.
File
Severity
Note
(none)
—
CI-only scope; no runtime impact
Security Assessment
No injected-runtime security findings. Trust boundaries in CI automation remain intact.
File
Lines
Severity
Note
.github/scripts/anthropic-config.mjs
7
info
Model ID bump claude-sonnet-4-6 → claude-sonnet-5; CI-only, no page-world exposure
New SOURCE_DIFF input sent to Anthropic API — same trust boundary as existing BUILD_DIFF; fork PRs remain excluded by workflow guard (!head.repo.fork)
.github/workflows/semver-label.yml
13
info
Existing fork-PR exclusion prevents untrusted PRs from triggering LLM analysis with repo secrets
No changes to: captured globals, message bridge, nativeData handling, origin validation, postMessage, stack-trace exemptions, or config-gated feature code.
Risk Level
Low Risk — CI-only automation changes with no injected runtime, messaging, or page-world code modifications.
Recommendations
Monitor first CI runs on semver-label.yml and dependabot-auto-merge.yml after merge — Sonnet 5 uses adaptive thinking by default; verify max_tokens (400/512/800) remains sufficient for tool-use / JSON responses.
Positive change: Removing the early-exit when BUILD_DIFF is empty (now always consulting the LLM with SOURCE_DIFF) improves semver accuracy for source-only API/schema changes without affecting runtime security.
No injected-runtime tests required; existing unit tests for semver-analysis prompt formatting are appropriate coverage.
Separate shallow checkouts do not share the base SHA object, so
`git -C pr diff $BASE_SHA` fails with 'bad object'. Use gh pr diff
instead, which fetches the diff via the GitHub API.
Co-authored-by: Jonathan Kingston <jonathanKingston@users.noreply.github.com>
The reason will be displayed to describe this comment to others. Learn more.
Stale comment
Injected PR Evaluation: Web Compatibility & Security
Scope: This PR touches only .github/scripts/ and .github/workflows/ — CI LLM judge configuration and semver labeling. No changes under injected/, messaging/, or special-pages/ runtime code.
Web Compatibility Assessment
No findings. This PR does not modify page-world injection, API shims, messaging transports, captured globals, or any code shipped to DuckDuckGo browsers.
Security Assessment
File
Severity
Finding
(none)
—
No injected-runtime security surface affected.
.github/scripts/semver-analysis.mjs
info
SOURCE_DIFF, PR_TITLE, PR_BODY, and PR_FILES are sent to the Anthropic API. This extends an existing CI trust boundary (build diff was already sent). Fork PRs remain excluded by the semver-label.yml guard (!github.event.pull_request.head.repo.fork).
Removing temperature: 0 is required for Sonnet 5 (non-default sampling params return HTTP 400). May increase LLM output variance for semver labels and Dependabot gate decisions — operational, not a page-world vulnerability.
.github/workflows/semver-label.yml
info
gh pr diff adds full source diff to the LLM prompt. Pre-existing prompt-injection risk via PR title/body remains; no new exfiltration vector to end users.
Monitor first CI runs after merge for Sonnet 5 adaptive-thinking behavior under existing max_tokens caps (512 for semver, 400/800 for Dependabot gate) — truncated or malformed JSON would fail the label step rather than mislabel silently.
No injected-code action required. The SOURCE_DIFF change is a positive semver-classification improvement for source-only API changes that produce identical build artifacts.
Unit tests for formatBuildDiffSection / formatSourceDiffSection / buildUserPrompt pass locally; semver-analysis.test.mjs is wired into tests.yml.
Fetch the base commit into the pr checkout via a local remote so
git diff matches the merge ref used for builds. Remove silent error
swallowing on diff generation and fail when changed files exist but
the diff is empty.
Co-authored-by: Jonathan Kingston <jonathanKingston@users.noreply.github.com>
The reason will be displayed to describe this comment to others. Learn more.
Stale comment
Web Compatibility Assessment
No findings. This PR touches only .github/scripts/ and .github/workflows/ — CI automation for LLM semver classification and Dependabot gating. No changes to injected/, messaging/, special-pages/ runtime bundles, wrapper utilities, captured globals, or message bridge code. Zero page-world API surface impact.
Latest commit (6d8b7d) aligns source diff with the merge-ref build checkout via local base-checkout fetch. Removes prior gh pr diff silent error swallowing; fails fast when changed files exist but diff is empty. Positive CI hardening — no new secret exposure.
.github/scripts/semver-analysis.mjs
184–191
info
SOURCE_DIFF sent to Anthropic alongside BUILD_DIFF under the same existing API-key trust boundary. Truncated at 80k chars each (pre-existing pattern).
.github/scripts/dependabot-anthropic-gate.mjs
882–988
info
temperature: 0 removed — required for Claude Sonnet 5 API (non-default sampling params return HTTP 400).
.github/scripts/semver-analysis.mjs
145
info
max_tokens: 512 with Sonnet 5 adaptive thinking may truncate reasoning on very large diffs; affects label accuracy only, not runtime security.
No changes to messaging transports, nativeData handling, origin validation, captured globals, or injected load()/init() lifecycle.
Risk Level
Low Risk — CI-only LLM model migration and semver workflow improvements with no injected runtime code changes.
Recommendations
Merge after CI green — unit tests for semver-analysis helpers pass locally (4/4).
Monitor first Sonnet 5 CI runs — confirm semver labels and Dependabot gate decisions remain stable without temperature: 0; watch for truncated JSON under max_tokens: 512 on large PRs.
No injected web-compat or security follow-ups required.
Shallow checkouts reject updating named refs (refs/base-sha). Fetch
the base commit with --depth=1 into FETCH_HEAD instead, then diff
FETCH_HEAD..HEAD to match the merge build checkout.
Co-authored-by: Jonathan Kingston <jonathanKingston@users.noreply.github.com>
The reason will be displayed to describe this comment to others. Learn more.
Injected PR Evaluation: Web Compatibility & Security
Reviewed PR #2829 (Migrate CI LLM judges to Claude Sonnet 5). All changes are confined to .github/scripts/ and .github/workflows/ — no injected/, messaging/, or special-pages/ runtime code is touched.
Web Compatibility Assessment
No findings. This PR does not modify browser-injected code, API shims, messaging transports, or feature initialization.
Security Assessment
File
Lines
Severity
Finding
.github/workflows/semver-label.yml
54–74
info
New source-diff step fetches base SHA from the local base checkout ($GITHUB_WORKSPACE/base), not from PR-controlled input. Fork PRs are already excluded (!github.event.pull_request.head.repo.fork). No injected-runtime trust-boundary impact.
.github/scripts/semver-analysis.mjs
184–190
info
SOURCE_DIFF and PR_BODY are forwarded to the Anthropic API, same pre-existing pattern as BUILD_DIFF. Acceptable for a CI-only semver judge; no page-world exposure.
.github/scripts/dependabot-anthropic-gate.mjs
882–989
info
Removing temperature: 0 may increase LLM output variance for merge-gate decisions. Not a security vulnerability, but worth monitoring gate consistency after the Sonnet 5 migration.
No critical, error, or warning findings.
Risk Level
Low Risk — CI/tooling-only changes (model default bump, semver prompt enrichment, test coverage). Zero impact on code shipped to end-user browsers.
Recommendations
(info) After merge, spot-check a few PRs where build output is unchanged but source changes are semver-sensitive (e.g. new message schemas) to confirm the new source-diff path classifies correctly under Sonnet 5.
(info) Consider asserting DEFAULT_ANTHROPIC_MODEL === 'claude-sonnet-5' in anthropic-config.test.mjs so future model bumps are caught by CI.
(info) The removal of the empty-build-diff early-exit (severity: patch shortcut) is a correctness improvement — CI-only PRs like this one will now be evaluated by the LLM instead of auto-labeled patch.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
semver-patchBug fix / internal — no release needed
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
Asana Task/Github Issue: N/A
Description
CI-only changes across three areas:
1. Migrate CI LLM judges to Claude Sonnet 5
DEFAULT_ANTHROPIC_MODELtoclaude-sonnet-5inanthropic-config.mjstemperature: 0from Anthropic API calls insemver-analysis.mjsanddependabot-anthropic-gate.mjs— Sonnet 5 returns HTTP 400 for non-default sampling parameters2. Improve semver classification
3. Fix semver-label workflow source diff generation
The workflow builds from the PR merge ref; source diff now matches that:
FETCH_HEAD..HEADin theprcheckout against thebasecheckout SHAprclone via a localbase-checkoutremote (named refs are rejected on shallow clones)Files changed
anthropic-config.mjsclaude-sonnet-5dependabot-anthropic-gate.mjstemperature: 0semver-analysis.mjstemperature, export prompt helperssemver-analysis.test.mjssemver-label.ymltests.ymlTesting Steps
Verify Semver Label workflow passes on this PR after merge.
Checklist
Please tick all that apply: