Skip to content

fix: defense-in-depth against duplicate code agent PRs (#1312, #1320, #1321) - #26

Closed
guyoron1 wants to merge 97 commits into
mainfrom
3-no-code-on-skip
Closed

fix: defense-in-depth against duplicate code agent PRs (#1312, #1320, #1321)#26
guyoron1 wants to merge 97 commits into
mainfrom
3-no-code-on-skip

Conversation

@guyoron1

@guyoron1 guyoron1 commented Jun 17, 2026

Copy link
Copy Markdown
Owner

Upstream PR

Mirror of fullsend-ai/fullsend#2373 by @jhutar

Summary

Prevents duplicate code agent PRs through three defense layers, all addressing a 2026-05-21 incident where 5 duplicate PRs were created:

QF Demo Value

Why this is a great QF target:

  • Multi-layer defense — QF can test each layer independently AND their interaction
  • Race conditions possible between layers — needs concurrent scenario testing
  • Shell script logic (pre-code.sh) with multiple exit paths — QF can map all branches
  • Has pre-code-test.sh — QF can validate existing test coverage and add gaps

Mirrored for QualityFlow integration demo — trigger with /fs-qf

ifireball and others added 30 commits June 10, 2026 16:18
@ifireball @cursoragent
feat(install): add --vendor for self-contained workflow and agent assets
436a7f8 
Introduce --vendor to install vendored binaries, reusable workflows,
actions, and agent content. Vendored upstream mirror content is committed
under .defaults/ (same layout as runtime sparse checkout); layered installs
fetch fullsend-ai/fullsend@v0 into .defaults when the marker file is absent.

Reusable workflows use inline workspace preparation and reference infra
from ./.defaults/, matching the pre-vendor layered design. Thin callers
render local reusable paths when --vendor is set.

--fullsend-source pins the source tree for both content and binary
cross-compile; --fullsend-binary remains an explicit ELF override.

Signed-off-by: Barak Korren <bkorren@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@ifireball @cursoragent
feat(vendor): add manifest-driven cleanup and split analyze reporting
0a0561b 
Write vendor-manifest.yaml on --vendor installs so cleanup and analyze work
without a local fullsend checkout. Workflows analyze stays embed-only;
vendor layer reports presence, manifest alignment, and optional source
alignment via admin analyze --fullsend-source.

Signed-off-by: Barak Korren <bkorren@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@ifireball @cursoragent
fix: address remaining PR review nits for vendor work
f19f1e3 
Consolidate thin-stage caller registry, reuse resolved source root for
binary vendoring, reject oversized tar members during extraction, restore
workflows scope comment, fix testing-workflows prose, and introduce
InstallFiles as the canonical collector return type.

Signed-off-by: Barak Korren <bkorren@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@ifireball @cursoragent
fix(binary): restore download tests dropped in prior commit
32aaf9d 
Re-add the full download_test.go suite and append extractSourceTree size
limit coverage.

Signed-off-by: Barak Korren <bkorren@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@ifireball @cursoragent
fix(vendor): batch stale cleanup and address review nits
b5baa69 
Delete vendored paths atomically via forge.DeleteFiles, reuse resolved
source root for cross-compile, preserve extracted file modes, and tighten
WouldFix deduplication to exact path matches.

Signed-off-by: Barak Korren <bkorren@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@ifireball @cursoragent
docs(vendor): note --vendor-fullsend-binary removal without alias
8a9681e 
Document intentional breaking change: old flag callers should use --vendor;
only known usage was e2e, already updated in this branch.

Signed-off-by: Barak Korren <bkorren@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@ifireball @cursoragent
fix(vendor): restore layer docs and normalize cleanup step messages
0b50f96 
Document VendorBinaryLayer legacy naming, restore Uninstall/Analyze
comments, and use Title Case for stale-cleanup progress messages.

Signed-off-by: Barak Korren <bkorren@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@ifireball @cursoragent
fix(vendor): single-commit upload and address Bugbot findings
1f678e7 
Batch binary, content, and manifest in one CommitFiles call; validate
manifest version on read; trim leading slash in extractSourceTree; wrap
DeleteFiles ref PATCH in retryOnTransient.

Signed-off-by: Barak Korren <bkorren@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@ifireball @cursoragent
fix(forge): include mode and type in DeleteFiles tree entries
1881e3b 
Use the existing blob mode from the recursive tree and set type blob
so deletion entries match GitHub Trees API expectations.

Signed-off-by: Barak Korren <bkorren@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@ifireball @cursoragent
test(forge): assert DeleteFiles tree entry mode and type
88ecef4 
Guard against regressions in delete-entry construction per review.

Signed-off-by: Barak Korren <bkorren@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@ifireball
Merge upstream/main into feat/distribution-mode-layered-standalone
0013c75 
Signed-off-by: Barak Korren <bkorren@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>

# Conflicts:
#	internal/forge/fake.go
#	internal/forge/forge.go

Signed-off-by: Barak Korren <bkorren@redhat.com>
@ifireball @cursoragent
fix(vendor): address post-review findings from fullsend-ai-review
893d1af 
Encode CommitFiles tree entries as base64 to preserve ELF binaries,
add tar extract containment check, consolidate stale cleanup with a
manifest/binary quick-check, and deduplicate cleanup between CLI and layer.

Signed-off-by: Barak Korren <bkorren@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@ifireball
Merge upstream/main into feat/distribution-mode-layered-standalone
52dc9d2 
Signed-off-by: Barak Korren <bkorren@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>

# Conflicts:
#	action.yml
#	docs/guides/dev/testing-workflows.md

Signed-off-by: Barak Korren <bkorren@redhat.com>
@ifireball @cursoragent
docs: address review feedback on ADR 0046 and testing guide
b7b04f5 
Clarify removed distribution-mode artifacts, drop e2e vendor line, and
document action.yml source-build fallback.

Signed-off-by: Barak Korren <bkorren@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@ifireball @cursoragent
chore: re-trigger fullsend-ai-review after doc fixes
7d71e38 
Empty commit to re-dispatch review; prior synchronize dispatch was cancelled.

Signed-off-by: Barak Korren <bkorren@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@ifireball @cursoragent
fix(scaffold): include check-e2e-authorization in vendored infra paths
d330766 
Keep enumerateVendoredPaths aligned with CollectVendoredAssets after
main added the composite action (fullsend-ai#2106); fixes CI parity test.

Signed-off-by: Barak Korren <bkorren@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@ifireball @cursoragent
docs(forge): note base64 encoding in CommitFiles comment
99ddc9d 
Signed-off-by: Barak Korren <bkorren@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@ifireball @cursoragent
fix(install): combine vendor commit with scaffold and retry enrollmen…
fed552c 
…t dispatch

GitHub Actions may return 422 when repo-maintenance is dispatched immediately
after a separate vendor CommitFiles on a fresh .fullsend repo. Merge scaffold
and vendored assets into one atomic commit and retry dispatch on indexing lag.

Signed-off-by: Barak Korren <bkorren@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@ifireball @cursoragent
fix(install): wait for workflow registration and activate repo-mainte…
1d3da39 
…nance

Poll GitHub until repo-maintenance.yml is active before dispatch, re-touch
config.yaml after scaffold so the push trigger can run enrollment when
dispatch is still rejected, and fall back to awaiting a push-triggered run.

Signed-off-by: Barak Korren <bkorren@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@ifireball @cursoragent
fix(forge): write text files as UTF-8 in CommitFiles, blob API for bi…
73dea45 
…nary

Tree entries with encoding:base64 stored base64 text literally on GitHub,
corrupting YAML workflows and vendor-manifest.yaml. Restore UTF-8 inline
content for text and upload binary via the Git Blob API instead.

Signed-off-by: Barak Korren <bkorren@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@ralphbean
docs: add design spec for triage prerequisites action (#401)
63c27e4 
Design for a new `prerequisites` triage action that replaces `blocked`.
The agent can now express both existing blockers and new issues that need
to be created upstream before progress can happen. Includes allowlist
configuration for cross-repo issue creation and a degraded path when
targets are not authorized.

Assisted-by: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ralph Bean <rbean@redhat.com>
@ralphbean
docs: add implementation plan for triage prerequisites action (#401)
ba99ae3 
Seven-task plan covering config structs, JSON schema, agent prompt,
post-script, user docs, and caller updates. TDD approach with exact
file paths and code blocks.

Assisted-by: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ralph Bean <rbean@redhat.com>
@ralphbean
feat(config): add create_issues allowlist config (#401)
9a35c91 
Add CreateIssuesConfig and AllowTargets types to both OrgConfig and
PerRepoConfig. NewOrgConfig populates defaults with the org and
fullsend-ai/fullsend. NewPerRepoConfig populates with the target repo
and fullsend-ai/fullsend.

Assisted-by: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ralph Bean <rbean@redhat.com>
@ralphbean
refactor: update NewOrgConfig/NewPerRepoConfig callers for create_iss…
d4a394e 
…ues (#401)

Pass org name and target repo to config constructors so create_issues
defaults are populated at install time.

Assisted-by: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ralph Bean <rbean@redhat.com>
@ralphbean
feat(schema): replace blocked with prerequisites action (#401)
e492ac7 
Replace the blocked action and blocked_by field with a prerequisites
action containing existing[] and create[] arrays. At least one array
must be non-empty.

Assisted-by: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ralph Bean <rbean@redhat.com>
@ralphbean
feat(triage): replace blocked action with prerequisites in agent prom…
b2055cb 
…pt (#401)

The triage agent can now recommend creating upstream issues via the
prerequisites action's create array, in addition to referencing existing
blockers. Adds hard constraint against emitting sufficient when
prerequisites exist.

Assisted-by: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ralph Bean <rbean@redhat.com>
@ralphbean
docs: document prerequisites action and create_issues config (#401)
c48a832 
Update triage agent docs to explain the new prerequisites action and the
create_issues.allow_targets configuration surface.

Assisted-by: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ralph Bean <rbean@redhat.com>
@ralphbean
feat(triage): handle prerequisites action in post-script (#401)
3a44b0c 
Replace the blocked handler with prerequisites. The post-script reads
the create_issues allowlist from config.yaml, creates permitted upstream
issues via gh, and includes collapsed draft bodies for disallowed or
failed creates so humans can file them manually.

Assisted-by: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ralph Bean <rbean@redhat.com>
@ralphbean
fix(triage): correct label name in agent prompt and remove dead code (#…
6f79d87 
…401)

The agent prompt referenced a nonexistent `prerequisites` label when
checking for prior blockers — the post-script actually applies the
`blocked` label. Also removed unused SOURCE_ORG variable from
post-triage.sh.

Assisted-by: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ralph Bean <rbean@redhat.com>
@ralphbean
fix(triage): update post-triage tests for prerequisites action (#401)
080368c 
Replace the four blocked-action test cases with five prerequisites-action
test cases that exercise the new schema (existing[], create[], allowlist
validation). Set up GITHUB_WORKSPACE with a config.yaml fixture and add
a mock gh issue-create handler that returns a fake URL.

Assisted-by: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Ralph Bean <rbean@redhat.com>
ralphbean and others added 14 commits June 16, 2026 21:16
@ralphbean
Merge pull request fullsend-ai#2295 from jhutar/2-protected
f322448 
fix: align protected-path enforcement to review layer
@ifireball
Merge pull request fullsend-ai#2331 from ifireball/feat/role-only-app…
560ace4 
…-ids

feat(mint): share ROLE_APP_IDS per role across orgs
@ggallen
Merge pull request fullsend-ai#2364 from ggallen/worktree-adr-0045-ph…
7aef782 
…ase3-pr5

refactor(cli): migrate uninstall flows to harness-first agent discovery
@ggallen @claude
refactor(harness): migrate loadKnownSlugs to harness-first discovery
f902ef8 
ADR-0045 Phase 3, PR 4: loadKnownSlugs now discovers agent identity
from harness wrapper files in the config repo via DiscoverRemoteAgents
before falling back to the config.yaml agents: block. When the legacy
path is used, a deprecation warning is emitted.

Signed-off-by: Greg Allen <gallen@redhat.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Greg Allen <gallen@redhat.com>
@ggallen
feat(harness): wire Lint() diagnostics into fullsend run and lock
f4e19d5 
Call h.Lint() after harness loading in both `fullsend run` and
`fullsend lock` commands to surface non-fatal warnings. Currently
warns when the `role` field is missing from a harness file.

This is Phase 3 PR 3 of ADR-0045. Lint diagnostics are informational
only — commands still succeed regardless of warnings.

For `fullsend lock`, diagnostics are deduplicated across forge
variants and include the agent name for context.

Severity-aware emission: warnings use StepWarn, errors use StepFail
to ensure future SeverityError diagnostics are visually distinct.

Signed-off-by: Greg Allen <gallen@redhat.com>
Signed-off-by: Claude <noreply@anthropic.com>
Signed-off-by: Greg Allen <gallen@redhat.com>
@ggallen
Merge pull request fullsend-ai#2361 from ggallen/worktree-adr-0045-ph…
f71504f 
…ase3-pr4

refactor(harness): migrate loadKnownSlugs to harness-first discovery
@ggallen
Merge pull request fullsend-ai#2362 from ggallen/worktree-adr-0045-ph…
b04ecb3 
…ase3-pr3

feat(harness): wire Lint() diagnostics into fullsend run and lock
@ggallen
Merge pull request fullsend-ai#2365 from ggallen/worktree-cleanup-sta…
ed8c416 
…tus-token-deprecations

refactor: remove deprecated status-token fallback paths
@jhutar
fix(fullsend-ai#1312): gate code agent steps on pre-code skip output
a06f386 
pre-code.sh correctly detected existing PRs and posted a skip comment,
but exited 0 without signaling the workflow to stop — so all downstream
steps (GCP setup, bot identity, agent run) executed anyway, producing
duplicate PRs.

Write skip=true/false to GITHUB_OUTPUT on every exit path and gate all
post-validation steps on steps.validate.outputs.skip != 'true'.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Generated-by: Claude

rh-pre-commit.version: 2.4.0
rh-pre-commit.check-secrets: ENABLED

Signed-off-by: Jan Hutar <jhutar@redhat.com>
@jhutar
fix(fullsend-ai#1321): add existing-PR gate to triage agent definition
4bf685e 
The triage agent correctly identified existing PRs during its search but
still emitted action "sufficient", applying ready-to-code and triggering
duplicate code agent dispatches. Add a hard constraint in Step 2b: when
an open PR already addresses the issue, use action "prerequisites" with
the PR URL instead of "sufficient".

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Generated-by: Claude

rh-pre-commit.version: 2.4.0
rh-pre-commit.check-secrets: ENABLED

Signed-off-by: Jan Hutar <jhutar@redhat.com>
@jhutar
fix(fullsend-ai#1320): skip code dispatch when open PRs mention the i…
0a2a23b 
…ssue

The dispatch router had no check for existing PRs that reference an
issue without formal closing keywords. Add a pr-check step in both
dispatch files (reusable-dispatch.yml and scaffold dispatch.yml) that
searches for open PRs mentioning the issue number and skips code
dispatch when any are found.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Generated-by: Claude

rh-pre-commit.version: 2.4.0
rh-pre-commit.check-secrets: ENABLED

Signed-off-by: Jan Hutar <jhutar@redhat.com>
@jhutar
test(fullsend-ai#1312): cover no-GH_TOKEN path in GITHUB_OUTPUT skip …
545171a 
…tests

The no-token exit path writes skip=false to GITHUB_OUTPUT but the
existing test only asserted on stdout. Add a run_test_github_output
variant to verify the output file.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Generated-by: Claude

rh-pre-commit.version: 2.4.0
rh-pre-commit.check-secrets: ENABLED

Signed-off-by: Jan Hutar <jhutar@redhat.com>
@jhutar
refactor: rename skip output to skipped for consistency
481de7c 
Align with the existing convention used by role-check steps in the
dispatch workflows, which output skipped=true. Rename skip→skipped in
pre-code.sh, reusable-code.yml, reusable-dispatch.yml, scaffold
dispatch.yml, and corresponding tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Generated-by: Claude

rh-pre-commit.version: 2.4.0
rh-pre-commit.check-secrets: ENABLED

Signed-off-by: Jan Hutar <jhutar@redhat.com>
@jhutar
fix(fullsend-ai#1320): exclude bot-authored PRs from dispatch-level p…
2c94f53 
…r-check

The dispatch pr-check step did not filter out fullsend-ai[bot] and
fullsend-ai-coder[bot] PRs, which would block re-runs even when only
a bot PR existed — making the /fs-code --force escape hatch unreachable.
Add --jq filtering to match the logic in pre-code.sh.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Generated-by: Claude

rh-pre-commit.version: 2.4.0
rh-pre-commit.check-secrets: ENABLED
@guyoron1

guyoron1 commented Jun 17, 2026

Copy link
Copy Markdown
Owner Author

/fs-qf

QualityFlow and others added 7 commits June 17, 2026 15:02
QualityFlow
Add QualityFlow output for GH-26 [skip ci]
d9eff5a
QualityFlow
Add STP output for GH-26 [skip ci]
bced461
QualityFlow
Add QualityFlow STP review for GH-26 [skip ci]
7e2f3b4
QualityFlow
Add STD output for GH-26 [skip ci]
2775b08
QualityFlow
Add QualityFlow STD review output for GH-26 [skip ci]
790d88c
QualityFlow @claude
Add test output for GH-26 [skip ci]
438b129 
Generated 30 tests (26 Go, 4 Python) from STD YAML for
Defense-in-Depth Against Duplicate Code Agent PRs.

Go tests cover: pre-code skip detection (12), dispatch gate (4),
triage defense (3), workflow gating (3), mint-url migration (4+1).
Python E2E tests cover layered defense independence (4).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@guyoron1
Add QualityFlow tests for GH-26
93a1567 
Replaces intermediate pipeline artifacts with organized test files.

Total: 7 test files → qf-tests/GH-26/
Jira: GH-26
[skip ci]
@github-actions

github-actions Bot commented Jun 17, 2026

Copy link
Copy Markdown

QualityFlow Pipeline Summary

Stage Agent Status
1 STP Builder ✅
2 STP Reviewer ✅
3 STP Refiner ✅
4 STD Builder ✅
5 STD Reviewer ✅
6 STD Refiner ✅
7 Test Generator ✅

Test Output

Language Count Location
Go 5 files qf-tests/GH-26/go/
Python 2 files qf-tests/GH-26/python/

Issue: GH-26

Generated by QualityFlow

@guyoron1 guyoron1 closed this Jun 21, 2026
This was referenced Jul 7, 2026
Closed
Closed
Closed
@github-actions
github-actions Bot deleted the 3-no-code-on-skip branch July 19, 2026 05:44
This was referenced Aug 1, 2026
Closed
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@guyoron1 @ifireball @ralphbean @jhutar @ggallen