feat: extract code, fix, review, retro, prioritize agents from fullsend#12
Merged
Conversation
Merge content from #1111 into this PR: - Escape literal HTML comment patterns in pr-review SKILL.md that trigger fullsend scan context hidden_html_comment findings - Add JSON schema tables (fields, types, constraints, required-per-action) to the agent definition alongside the existing jq examples - Mark schemas/review-result.schema.json as source of truth - Replace inline HTML comment reference in SKILL.md constraints with back-reference to step 6 Supersedes #1111. Signed-off-by: Wayne Sun <gsun@redhat.com>
…elds Remove the paragraph in review.md that duplicated the finding object table. Clarify in SKILL.md that action/pr_number/repo are always required and the per-action table lists additional required fields. Signed-off-by: Wayne Sun <gsun@redhat.com>
…-agent fix(review-agent): move output schema from pr-review skill to agent definition
Add step 8 to the planning phase of the code-implementation skill instructing the agent to search for old literal values when changing constants or defaults. When the task changes a constant from X to Y, the agent now searches for both the symbol name and the old value X as a string literal in test files, docs, and config. This addresses missed hardcoded assertions like those found in PR #1180. Note: make lint could not run (Go toolchain permission denied in sandbox). Change is documentation-only (Markdown). Closes #1221 Signed-off-by: fullsend-code <fullsend-code@users.noreply.github.com>
Strip all helper functions, constants, and CLI flag that existed only to support follow-up issue creation, which is now disabled. Update agent docs and skill to note the feature is temporarily disabled pending #1137. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ow-up-issues fix(review): disable follow-up issue creation while PR is open
Address review findings from review-squad: - Add plugins/ to PROTECTED_PATHS in post-fix.sh and REVIEW_PROTECTED_PATHS in post-review.sh to prevent autonomous agents from modifying/approving plugin config changes - Add customized/plugins/.gitkeep to e2e admin expected files - Update ADR 0035 to include plugins/ in all 4 directory enumerations and correct .gitkeep counts (7 → 8) - Reorder LAYERED_DIRS in 5 reusable workflows to match scaffold.go canonical order Signed-off-by: Wayne Sun <gsun@redhat.com>
…agram Address review-agent LOW findings: - Add plugins/ to protected paths list in pr-review SKILL.md (kept in sync with post-review.sh) - Add plugins/ to layered directories diagram in customizing-agents.md user guide Signed-off-by: Wayne Sun <gsun@redhat.com>
fix(scaffold): add plugins/ to layered directories
Add fullsend-check-output script that agents can call inside the sandbox to validate their output JSON against the declared schema before finishing. This catches wrong filenames, extra properties, and missing fields without burning a full harness retry iteration. The harness bootstrap now copies the check script into /tmp/workspace/bin/ and uploads the output schema into the sandbox with the env vars FULLSEND_OUTPUT_SCHEMA and FULLSEND_OUTPUT_FILE so the agent can reference them at runtime. Updated all 5 agents that produce JSON output: - pr-review, fix-review, retro-analysis skills - triage, prioritize agent definitions The harness validation_loop remains as a safety net. Closes: #1107 Signed-off-by: Wayne Sun <gsun@redhat.com>
- Log warnings instead of silently skipping when scaffold embed read or schema upload fails, preventing opaque "command not found" errors - Use defer for temp file cleanup in bootstrapSandbox - Add jsonschema availability pre-flight check matching validate-output-schema.sh - Make filename mismatch a hard failure (exit 1) instead of a warning - Shell-quote FULLSEND_OUTPUT_FILE with single quotes to prevent injection - Add 3-attempt retry limit to all agent/skill self-check instructions - Add fullsend-check-output to TestFullsendRepoFilesExist expected list Signed-off-by: Wayne Sun <gsun@redhat.com>
…gents The triage and prioritize agent definitions already included the fullsend-check-output validation step, but review, retro, and fix agent definitions were missing it. Since agent definitions are authoritative over skills, agents following only the agent file would skip output validation entirely. Add consistent validation blocks with 3-attempt retry wording to all three agent definitions, matching the pattern already used by triage and prioritize agents. Signed-off-by: Wayne Sun <gsun@redhat.com>
feat: agent self-check tool for output schema validation
Switch network policies from protocol: tcp (L4) to protocol: rest (L7) for HTTP-aware inspection in openshell 0.0.38. Fix access field from invalid "allow" to "read-write" per OpenShell policy schema. Add api.anthropic.com endpoint for Claude CLI direct API access. Also increase sandbox create/ready timeouts for Docker image pulls and capture gateway logs + docker container logs on sandbox readiness failure. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The github_artifacts section was added to retro.yaml on main after the PR branched. Rebase preserved it but left it using the old protocol: tcp / access: allow values which are rejected by OpenShell v0.0.38. Update to protocol: rest / access: read-write to match all other policy endpoints. Signed-off-by: Wayne Sun <gsun@redhat.com>
- sandbox.go: CheckGateway captures CombinedOutput and validates non-empty output for better error messages - sandbox.go: Extract collectPodmanLogs helper with name filtering, --tail cap, LookPath guard, timeout, and 1MB size limit - harness.go: ValidateRunnerEnvWith uses LookupEnv semantics to distinguish unset vars from empty-string vars (fixes fix agent HUMAN_INSTRUCTION validation failure) - run.go: Add gateway check timing and wire new lookup function - action.yml: Pin setup-uv to SHA, add arch detection and retry loop for gateway download, mask SSH secret, bind to 127.0.0.1 - policies: Split triage vertex_ai into vertex_ai + github_api, downgrade review/retro github_api access to read-only - docs: Fix fullsend version -> --version, document GITHUB_WORKSPACE and fix agent env vars for local runs, add policy protocol troubleshooting entry Signed-off-by: Wayne Sun <gsun@redhat.com>
- Remove unused RUNNER_ARCH env var from gateway download step - Remove curl from triage github_api binaries for policy consistency - Tighten gitleaks_releases and package_registries to read-only access - Add test for ValidateRunnerEnvWith empty-string-allowed semantics - Report container log collection errors instead of silently skipping - Remove unused OPENSHELL_PODMAN_SOCKET export from GITHUB_ENV - Deduplicate OPENSHELL_VERSION into a single GITHUB_ENV step - Use openssl instead of python3 for handshake secret generation - Add comment header and formatting to prioritize.yaml policy - Surface gateway logs on health check timeout failure Signed-off-by: Wayne Sun <gsun@redhat.com>
…0.38 feat(#780): migrate OpenShell to Podman driver
…earch-guidance feat(#1221): add constant-value search guidance to code skill
…-manifest feat(code-review): flag permission manifest changes as mandatory finding
The fix agent's TRIGGER_SOURCE env var contains a GitHub username (e.g. "orgname-review[bot]" or "alice"), but the fix-result.json schema requires trigger_source to be the enum "bot" or "human". The skill and agent definition never explicitly instructed the agent to map the username to the enum value, so the LLM would copy the raw username into the JSON — causing schema validation to reject the output (5 of 8 recent failures). Changes: - agents/fix.md: add explicit mapping instruction in the trigger modes section — if username ends in [bot] use "bot", else "human" - skills/fix-review/SKILL.md step 1: instruct the agent to derive the normalized trigger type early - skills/fix-review/SKILL.md step 9: add schema compliance section covering additionalProperties: false, the trigger_source enum constraint, and per-action-type required fields (description for fix, reason for disagree) Note: make lint could not run in sandbox (Go toolchain permission denied). Pre-existing infrastructure issue unrelated to this change. Closes #1300 Signed-off-by: fullsend-code <fullsend-code@users.noreply.github.com>
When a human closes a code agent PR and re-triggers /fs-code, the old remote branch persists. The post-code script's plain git push fails with non-fast-forward because the new local branch diverges from the stale remote. This caused silent failures requiring manual intervention (e.g., PR #1234 needed 3 attempts). Changes to post-code.sh: - Section 7a: before pushing, check if the remote branch exists via git ls-remote. If it does and no open PR references it, delete the stale remote branch so the fresh push succeeds. - Section 7b: capture push exit code and output. On non-fast-forward errors, retry with --force-with-lease as a fallback safety net. - Error reporting: add an ERR trap that posts a comment on the originating issue when the post-code script fails, including the exit code and a link to the workflow run. This ensures humans get feedback without checking workflow logs. - Move GH_TOKEN export earlier (before section 7a) since both the stale branch check and error reporting need it. Changes to post-code-test.sh: - Add stale branch decision tests (no remote, stale with no PR, branch with open PR). - Add push retry decision tests (success, non-fast-forward, rejected, unexpected error). - Add error comment content tests (exit code, workflow link, retry hint, warning emoji). Note: make lint could not run due to a Go toolchain permission error in the sandbox (unrelated to these shell script changes). The post-code-test.sh suite passes all 38 tests. Closes #1301 Signed-off-by: fullsend-code <fullsend-code@users.noreply.github.com>
- Rename "Bring Your Own Agent (BYOA)" to "Custom Agents" - Scrub "OOTB" references to "default" in lint script, scaffold comment - Add trigger line to code agent intro - Add ready-for-review to code agent control labels - Simplify fix agent sandbox description Assisted-by: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Ralph Bean <rbean@redhat.com>
The prioritize agent is not triggered after triage — it runs on a schedule (polling for unscored/stale issues) or on-demand via /fs-prioritize. Assisted-by: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Ralph Bean <rbean@redhat.com>
- prioritize: remove read-only paragraph, rename to Skill section, apply reworded intro, drop salesforce/gdrive script examples - retro: apply sandbox description suggestion, trim control labels - review: add schema validation step, remove token mention, add ready-for-review label - triage: add clarification to intro, simplify /fs-triage, apply label management text, rename to Skill section - agents-md: fix pronoun (your → its) - skills: add prioritize agent to built-in skills table Assisted-by: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Ralph Bean <rbean@redhat.com>
Add step 8 planning guidance so the code agent checks external API requirements for each code path before removing or emptying parameters. Regression test locks the guidance into the scaffold. Closes #669 Co-authored-by: Cursor <cursoragent@cursor.com> Signed-off-by: Barak Korren <bkorren@redhat.com>
The review agent was using `gh pr view --json` which hits the GraphQL API, but the minted review token only has REST API permissions. This caused two 403 failures per run before falling back to `gh api` REST calls. Additionally, the agent had no guidance on where the target repo is checked out, leading to wrong-path file access attempts. Changes: - agents/review.md: add Workspace section documenting the target repo path (/tmp/workspace/target-repo/), add GitHub API section with REST-only guidance and examples, document REPO_FULL_NAME in Inputs section - skills/pr-review/SKILL.md: replace all `gh pr view --json` calls with equivalent `gh api` REST endpoints for PR metadata, file lists, and head SHA lookups Note: `make lint` could not run (Go toolchain download permission denied in sandbox). Manual lint verification is required. Closes #1387 Signed-off-by: fullsend-code <fullsend-code@users.noreply.github.com>
Signed-off-by: Hector Martinez <hemartin@redhat.com>
…ql-403 fix(#1387): use REST API and explicit workspace path in review agent
|
🤖 Finished Review · ✅ Success · Started 2:45 PM UTC · Completed 2:52 PM UTC |
![fullsend-ai-review[bot]](https://avatars.githubusercontent.com/in/3478501?s=60&v=4){kind=small}
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
fullsend-ai-review
Bot
added
requires-manual-review
Signed-off-by: Claude <noreply@anthropic.com> Signed-off-by: Greg Allen <gallen@redhat.com>
|
🤖 Finished Review · ✅ Success · Started 2:58 PM UTC · Completed 3:02 PM UTC |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
fullsend-ai-review
Bot
added
requires-manual-review
Member
Author
|
/fs-review |
|
🤖 Finished Review · ✅ Success · Started 3:08 PM UTC · Completed 3:13 PM UTC |
fullsend-ai-review
Bot
added
requires-manual-review
|
🤖 Finished Retro · ✅ Success · Started 3:17 PM UTC · Completed 3:25 PM UTC |
2 tasks
Retro: PR #12 — Extract remaining agents from fullsendTimeline: PR created 13:38 → first review failed 13:39 → human triggered What went well:
What could go better:
Proposals: 2 repo-specific issues for bugs that the review agent correctly identified but were merged without being addressed. Proposals filed
|
This was referenced Jul 2, 2026
This was referenced Jul 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
9 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Extracts the 5 remaining first-party agents (code, fix, review, retro, prioritize) from
fullsend-ai/fullsendscaffold (internal/scaffold/fullsend-repo/) to this repo, per ADR 0058 Phase 4.git filter-repo—git log --followandgit blamework for all extracted filesenv/gcp-vertex.env→common/env/gcp-vertex.envvalidation_loop.schemareplacesenv.runner.FULLSEND_OUTPUT_SCHEMA(avoids${FULLSEND_DIR}resolution issues for externally-loaded harnesses)host_filescommon/env/gcp-vertex.env,scripts/validate-output-schema.sh,skills/issue-labels/) already existed from triage extraction — not re-importedFiles added per agent
Test plan
git log --follow -- agents/<agent>.mdshows scaffold-era commits for all 5 agentsbash scripts/validate-output-schema-test.sh— all 36 tests pass (covers triage, fix, and review schemas)FULLSEND_DIRor bareenv/gcp-vertex.envvalidation_loophaveschema:fielddoc:paths point to existing files/fs-<agent>after config registration (Step 8 of extraction plan)Next steps
After this PR merges:
fullsend-ai/.fullsend/config.yamlusingfullsend agent addfullsend-ai/fullsend🤖 Generated with Claude Code