fix(triage-panel): use search_issues with -label:status/triaged for sweep#1194
Merged
Conversation
…weep
The previous SCHEDULED_SWEEP gather step instructed the agent to call
list_issues paginated oldest-first and filter status/triaged in its
reasoning step. Empirical observation of run 25514877383 showed this
fails for two compounding reasons:
1. The MCP gateway DIFC integrity filter silently drops issues from
non-collaborator authors *after* the API response returns. Result:
the agent sees fewer items than perPage requested even though
pageInfo.hasNextPage is true.
2. The LLM ignores the pageInfo block (75 KB into the response) and
infers 'hasNextPage implicitly false' from the short item count,
then noops the entire sweep. Prose mandating pagination loops is
not enforceable in this failure mode.
Fix: replace list_issues with search_issues using the query
'repo:microsoft/apm is:open is:issue -label:status/triaged
sort:created-asc'. GitHub Search supports negative label filters, so
the response contains *only* untriaged candidates oldest-first. One
page is enough; no pagination loop, no LLM-driven label exclusion.
Lockfile is unchanged because triage-panel.lock.yml uses
{{#runtime-import .github/workflows/triage-panel.md}}, so .md edits
take effect at next workflow run.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Updates the triage-panel scheduled sweep to avoid LLM-driven pagination failures by switching candidate collection to a server-side search_issues query that returns only untriaged issues.
Changes:
- Replace
list_issues+ prose pagination instructions with a singlesearch_issuesquery using-label:status/triaged sort:created-asc. - Remove redundant client-side “exclude triaged label” and “sort oldest-first” steps from the sweep instructions.
- Add changelog entries describing the sweep behavior change and the underlying RCA.
Show a summary per file
| File | Description |
|---|---|
CHANGELOG.md |
Documents the sweep behavior changes and rationale under Unreleased/Fixed. |
.github/workflows/triage-panel.md |
Rewrites the scheduled sweep gather instructions to use search_issues with server-side filtering and no pagination. |
Copilot's findings
- Files reviewed: 2/2 changed files
- Comments generated: 4
Comment on lines
+19
to
+20
| - `triage-panel` scheduled sweep now paginates the candidate query oldest-first via the GitHub MCP `list_issues` tool instead of a single 200-issue page, so daily runs actually drain the untriaged backlog rather than processing one issue per cron tick. (#1193) | ||
| - `triage-panel` scheduled sweep switches the candidate query from `list_issues`+prose-driven pagination to `search_issues` with `-label:status/triaged sort:created-asc`, so untriaged candidates are filtered server-side; the previous approach silently noop'd because the MCP gateway DIFC filter dropped non-collaborator issues mid-page and the agent inferred a false `hasNextPage:false`. |
| direction: "ASC", | ||
| perPage: 30, | ||
| search_issues( | ||
| query: "repo:microsoft/apm is:open is:issue -label:status/triaged sort:created-asc", |
Comment on lines
+296
to
+297
| want to drain. **One page is enough.** If `total_count` is greater | ||
| than 30 the extras roll to tomorrow's sweep; do not paginate. |
Comment on lines
+279
to
282
| search_issues( | ||
| query: "repo:microsoft/apm is:open is:issue -label:status/triaged sort:created-asc", | ||
| perPage: 30, | ||
| ) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TL;DR
#1193(paginatelist_issuesoldest-first) did not fix the throughput problem. The first dispatch on the fixedmain(run 25514877383) noop'd the sweep — 0 issues triaged — while 11 untriaged issues sat in the queue. This PR replaces the prose-driven pagination loop with a server-sidesearch_issuesquery that returns only untriaged candidates.Why #1193 failed (RCA from MCP logs)
Two compounding effects sabotaged the pagination approach:
1. MCP gateway DIFC filter silently drops non-collaborator issues mid-page.
/tmp/fixed-agent/mcp-logs/rpc-messages.jsonlshows 17 of 30 returned issues were filtered with messages like:The agent saw 13 of 30 items even though the underlying API returned 30. This breaks any "did I get a full page?" heuristic.
2. The LLM hallucinated
hasNextPage: false.The actual MCP response (75 KB into the payload) clearly contained:
But the agent's final output said:
It ignored the explicit field and invented an inference from the short count. Prose pagination instructions cannot recover from this — the LLM is unreliable at multi-page loops driven by deeply-nested JSON fields.
Fix
Replace
list_issues-with-prose-pagination withsearch_issuesusing a server-side filter:GitHub Search supports
-label:negation, so the response contains only the candidates we want. One page = one tool call = no LLM reasoning about pagination.sort:created-ascreturns oldest-first; the first 30 results are the correct batch.What changed
.github/workflows/triage-panel.mdSCHEDULED_SWEEP gather step rewritten to usesearch_issuesand explain why (~33 lines deleted, ~28 added).Lockfile
triage-panel.lock.ymluses{{#runtime-import .github/workflows/triage-panel.md}}(line 280), so.mdedits take effect on the next workflow run with no recompile needed.Validation
uv run --extra dev ruff check src/ tests/→ All checks passeduv run --extra dev ruff format --check src/ tests/→ 705 files already formattedManual validation will be on the next scheduled cron tick (or a manual dispatch). Expected behavior:
search_issuesreturns 11 untriaged candidates oldest-first; the agent triages up to 10 of them and rolls 1 to tomorrow.Why this is robust
hasNextPageinference for the LLM to get wrong.Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>