Context
The current review pipeline uses a two-tier severity system for GitHub Issues:
finding (bug/security) — filed by the commit watcher, acted on immediately by the builder
note (cleanup/robustness) — filed by the commit watcher as observations, only promoted to finding by the milestone reviewer if a pattern recurs in 2+ locations
The intent was to reduce noise for the builder by deferring low-severity issues until the milestone reviewer confirms they represent a real pattern. In practice, it's unclear whether note issues are providing value or just adding complexity.
Questions to investigate
- Are notes ever getting promoted? Do milestone reviews actually promote notes to findings, or do they just accumulate and get ignored?
- Should notes just be findings? The commit watcher could file everything as
finding and let the builder triage. Simpler pipeline, fewer moving parts.
- Should notes be eliminated entirely? If cleanup/robustness issues aren't worth the builder's time at all, maybe the commit watcher should only file bug/security issues and skip the rest.
- Is the frequency filter (2+ locations) the right threshold? Maybe one-off cleanup issues are still worth fixing but the current system filters them out.
- Impact on the issue builder: The dedicated issue builder only picks up
bug/finding issues. If notes were findings, the issue builder would handle them too. Is that desirable?
- Cost/token overhead: Each note issue costs API calls to create and later evaluate. Is the two-pass approach (file note → milestone reviewer evaluates) worth the token spend vs. just filing or not filing?
Affected code
src/agentic_dev/prompts/reviewer.py — commit watcher prompts that distinguish note vs finding
src/agentic_dev/prompts/reviewer.py — milestone reviewer prompt that does the frequency filter / promotion
src/agentic_dev/milestone_reviewer.py — milestone review orchestration
src/agentic_dev/watcher.py — commit watcher orchestration
src/agentic_dev/builder.py — issue builder only queries bug/finding
src/agentic_dev/utils.py — list_open_issues_for_milestones() filters by severity label
Context
The current review pipeline uses a two-tier severity system for GitHub Issues:
finding(bug/security) — filed by the commit watcher, acted on immediately by the buildernote(cleanup/robustness) — filed by the commit watcher as observations, only promoted tofindingby the milestone reviewer if a pattern recurs in 2+ locationsThe intent was to reduce noise for the builder by deferring low-severity issues until the milestone reviewer confirms they represent a real pattern. In practice, it's unclear whether
noteissues are providing value or just adding complexity.Questions to investigate
findingand let the builder triage. Simpler pipeline, fewer moving parts.bug/findingissues. If notes were findings, the issue builder would handle them too. Is that desirable?Affected code
src/agentic_dev/prompts/reviewer.py— commit watcher prompts that distinguish note vs findingsrc/agentic_dev/prompts/reviewer.py— milestone reviewer prompt that does the frequency filter / promotionsrc/agentic_dev/milestone_reviewer.py— milestone review orchestrationsrc/agentic_dev/watcher.py— commit watcher orchestrationsrc/agentic_dev/builder.py— issue builder only queries bug/findingsrc/agentic_dev/utils.py—list_open_issues_for_milestones()filters by severity label