Bug Hunter: require agent-authored reproduction, prohibit running existing test suite

[Copilot]

Bug Hunter was running the existing test suite (make test, pytest, etc.) and reporting failures as bugs. The intent is to find logic bugs through code analysis and prove them with a new minimal failing test — not to be a flaky-test detector.

Prompt changes (gh-aw-bug-hunter.md)

• Reproduce step: must write a new script/test targeting the specific bug found via code analysis; explicitly bans running the existing test suite ("if you did not write the test, a failure is not your finding")
• What to Look For: adds logic errors (wrong conditionals, off-by-one, wrong variable, missing edge-case handling) as the primary target class
• What to Skip: adds — "Test suite failures from running existing tests — do not run make test or similar and report the output as a bug"
• Quality Gate: adds noop trigger — "Your only evidence is an existing test failure you did not cause"
• Issue format: renames Suggested Failing Test → Failing Test; requires full code of the new test, not just an outline

Lock file regenerated via make compile.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• https://api.github.com/repos/ruby/setup-ruby/git/ref/tags/v1
  • Triggering command: /usr/bin/gh gh api /repos/ruby/setup-ruby/git/ref/tags/v1 --jq .object.sha -c=4 -nolocalimports -importcfg /tmp/go-build3071520046/b334/importcfg -pack /home/REDACTED/go/pkg/mod/github.com/spf13/pflag@v1.0.10/bool.go /home/REDACTED/go/pkg/mod/github.com/spf13/pflag@v1.0.10/bool_func.go ux-a�� 0.1-go1.25.7.lin--gdwarf-5 (http block)
  • Triggering command: /usr/bin/gh gh api /repos/ruby/setup-ruby/git/ref/tags/v1 --jq .object.sha /tmp/go-build3071520046/b338/symabis -c=4 -nolocalimports -importcfg /tmp/go-build3071520046/b338/importcfg -pack -asmhdr (http block)
  • Triggering command: /usr/bin/gh gh api /repos/ruby/setup-ruby/git/ref/tags/v1 --jq .object.sha /tmp/go-build3071520046/b337/symabis -c=4 -nolocalimports -importcfg /tmp/go-build3071520046/b337/importcfg -pack -asmhdr (http block)
• https://api.github.com/user
  • Triggering command: /usr/bin/gh gh api user --jq .login go1.25.7 -c=4 -nolocalimports -importcfg /tmp/go-build3071520046/b325/importcfg -pack /home/REDACTED/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.7.linux-amd64/src/testing/allocs.go (http block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Bug Hunter Improvements</issue_title>
<issue_description>See https://github.com/elastic/beats/actions/runs/22206903339

We already know tests are passing, so why is it running tests?

The goal is to identify a bug / logic issue / etc</issue_description>

Comments on the Issue (you are @copilot in this section)

@strawgate I want it to find a bug, build a test that shows that the bug exists, etc. It's not supposed to be just running tests hoping one fails and then calling that a bug

Custom agent used: default-agent
The default agent

• Fixes Bug Hunter Improvements #185

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

[github-actions]

Verdict: APPROVE. I found no actionable issues in this PR.

---

What is this? | From workflow: PR Review Agent

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not. for issue #189

[github-actions]

The failure is from Verify compiled workflows → Ensure lock files are up to date; make compile rewrites generated workflow artifacts, so the branch is out of sync with generated outputs.

Root cause observed in the failed log and reproduced locally with make compile:

• many tracked files under .github/workflows/*.lock.yml and .github/workflows/agentics-maintenance.yml become modified
• two generated trigger files are missing from git: .github/workflows/trigger-flaky-test-triage.yml and .github/workflows/trigger-issue-triage-pr.yml

Validation run:

• make compile (reproduced the same drift)

Recommended fix on the PR branch:

1. Run make compile
2. Commit all generated workflow changes (including the two missing trigger files above)
3. Push and re-run CI

---

What is this? | From workflow: PR Checks Fixer

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

[github-actions]

CI failure is from Verify compiled workflows → Ensure lock files are up to date, not from tests.

make compile succeeds, but the step immediately fails because the repo is dirty after compile:

• Working tree dirty after make compile.
• Modified: many gh-aw-*.lock.yml files plus .github/workflows/agentics-maintenance.yml
• Untracked: .github/workflows/trigger-flaky-test-triage.yml and .github/workflows/trigger-issue-triage-pr.yml

This means generated workflow artifacts are out of sync with the source templates in this branch.

Recommended fix (minimal)

1. Rebase/merge latest main into copilot/fix-bug-hunter-logic.
2. Run make compile locally.
3. Commit all resulting generated workflow updates (including newly generated trigger files).
4. Push and re-run CI.

What I ran

• Inspected workflow run 22207976958 jobs and failed job logs.
• No additional local test suite was run (failure occurs before tests; this is a generation-sync check).

---

What is this? | From workflow: PR CI Detective

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.