Skip to content

⚡ Optimize secret-digger-copilot: remove unused tools, condense prompt, add emergency exit rule#1661

Merged
lpcox merged 4 commits intomainfrom
copilot/optimize-token-usage
Apr 4, 2026
Merged

⚡ Optimize secret-digger-copilot: remove unused tools, condense prompt, add emergency exit rule#1661
lpcox merged 4 commits intomainfrom
copilot/optimize-token-usage

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 4, 2026

The secret-digger-copilot workflow loads 22 GitHub MCP tools it never uses, carries ~300-char per-technique command sub-lists that are redundant, and has no guard against the confirmed retry-loop bug where the model emits 2 tokens and triggers 5 extra full-cost retries.

Changes to shared/secret-audit.md

  • Remove github: toolsets — agent uses zero GitHub MCP calls; all container exploration is bash, issue filing is safeoutputs.create_issue
  • Remove edit: tool — loaded but never used; cache writes go through bash
  • Add Emergency Exit Rule — explicit noop instruction when investigation yields nothing substantive, breaking the near-empty-response retry loop:

    "If you have nothing substantive to report after completing your investigation, call noop with a summary message instead of producing empty or minimal output."

  • Condense technique sections — 10 multi-bullet sub-lists collapsed to single-line summaries with one concrete command example each (~3KB prompt reduction per run)

Lock file

Recompiled secret-digger-copilot.lock.yml with gh aw compile (v0.47.0 → v0.66.1) and post-processed with postprocess-smoke-workflows.ts.

Not applied

max-turns: 4 (Rec 1A) — compiler rejects it: "engine 'copilot' does not support the max-turns feature". The Emergency Exit Rule is the fallback guard.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v0.66.1
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v0.66.1 --jq .object.sha (http block)
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v0.66.1 --jq .object.sha d -token-usage (http block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot AI linked an issue Apr 4, 2026 that may be closed by this pull request
10 tasks
… add emergency exit rule

- Remove github toolsets and edit tool from shared/secret-audit.md frontmatter
- Add Emergency Exit Rule to prevent near-empty response retry loops
- Condense technique examples from 8-bullet to 1-line-with-examples per item (~3KB savings)
- Recompile secret-digger-copilot.lock.yml with gh-aw v0.66.1
- Post-process all lock files

Note: max-turns:4 not added - copilot engine does not support max-turns

Agent-Logs-Url: https://github.com/github/gh-aw-firewall/sessions/5a262b3d-b3ef-4910-9c45-598c3d00ef31

Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
Copilot AI changed the title [WIP] Optimize Copilot token usage ⚡ Optimize secret-digger-copilot: remove unused tools, condense prompt, add emergency exit rule Apr 4, 2026
Copilot AI requested a review from lpcox April 4, 2026 15:08
Copilot finished work on behalf of lpcox April 4, 2026 15:08
@lpcox lpcox marked this pull request as ready for review April 4, 2026 15:30
@lpcox lpcox requested a review from Mossaka as a code owner April 4, 2026 15:30
Copilot AI review requested due to automatic review settings April 4, 2026 15:30
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 4, 2026

✅ Coverage Check Passed

Overall Coverage

Metric Base PR Delta
Lines 86.11% 86.29% 📈 +0.18%
Statements 85.98% 86.16% 📈 +0.18%
Functions 87.41% 87.41% ➡️ +0.00%
Branches 78.50% 78.62% 📈 +0.12%
📁 Per-file Coverage Changes (2 files)
File Lines (Before → After) Statements (Before → After)
src/cli.ts 61.3% → 61.8% (+0.48%) 61.8% → 62.3% (+0.47%)
src/docker-manager.ts 86.4% → 87.0% (+0.58%) 85.9% → 86.5% (+0.57%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

@github-actions

This comment has been minimized.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR optimizes the secret-digger-copilot agentic workflow prompt/config to reduce unused tool exposure and shrink the investigation instructions, while adding an explicit “emergency exit” path to avoid near-empty outputs that can trigger costly retry loops.

Changes:

  • Remove unused tool declarations from the shared secret-audit prompt component and condense “technique” sections.
  • Add an “Emergency Exit Rule” instructing the agent to call noop with a summary instead of emitting minimal output.
  • Recompile and update secret-digger-copilot.lock.yml to newer gh-aw outputs (v0.47.0 → v0.66.1), including updated activation/agent plumbing and artifact handling.
Show a summary per file
File Description
.github/workflows/shared/secret-audit.md Trims declared tools, condenses technique lists, and adds an emergency noop rule to prevent empty-output retry loops.
.github/workflows/secret-digger-copilot.lock.yml Regenerated workflow lockfile reflecting new prompt content and updated gh-aw compilation/runtime steps.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comments suppressed due to low confidence (5)

.github/workflows/secret-digger-copilot.lock.yml:294

  • github/gh-aw-actions/setup is referenced by a mutable tag (@v0.66.1). For supply-chain safety and reproducibility, pin this action to a commit SHA (most other workflows do this, e.g. .github/workflows/claude-token-optimizer.lock.yml:73).
      - name: Setup Scripts
        uses: github/gh-aw-actions/setup@v0.66.1
        with:
          destination: ${{ runner.temp }}/gh-aw/actions

.github/workflows/secret-digger-copilot.lock.yml:837

  • github/gh-aw-actions/setup is referenced by a mutable tag (@v0.66.1). For supply-chain safety and reproducibility, pin this action to a commit SHA (most other workflows do this, e.g. .github/workflows/claude-token-optimizer.lock.yml:73).
      - name: Setup Scripts
        uses: github/gh-aw-actions/setup@v0.66.1
        with:
          destination: ${{ runner.temp }}/gh-aw/actions

.github/workflows/secret-digger-copilot.lock.yml:923

  • github/gh-aw-actions/setup is referenced by a mutable tag (@v0.66.1). For supply-chain safety and reproducibility, pin this action to a commit SHA (most other workflows do this, e.g. .github/workflows/claude-token-optimizer.lock.yml:73).
      - name: Setup Scripts
        uses: github/gh-aw-actions/setup@v0.66.1
        with:
          destination: ${{ runner.temp }}/gh-aw/actions

.github/workflows/secret-digger-copilot.lock.yml:1111

  • github/gh-aw-actions/setup is referenced by a mutable tag (@v0.66.1). For supply-chain safety and reproducibility, pin this action to a commit SHA (most other workflows do this, e.g. .github/workflows/claude-token-optimizer.lock.yml:73).
      - name: Setup Scripts
        uses: github/gh-aw-actions/setup@v0.66.1
        with:
          destination: ${{ runner.temp }}/gh-aw/actions

.github/workflows/secret-digger-copilot.lock.yml:1174

  • github/gh-aw-actions/setup is referenced by a mutable tag (@v0.66.1). For supply-chain safety and reproducibility, pin this action to a commit SHA (most other workflows do this, e.g. .github/workflows/claude-token-optimizer.lock.yml:73).
      - name: Setup Scripts
        uses: github/gh-aw-actions/setup@v0.66.1
        with:
          destination: ${{ runner.temp }}/gh-aw/actions
  • Files reviewed: 2/2 changed files
  • Comments generated: 2

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 4, 2026

🏗️ Build Test Suite Results

Ecosystem Project Build/Install Tests Status
Bun elysia 1/1 passed ✅ PASS
Bun hono 1/1 passed ✅ PASS
C++ fmt N/A ✅ PASS
C++ json N/A ✅ PASS
Deno oak N/A 1/1 passed ✅ PASS
Deno std N/A 1/1 passed ✅ PASS
.NET hello-world N/A ✅ PASS
.NET json-parse N/A ✅ PASS
Go color 1/1 passed ✅ PASS
Go env 1/1 passed ✅ PASS
Go uuid 1/1 passed ✅ PASS
Java gson 1/1 passed ✅ PASS
Java caffeine 1/1 passed ✅ PASS
Node.js clsx All passed ✅ PASS
Node.js execa All passed ✅ PASS
Node.js p-limit All passed ✅ PASS
Rust fd 1/1 passed ✅ PASS
Rust zoxide 1/1 passed ✅ PASS

Overall: 8/8 ecosystems passed — ✅ PASS

Generated by Build Test Suite for issue #1661 ·

lpcox and others added 2 commits April 4, 2026 08:53
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 4, 2026

Smoke Test Results — Claude Engine

✅ GitHub MCP: #1662 test: add missing Gemini API target test coverage | #1656 chore(deps): bump defu from 6.1.4 to 6.1.6
✅ Playwright: github.com title contains "GitHub"
✅ File Write: /tmp/gh-aw/agent/smoke-test-claude-23982295391.txt created
✅ Bash: file content verified

Overall: PASS

💥 [THE END] — Illustrated by Smoke Claude

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 4, 2026

🔥 Smoke Test Results

Test Result
GitHub MCP (PR list)
GitHub.com HTTP 200
File write/read smoke-test-copilot-23982295396.txt

Overall: PASS

PR: "⚡ Optimize secret-digger-copilot: remove unused tools, condense prompt, add emergency exit rule" by @Copilot — assignees: @lpcox, @Copilot

📰 BREAKING: Report filed by Smoke Copilot

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 4, 2026

Smoke test results:

  • GitHub MCP (last 2 merged PR titles): ✅ feat: add Google Gemini API proxy support (port 10003); test: add missing Gemini API target test coverage to fix CI branch coverage regression
  • safeinputs-gh (pr list --repo github/gh-aw-firewall --limit 2 --json number,title,author): ❌ tool unavailable in this runtime
  • Playwright (https://github.com title contains GitHub): ❌ MCP EACCES writing /tmp/gh-aw/mcp-logs/playwright/...
  • Tavily search (GitHub Agentic Workflows Firewall): ❌ Tavily MCP not available
  • File write (/tmp/gh-aw/agent/smoke-test-codex-23982295425.txt): ✅
  • Bash verify (cat file): ✅
  • Discussion query + mystical discussion comment: ❌ github-discussion-query safe-input tool unavailable
  • Build (npm ci && npm run build): ✅
    Overall status: FAIL

🔮 The oracle has spoken through Smoke Codex

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 4, 2026

Smoke Test: GitHub Actions Services Connectivity ✅

All connectivity checks passed:

Service Check Result
Redis PINGhost.docker.internal:6379 PONG
PostgreSQL pg_isreadyhost.docker.internal:5432 ✅ accepting connections
PostgreSQL psql SELECT 1 on smoketest db as postgres ✅ returns 1

🔌 Service connectivity validated by Smoke Services

@github-actions github-actions bot mentioned this pull request Apr 4, 2026
@lpcox lpcox merged commit bc01736 into main Apr 4, 2026
56 of 59 checks passed
@lpcox lpcox deleted the copilot/optimize-token-usage branch April 4, 2026 16:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

⚡ Copilot Token Optimization2026-04-04 — Secret Digger (Copilot)

3 participants