Skip to content

optimize(secret-digger-claude): default threat detection to Haiku, drop version-reporting import#1974

Merged
lpcox merged 3 commits intomainfrom
copilot/optimize-claude-token-usage-again
Apr 14, 2026
Merged

optimize(secret-digger-claude): default threat detection to Haiku, drop version-reporting import#1974
lpcox merged 3 commits intomainfrom
copilot/optimize-claude-token-usage-again

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 14, 2026

The secret-digger-claude threat detection step was defaulting to Sonnet (claude-sonnet-4-6) because vars.GH_AW_MODEL_DETECTION_CLAUDE was unset, making 2 of 3 API calls use the expensive model. Projected cost: ~$0.51/run → ~$0.14/run (−73%).

Changes

  • secret-digger-claude.lock.yml — change detection model fallback from '' to 'claude-haiku-4-5-20251001':

    # Before
    GH_AW_MODEL_DETECTION_CLAUDE: ${{ vars.GH_AW_MODEL_DETECTION_CLAUDE || '' }}
    
    # After
    GH_AW_MODEL_DETECTION_CLAUDE: ${{ vars.GH_AW_MODEL_DETECTION_CLAUDE || 'claude-haiku-4-5-20251001' }}

    The repo variable still takes precedence when set; this just eliminates the silent Sonnet fallback.

  • secret-digger-claude.md — remove shared/version-reporting.md import; version metadata adds no investigative value in a security scan (~120 token savings per run).

  • All *.lock.yml — recompiled via gh aw compile + post-processing script after the .md change.

…n-reporting import

- Set GH_AW_MODEL_DETECTION_CLAUDE default to claude-haiku-4-5-20251001 in lock file
  (was empty string, causing fallback to Sonnet when repo var unset — ~73% cost savings)
- Remove shared/version-reporting.md import from secret-digger-claude.md (~120 token savings)
- Recompile all lock files via gh aw compile + postprocess script

Agent-Logs-Url: https://github.com/github/gh-aw-firewall/sessions/e49af1c2-1b78-4f62-b8cf-8b4da06508e8

Co-authored-by: lpcox <15877973+lpcox@users.noreply.github.com>
Copilot AI changed the title [WIP] Optimize token usage for Claude workflow optimize(secret-digger-claude): default threat detection to Haiku, drop version-reporting import Apr 14, 2026
Copilot AI requested a review from lpcox April 14, 2026 15:14
@lpcox lpcox marked this pull request as ready for review April 14, 2026 16:44
@lpcox lpcox requested a review from Mossaka as a code owner April 14, 2026 16:44
Copilot AI review requested due to automatic review settings April 14, 2026 16:44
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR reduces secret-digger-claude threat-detection cost and prompt size by defaulting the detection model to Haiku when the repo variable is unset, and by removing version-reporting prompt content. It also includes a repo-wide refresh of generated workflow lock files (including scattered cron schedule updates) from recompilation.

Changes:

  • Default GH_AW_MODEL_DETECTION_CLAUDE to claude-haiku-4-5-20251001 in the compiled secret-digger-claude workflow to avoid the implicit expensive-model fallback when the repo variable is unset.
  • Remove the shared/version-reporting.md import from secret-digger-claude.md (and corresponding compiled prompt assembly).
  • Recompile/update generated workflow artifacts (*.lock.yml) and the generated maintenance workflow content.
Show a summary per file
File Description
.github/workflows/test-coverage-improver.lock.yml Recompiled lock file; updates scattered weekly cron time.
.github/workflows/smoke-services.lock.yml Recompiled lock file; updates scattered cron minute for 12-hour schedule.
.github/workflows/smoke-copilot.lock.yml Recompiled lock file; updates scattered cron minute for 12-hour schedule.
.github/workflows/smoke-codex.lock.yml Recompiled lock file; updates scattered cron minute for 12-hour schedule.
.github/workflows/smoke-claude.lock.yml Recompiled lock file; updates scattered cron minute for 12-hour schedule.
.github/workflows/security-review.lock.yml Recompiled lock file; updates scattered daily cron minute.
.github/workflows/secret-digger-claude.md Removes shared/version-reporting.md import to reduce prompt tokens.
.github/workflows/secret-digger-claude.lock.yml Removes version-reporting runtime import and defaults detection model to Haiku in env fallback.
.github/workflows/pelis-agent-factory-advisor.lock.yml Recompiled lock file; updates scattered daily cron time.
.github/workflows/issue-monster.lock.yml Recompiled lock file; updates scattered hourly cron minute.
.github/workflows/firewall-issue-dispatcher.lock.yml Recompiled lock file; updates scattered 6-hour cron minute.
.github/workflows/doc-maintainer.lock.yml Recompiled lock file; updates scattered daily cron time.
.github/workflows/dependency-security-monitor.lock.yml Recompiled lock file; updates scattered daily cron time.
.github/workflows/copilot-token-usage-analyzer.lock.yml Recompiled lock file; updates scattered daily cron time.
.github/workflows/cli-flag-consistency-checker.lock.yml Recompiled lock file; updates scattered weekly cron time.
.github/workflows/claude-token-usage-analyzer.lock.yml Recompiled lock file; updates scattered daily cron time.
.github/workflows/ci-cd-gaps-assessment.lock.yml Recompiled lock file; updates scattered daily cron time.
.github/workflows/agentics-maintenance.yml Generated maintenance workflow updated (tooling/action versions + workflow_dispatch input now a choice list).

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 18/18 changed files
  • Comments generated: 1

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@github-actions
Copy link
Copy Markdown
Contributor

✅ Coverage Check Passed

Overall Coverage

Metric Base PR Delta
Lines 85.24% 85.33% 📈 +0.09%
Statements 85.11% 85.20% 📈 +0.09%
Functions 87.57% 87.57% ➡️ +0.00%
Branches 77.58% 77.63% 📈 +0.05%
📁 Per-file Coverage Changes (1 files)
File Lines (Before → After) Statements (Before → After)
src/docker-manager.ts 86.3% → 86.6% (+0.31%) 85.8% → 86.2% (+0.31%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

@github-actions
Copy link
Copy Markdown
Contributor

Smoke Test Results

Overall: PASS

💥 [THE END] — Illustrated by Smoke Claude

@github-actions
Copy link
Copy Markdown
Contributor

🔥 Smoke Test Results — PASS

Test Result
GitHub MCP (list_pull_requests) — latest merged: "secret-digger-claude: switch to Haiku, lower max-turns to 4"
GitHub.com connectivity
File write/read (smoke-test-copilot-24411708834.txt)

Overall: PASS · PR: "optimize(secret-digger-claude): default threat detection to Haiku, drop version-reporting import" · Author: @app/copilot-swe-agent · Assignees: @lpcox @Copilot

📰 BREAKING: Report filed by Smoke Copilot

@github-actions

This comment has been minimized.

@github-actions
Copy link
Copy Markdown
Contributor

🔮 The ancient spirits stir: this smoke-test agent has passed through these halls, and where tools were veiled, it marked the omens plainly.

🔮 The oracle has spoken through Smoke Codex

@github-actions
Copy link
Copy Markdown
Contributor

Smoke Test: GitHub Actions Services Connectivity

Check Status
Redis PING (host.docker.internal:6379) PONG
PostgreSQL ready (host.docker.internal:5432) ✅ accepting connections
PostgreSQL SELECT 1 (smoketest db, user postgres) ✅ returned 1

All checks passed. (redis-cli was unavailable; Redis was tested via nc with raw RESP protocol — same result.)

🔌 Service connectivity validated by Smoke Services

@github-actions
Copy link
Copy Markdown
Contributor

🏗️ Build Test Suite Results

Ecosystem Project Build/Install Tests Status
Bun elysia 1/1 passed ✅ PASS
Bun hono 1/1 passed ✅ PASS
C++ fmt N/A ✅ PASS
C++ json N/A ✅ PASS
Deno oak N/A 1/1 passed ✅ PASS
Deno std N/A 1/1 passed ✅ PASS
.NET hello-world N/A ✅ PASS
.NET json-parse N/A ✅ PASS
Go color 1/1 passed ✅ PASS
Go env 1/1 passed ✅ PASS
Go uuid 1/1 passed ✅ PASS
Java gson 1/1 passed ✅ PASS
Java caffeine 1/1 passed ✅ PASS
Node.js clsx All passed ✅ PASS
Node.js execa All passed ✅ PASS
Node.js p-limit All passed ✅ PASS
Rust fd 1/1 passed ✅ PASS
Rust zoxide 1/1 passed ✅ PASS

Overall: 8/8 ecosystems passed — ✅ PASS

Generated by Build Test Suite for issue #1974 · ● 605.5K ·

@lpcox lpcox merged commit 80e0622 into main Apr 14, 2026
50 of 52 checks passed
@lpcox lpcox deleted the copilot/optimize-claude-token-usage-again branch April 14, 2026 17:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

⚡ Claude Token Optimization2026-04-13 — Secret Digger (Claude)

3 participants