Skip to content

feat: optimize Smoke Services workflow for token efficiency#2214

Merged
lpcox merged 2 commits intomainfrom
copilot/update-token-usage-report
Apr 25, 2026
Merged

feat: optimize Smoke Services workflow for token efficiency#2214
lpcox merged 2 commits intomainfrom
copilot/update-token-usage-report

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 25, 2026

Smoke Services averaged 173K tokens/run and 14 turns/run — highest turn count of all smoke workflows — largely because the agent was installing redis-tools and postgresql-client via apt-get on every run and running each connectivity check as a separate tool call.

Changes

smoke-services.md

  • Pre-install client tools in steps:redis-tools and postgresql-client are installed on the runner before the agent starts. AWF bind-mounts /usr read-only into the agent chroot, so the binaries are immediately available inside the sandbox without any agent-side installation.
  • Single compound bash command — all three checks (Redis PING, pg_isready, psql SELECT 1) are collapsed into one bash invocation; agent just reads output and posts a comment (~3 turns vs ~14).
  • Remove agentic-workflows tool — unused for a connectivity test; eliminates the Install gh-aw extension step, agenticworkflows MCP server, and alpine:latest container pull.
  • Remove cache-memory tool — stateless test needs no cross-run memory; drops 7 steps (create dir, TTL key, restore, setup git repo, strip execute bits, scan injection, commit) and the actions/cache action.
  • Remove node from network.allowed — no npm/Node.js usage; shrinks the domain allowlist by ~24 entries, reducing system prompt size on every turn.
  • Reduce timeout-minutes 15 → 10 — task is now simple enough that 10 min is generous.

smoke-services.lock.yml

Manually updated to match all .md changes while preserving the v0.71.1 compiled infrastructure. Notable: --enable-host-access --allow-host-ports 80,443,8080 replaced with --allow-host-service-ports 6379,5432 (more restrictive — only the two service ports are opened to host.docker.internal).

Expected impact: ~40–50K tokens/run (~3 turns) vs current ~173K/run (~14 turns).

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/repos/actions/github-script/git/ref/tags/v9
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v9 --jq .object.sha (http block)
  • https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v0.68.3
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v0.68.3 --jq .object.sha (http block)
  • https://api.github.com/repos/github/gh-aw/releases/latest
    • Triggering command: /usr/bin/gh gh extension install github/gh-aw --pin v0.71.1 (http block)
    • Triggering command: /usr/bin/gh gh extension install github/gh-aw (http block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot AI linked an issue Apr 25, 2026 that may be closed by this pull request
Copilot AI changed the title [WIP] Update Copilot token usage report for April 24, 2026 feat: optimize Smoke Services workflow for token efficiency Apr 25, 2026
Copilot AI requested a review from lpcox April 25, 2026 20:56
@lpcox lpcox marked this pull request as ready for review April 25, 2026 22:06
@lpcox lpcox requested a review from Mossaka as a code owner April 25, 2026 22:06
Copilot AI review requested due to automatic review settings April 25, 2026 22:06
@github-actions
Copy link
Copy Markdown
Contributor

🔥 Smoke Test: Copilot BYOK (Offline Mode)

Test Result
GitHub MCP (list PRs)
GitHub.com connectivity
File write/read
BYOK inference (agent → api-proxy → api.githubcopilot.com)

Running in BYOK offline mode (COPILOT_OFFLINE=true) via api-proxy → api.githubcopilot.com

Overall: PASS | PR by @Copilot · Assignees: @lpcox, @Copilot

🔑 BYOK report filed by Smoke Copilot BYOK

@github-actions
Copy link
Copy Markdown
Contributor

Smoke Test Results: PASS

  • ✅ GitHub API: Last 2 merged PRs retrieved
  • ✅ Playwright: github.com page title verified
  • ✅ File I/O: Test file created with timestamp
  • ✅ Bash: File contents verified

All smoke tests passed successfully.

💥 [THE END] — Illustrated by Smoke Claude

@github-actions
Copy link
Copy Markdown
Contributor

🔥 Smoke Test Results

Test Status
GitHub MCP connectivity
GitHub.com HTTP connectivity
File write/read ✅ (smoke-test-copilot-24940470956.txt confirmed)

PR: feat: optimize Smoke Services workflow for token efficiency
Author: @Copilot | Assignees: @lpcox, @Copilot

Overall: PASS 🟢

📰 BREAKING: Report filed by Smoke Copilot

@github-actions
Copy link
Copy Markdown
Contributor

Smoke Test

PRs: chore: upgrade agentic workflows from v0.69.3 to v0.71.1; fix: strip IPv6 http_port listeners when IPv6 is disabled in Squid container
GitHub merged PR review: ✅
safeinputs-gh PR query: ❌ tool unavailable in this session
Playwright github.com title: ✅
Tavily search: ❌ tool unavailable in this session
File write: ✅
Bash cat readback: ✅
Discussion comment: ✅
Build npm ci && npm run build: ✅
Overall: FAIL

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • registry.npmjs.org

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "registry.npmjs.org"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Optimizes the “Smoke Services” agentic workflow to reduce token/turn usage by removing unused tools, tightening network/host access, and pre-installing required service client binaries outside the agent sandbox.

Changes:

  • Removes agentic-workflows and cache-memory, and collapses Redis/Postgres checks into a single bash invocation in the workflow instructions.
  • Adds a runner steps: pre-install for redis-tools and postgresql-client, and reduces timeouts to 10 minutes.
  • Updates the compiled lock workflow to match the .md source, including switching host access to --allow-host-service-ports 6379,5432 and shrinking the allowed domains list.
Show a summary per file
File Description
.github/workflows/smoke-services.md Updates workflow config/instructions to reduce tool usage and token footprint; adds runner pre-install step and shorter timeout.
.github/workflows/smoke-services.lock.yml Manually aligns compiled workflow with .md updates (tool removal, host port restrictions, allowed domains, timeouts, pre-install step).
.github/aw/actions-lock.json Removes gh-aw action pin entries (intended as part of workflow/tool simplification), but may impact future determinism of upgrades/compiles.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comments suppressed due to low confidence (1)

.github/aw/actions-lock.json:63

  • .github/aw/actions-lock.json no longer contains pins for github/gh-aw-actions/setup@v0.71.1 and github/gh-aw-actions/setup-cli@v0.71.1, but the repo still uses those actions widely (e.g. .github/workflows/agentics-maintenance.yml and many *.lock.yml files). This makes future gh aw upgrade / gh aw compile runs less reproducible and can force network lookups for those tags, which is especially problematic in firewalled environments. Please restore entries for these actions (repo, version, sha) so action resolution stays deterministic.
      "version": "v4.0.0",
      "sha": "4d04d5d9486b7bd6fa91e7baf45bbb4f8b9deedd"
    },
    "github/gh-aw/actions/setup@v0.69.3": {
      "repo": "github/gh-aw/actions/setup",
      "version": "v0.69.3",
      "sha": "6abd7107cebd8b300f4d64013201ad2d8e0a994f"
    },
    "softprops/action-gh-release@v3.0.0": {
  • Files reviewed: 3/3 changed files
  • Comments generated: 1

@@ -213,7 +213,6 @@ jobs:
cat "${RUNNER_TEMP}/gh-aw/prompts/temp_folder_prompt.md"
cat "${RUNNER_TEMP}/gh-aw/prompts/markdown.md"
cat "${RUNNER_TEMP}/gh-aw/prompts/agentic_workflows_guide.md"
Copy link

Copilot AI Apr 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This workflow removes the agentic-workflows tool, but the prompt assembly still includes cat "${RUNNER_TEMP}/gh-aw/prompts/agentic_workflows_guide.md". Keeping this guide increases prompt size (hurting the token-efficiency goal) and can confuse the agent by documenting a tool that is no longer available. Please remove this prompt include (and any related prompt section) now that agentic-workflows is not enabled for this workflow.

Suggested change
cat "${RUNNER_TEMP}/gh-aw/prompts/agentic_workflows_guide.md"

Copilot uses AI. Check for mistakes.
@github-actions
Copy link
Copy Markdown
Contributor

🏗️ Build Test Suite Results

Ecosystem Project Build/Install Tests Status
Bun elysia 1/1 passed ✅ PASS
Bun hono 1/1 passed ✅ PASS
C++ fmt N/A ✅ PASS
C++ json N/A ✅ PASS
Deno oak N/A 1/1 passed ✅ PASS
Deno std N/A 1/1 passed ✅ PASS
.NET hello-world N/A ✅ PASS
.NET json-parse N/A ✅ PASS
Go color 1/1 passed ✅ PASS
Go env 1/1 passed ✅ PASS
Go uuid 1/1 passed ✅ PASS
Java gson 1/1 passed ✅ PASS
Java caffeine 1/1 passed ✅ PASS
Node.js clsx all passed ✅ PASS
Node.js execa all passed ✅ PASS
Node.js p-limit all passed ✅ PASS
Rust fd 1/1 passed ✅ PASS
Rust zoxide 1/1 passed ✅ PASS

Overall: 8/8 ecosystems passed — ✅ PASS

Generated by Build Test Suite for issue #2214 · ● 480.7K ·

@lpcox lpcox merged commit 3c8fff3 into main Apr 25, 2026
66 of 71 checks passed
@lpcox lpcox deleted the copilot/update-token-usage-report branch April 25, 2026 22:13
lpcox added a commit that referenced this pull request Apr 25, 2026
* chore: recompile all agentic workflows

Recompile all 30 lock files with gh-aw v0.71.1 compiler to pick up
changes from recently merged workflow .md updates. Key changes:
- Remove sha256 digest pins from container image references
- smoke-services.lock.yml updates from #2214

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore: upgrade mcpg from v0.2.29 to v0.3.0 in remaining workflows

Update sandbox.mcp.version from v0.2.29 to v0.3.0 in:
- smoke-claude.md
- smoke-codex.md
- smoke-copilot.md

All 30 workflows now use mcpg v0.3.0.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

📊 Copilot Token Usage Report2026-04-24

3 participants