Skip to content

[ambient-context] Daily Ambient Context Optimizer - 2026-07-18 #46485

Description

@github-actions

Executive Summary

  • 4 runs sampled from 4 distinct workflows over the last 24 hours
  • Median first-request size: 19,611 chars; P95: 22,761 chars
  • System-injected context (safe-outputs boilerplate, gh-cli instructions) accounts for ~9–11 KB of every request — roughly 50% of total prompt size
  • Highest-cost run: Daily Code Metrics (AIC 279) despite a moderately-sized prompt, indicating many turns; second-highest: Failure Investigator (AIC 250) with the smallest prompt (12K chars)
  • copilot-agent-analysis.md is the largest compiled prompt (22,761 chars) and is missing gh-proxy and cli-proxy configuration

Highest-Leverage Changes

  1. Enable gh-proxy + cli-proxy in copilot-agent-analysis.md — missing despite heavy GitHub API usage; enables MCP-proxied reads and reduces raw shell overhead
  2. Trim dual experiment output templates in copilot-agent-analysis.md — both structured and prose variant templates are inlined (~1,900 chars combined); extract to a shared file or <details> block
  3. Collapse 31-heading step structure in daily-formal-spec-verifier.md — 9.1% duplicate-line ratio and 31 headings for a sequential 7-step workflow; merge sub-steps into checklists
  4. Shorten "Metrics to Collect" section in daily-code-metrics.md — largest named section (985 chars); metrics list can be expressed as a compact table or bullet set
  5. Audit turn count driver in aw-failure-investigator.md — AIC=250 on a 12K prompt means high per-turn cost; review whether all 3 inline sub-agents are each necessary or if issue-matcher can be folded into failure-classifier

CI-Validation Checklist for Implementing Agents

Any agent implementing workflow-file recommendations must complete every item below before opening a PR:

  • Run make recompile for every modified .github/workflows/*.md file — zero compilation errors required
  • Confirm no target workflow file is reduced by more than 40% of its current character count
  • Run make agent-report-progress before the final commit and confirm it passes
  • Check the blocked_files list in /tmp/gh-aw/ambient-context/closed-pr-targets.json (written by Step 4) — do not re-attempt changes to any file that appears in a closed ambient-context optimization PR from the last 14 days
  • Include a smoke-test result or dry-run output in the PR description for every changed workflow spec
  • Document all generated .lock.yml changes in the PR body
  • Do not submit as a draft PR

Key Metrics

Metric Value
Sampled runs 4
Distinct workflows 4
Median chars 19,611
P95 chars 22,761
Largest sampled request 22,761 chars (Copilot Agent PR Analysis)
Merged optimizer PRs (7d) 0
Closed optimizer PRs (7d) 0
Optimizer PR close-rate (7d) N/A (< 3 settled PRs)
Per-Run First-Request Metrics
Run Workflow Chars Lines Headings Code Fences Dup% AIC Source
§29656277112 Daily Code Metrics 19,540 366 30 12 6.1% 279.3 prompt.txt
§29657200469 Failure Investigator 12,335 178 15 4 1.6% 250.5 prompt.txt
§29650426891 Daily Formal Spec Verifier 19,681 392 31 14 9.1% 129.7 prompt.txt
§29655940724 Copilot Agent PR Analysis 22,761 455 40 22 6.2% 119.5 prompt.txt
Repeated Ambient Context Signals
  • - Use \###` (or lower) headers only.` — appears in 2 runs (reporting guidelines repeated per-workflow)
  • - Keep summary and critical actions visible; move long detail into \
    Details` blocks.` — 2 runs
  • - Structure reports as: overview → key metrics/issues → collapsible detail → next actions. — 2 runs
  • **X items found** — [brief description] — 2 runs (output format placeholder repeated)
  • # or write to a file: safeoutputs create_pull_request . < /tmp/payload.json — appears as a heading in 3 of 4 runs (safeoutputs usage example in system context parsed as h1)
  • Reporting format guidelines are inlined per-workflow rather than delegated to a shared import
Deterministic Analysis Output

Key signals from analyze_requests.py:

  • All 4 runs: 0 inline agents, 0 inline linters, 0 SKILL.md references in compiled prompts — skill loading is clean
  • System context (safe-outputs boilerplate + gh-cli instructions): ~9–11 KB per run (~50% of total)
  • copilot-agent-analysis.md is missing tools.github.mode: gh-proxy and tools.cli-proxy: true; the other 3 sampled workflows have both enabled
  • daily-formal-spec-verifier.md has the highest dup-line ratio (9.1%) — 31 headings and repeated constraint/output-quality sections
  • aw-failure-investigator.md has the smallest prompt (12,335 chars) but highest AIC relative to size (AIC=250) — cost is driven by turns, not prompt size; source file is 20 KB with 3 inline sub-agents

Recommendations by Category

Workflow Markdown

R1 — Add gh-proxy + cli-proxy to copilot-agent-analysis.md (high, safe immediately)

  • Affected: .github/workflows/copilot-agent-analysis.md
  • Evidence: missing tools.github.mode: gh-proxy and tools.cli-proxy: true; all other sampled workflows have both
  • Impact: enables MCP proxied reads; reduces redundant raw-CLI round trips
  • Add to frontmatter:
    tools:
      cli-proxy: true
      github:
        mode: gh-proxy

R2 — Extract dual variant templates to a shared file (medium, needs manual review)

  • Affected: .github/workflows/copilot-agent-analysis.md
  • Evidence: both structured and prose output templates are inlined (~1,900 chars); largest single section in any sampled run
  • Impact: moving to shared/copilot-agent-analysis-templates.md shrinks the main prompt by ~1,500–1,900 chars
  • Note: A/B experiment is ongoing — both templates must remain accessible at compile time

R3 — Collapse step granularity in daily-formal-spec-verifier.md (medium, needs manual review)

  • Affected: .github/workflows/daily-formal-spec-verifier.md
  • Evidence: 31 headings, 9.1% dup-line ratio, 19,681 chars; sub-steps like ## Step 1b and repeated Constraints/Key Invariants/Edge Cases blocks inflate the prompt
  • Impact: merging sub-steps into checklists and consolidating constraint sections could reduce by 10–15%

R4 — Shorten "Metrics to Collect" section in daily-code-metrics.md (medium, safe immediately)

  • Affected: .github/workflows/daily-code-metrics.md
  • Evidence: largest named section (985 chars) with individually named metrics in verbose prose form
  • Impact: expressing as a compact YAML-style list or table could save 300–500 chars

Skills

No skill-loading issues detected. All 4 workflows produced 0 SKILL.md references in compiled prompts — skill loading is already clean.

Agents

R5 — Evaluate folding issue-matcher into failure-classifier in aw-failure-investigator.md (low, needs manual review)

  • Affected: .github/workflows/aw-failure-investigator.md
  • Evidence: AIC=250 on a 12K compiled prompt (3K workflow context) — cost is turn-driven; source is 20 KB with 3 inline sub-agents; issue-matcher is the lightest sub-agent (clusters + issues → matched/gap JSON)
  • Impact: eliminating one sub-agent turn could reduce AIC by ~10–20%
  • Caution: all 3 sub-agents already use model: small; only merge if combined context fits small-model limits

References

  • §29656277112 — Daily Code Metrics (AIC 279, highest cost)
  • §29657200469 — Failure Investigator (AIC 250, turn-driven cost)
  • §29655940724 — Copilot Agent PR Analysis (largest prompt, missing proxies)

Generated by 🌫️ Daily Ambient Context Optimizer · 136.9 AIC · ⌖ 12 AIC · ⊞ 7.7K ·

  • expires on Jul 25, 2026, 12:17 PM UTC-08:00

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions