[agentrx-optimizer] Daily Workflow Optimization - 2026-07-09

[github-actions]

Executive Summary

AgentRx normalized the 15 most recent gh-aw agent session runs (MCP logs + audit) into trajectory IR and ran a deterministic invariant check. 4 of 15 runs failed (27%), and every failure occurred at the same critical step: the agent job's engine-execution stage (Execute <engine> CLI / Start MCP Gateway). Two of the four failures never reached the model at all (0 tokens, ~6 min) — a classic transient startup/gateway failure signature. Top recommendation: wrap the agent-execution stage in a bounded retry-with-backoff plus an MCP-gateway health precondition, which addresses the critical step behind 100% of failures and recovers the cheap transient ones at near-zero cost.

AgentRx Evidence

• Critical step: agent job → Execute <engine> CLI (3 runs) / Start MCP Gateway (1 run) — step index is the terminal agent stage in every failing trajectory.
• Failure category: Execution/ToolExecutionFailure → Outcome/TaskCompletionFailure (invariants agent_execution_stage_must_succeed, run_terminal_must_be_success).
• Frequency / impact: 10 invariant violations, all concentrated in 4 runs / 4 distinct workflows. agent_execution_stage_must_succeed = 5, run_terminal_must_be_success = 4, resource_budget_within_limit = 1. Failure rate 27% (4/15); the read-only automation cohort otherwise passed clean.
• Representative runs:
  • §28999022875 — GitHub Remote MCP Authentication Test — failed at Start MCP Gateway, 0 tokens, 6.2m (never reached model)
  • §28996995622 — Smoke CI — failed at Execute Copilot CLI, 0 tokens, 6.3m (never reached model)
  • §28997015233 — Avenger — failed at Execute Claude Code CLI after 1.54M tokens / 27 turns / 26.2m

Labeled violation classification

Judge stage was unavailable (see limitations); the table below is a deterministic classification of the real check.json violations against the candidate fix-type taxonomy.

violation	evidence	fix_type	rationale
agent_execution_stage_must_succeed (5×, 4 runs)	agent-job engine step → FAILED: Smoke CI (0 tok/6.3m), MCP Auth Test @ Start MCP Gateway (0 tok/6.2m), PR Reviewer (103k tok/1 turn/23.3m), Avenger (1.54M tok/27 turns/26.2m)	improving retry/backoff strategy (+ precondition health-check)	Critical step in 100% of failures. 2/4 never reached the model (0 tokens, ~6 min) → transient startup/gateway failures a bounded retry recovers cheaply.
run_terminal_must_be_success (4×)	conclusion=failure terminal state in the same 4 runs	improving retry/backoff strategy	Downstream symptom of the agent-execution failure — same root cause, no independent fix.
resource_budget_within_limit (1×)	Avenger consumed 1.54M tokens over 27 turns / 26.2m, then failed	reducing token-heavy context payloads	Secondary: the one long-running failure burned >1M tokens before dying; context/turn bloat compounds the cost of the failure.

AgentRx Artifacts

IR summary — 15 trajectories normalized (flash domain converter, no LLM fallback), 2,082 steps total, 15/15 valid. One trajectory per run; job steps mapped to IR steps with the agent-execution stage marked.

Static invariants (deterministic python_check, authored from gh-aw success criteria)

• agent_execution_stage_must_succeed — flags any agent-job substep marked FAILED (conclusion=...)
• run_terminal_must_be_success — flags terminal conclusion=failure
• resource_budget_within_limit — flags runs consuming >1M tokens

Checker highlights — 10 violations across 4 trajectories:

• run-28997015233-Avenger → 4 (2× agent-stage, 1× terminal, 1× resource-budget)
• run-28996995622-Smoke_CI → 2 (agent-stage + terminal)
• run-28997020433-PR_Code_Quality_Reviewer → 2 (agent-stage + terminal)
• run-28999022875-GitHub_Remote_MCP_Authentication_Test → 2 (agent-stage + terminal)

Judge classification — unavailable this run.

Known limitations

• static, dynamic, judge, report stages require a live LLM endpoint (copilot CLI / azure / trapi). In this sandbox the static endpoint returned non-JSON and the judge call blocked past timeout, so those stages were skipped per the runbook fallback.
• The check stage was therefore driven by hand-authored deterministic python_check invariants grounded directly in the MCP telemetry (job-step conclusions, token/turn counts) — no fabricated LLM output.
• Per-turn tool-call traces were not in the usage artifact; step-level granularity comes from job_details + audit metrics.

Recommended Optimization

Wrap the agent-execution stage in a bounded retry-with-backoff and add an MCP-gateway readiness precondition.

• One specific change: In the compiled agent job, make Start MCP Gateway block on a health/readiness probe (poll the gateway port until ready or a short deadline), and give Execute <engine> CLI a bounded retry (e.g. 2 attempts with exponential backoff) that only re-runs on infrastructure/startup failures — not on genuine agent completions.
• Why highest impact: This single change targets the critical step behind 100% (4/4) of observed failures. The two zero-token, ~6-minute failures (Smoke CI, MCP Auth Test) demonstrably never reached the model — exactly the transient class a retry/health-gate eliminates, converting failed runs into passes with negligible added cost.
• Where to implement:
  • pkg/workflow/mcp_setup_generator.go:521 — Start MCP Gateway step generation (add readiness probe)
  • pkg/workflow/copilot_engine_execution.go:615 — Execute GitHub Copilot CLI step (add bounded retry/backoff)
  • pkg/workflow/claude_engine.go — equivalent Execute Claude Code CLI step
• Secondary (not the primary ask): trim Avenger's context/turn budget — it burned 1.54M tokens over 27 turns before failing; a lower turn cap or context pruning reduces the blast radius of a failing long run.

Validation Plan

• Confirm improvement next run: re-run this optimizer over the next 15 runs and compare agent_execution_stage_must_succeed violation count; expect the zero-token startup failures (Smoke CI / MCP Auth Test class) to drop to 0.
• Expected metric changes: failure rate 27% → ≤7% (1/15); run_terminal_must_be_success violations fall in lockstep with agent-stage recoveries; no increase in median run duration for healthy runs (retry only fires on failure).
• Guardrail: watch that retries don't mask genuine agent errors — retry scope must be limited to non-zero-exit before first model turn / gateway-unreachable conditions.

References

• §28997015233 — Avenger (1.54M tokens, then failed at Execute Claude Code CLI)
• §28999022875 — MCP Authentication Test (failed at Start MCP Gateway, 0 tokens)
• §28996995622 — Smoke CI (failed at Execute Copilot CLI, 0 tokens)

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

Generated by ⚡ Daily AgentRx Trace Optimizer · 372.8 AIC · ⌖ 39.6 AIC · ⊞ 6K · ◷

• expires on Jul 15, 2026, 11:05 PM UTC-08:00