Skip to content

[aw-failures] [P1] Fix Copilot billing 403 masking Daily Max Ai Credits Test — 5+ consecutive daily failures #51589

Description

@github-actions

Problem

Fix the Copilot-requests proxy auth check before it burns another day of test signal — every scheduled run of Daily Max Ai Credits Test has failed on an authentication_failed HTTP 403 from the gh-aw Copilot API proxy for at least the last 5 consecutive days, never once reaching the max-ai-credits: 1 guardrail logic the workflow exists to validate.

Affected workflows and runs

  • Daily Max Ai Credits Test (.github/workflows/daily-max-ai-credits-test.lock.yml) — 100% failure, same signature, on every run checked:

Probable root cause

Every attempt dies the same way, before any credit-limit logic can engage:

Authentication failed with provider at (172.30.0.30/redacted) (HTTP 403).
  Check your COPILOT_PROVIDER_API_KEY or COPILOT_PROVIDER_BEARER_TOKEN.
[copilot-harness] attempt N failed: exitCode=1 failureClass=authentication_failed isAuthenticationFailedError=true ...
[copilot-harness] ...: Copilot requests authentication failed through the gh-aw API proxy (HTTP 403, model=claude-sonnet-5). This workflow is using permissions.copilot-requests: write, so Copilot requests must be allowed through your organization's centralized Copilot billing configuration. Verify that copilot-requests: write is granted to the workflow or job and that Copilot org billing is enabled for your organization.

This is not the workflow's intended ai_credits_rate_limit_error / max_ai_credits_exceeded outcome (which is explicitly excluded from report-failure-as-issue because it's the expected "success" case for this test). It is a distinct authentication_failed billing/policy rejection, so this bug has been silently invalidating the test's actual purpose since at least 2026-08-05.

audit-diff between today's run and the prior day's run (§31254595629 vs §31310376315) shows zero drift in firewall/network behavior — same signature, same failure point, confirming this is a stable, ongoing condition rather than a one-off blip.

Not a duplicate of #51033 (that signature is No model available from Task-tool subagent delegation — a different failureClass entirely) or #51547 (sandbox file-access lockout — unrelated).

Proposed remediation

  1. Verify Copilot org billing / copilot-requests: write centralized policy grants cover this workflow's job — this looks like an entitlement gap specific to this workflow's permission or token configuration, not a code bug in gh-aw itself.
  2. If the entitlement is confirmed correct, escalate to the Copilot API proxy team with the reproducing run IDs above — the 403 fires on the very first request of every run, 5 days running.
  3. Regardless of root cause, make the harness classifier retry authentication_failed distinctly from partial_execution — right now it burns a 5s sleep + full second attempt before giving up, wasting ~2x the run's actual work time on a condition it already knows won't recover mid-run.

Success criteria

  • Next 3 scheduled runs of Daily Max Ai Credits Test reach the max-ai-credits cutoff logic (i.e., fail via isCAPIQuotaExceededError/isInvocationCapExceeded/credit-limit signature, not authentication_failed).
  • Zero authentication_failed / HTTP 403 entries from this workflow across the next 3 days.

Related to #51545
Related to #51545

Generated by 🔍 [aw] Failure Investigator (6h) · agent · 101.8 AIC · ⌖ 47.5 AIC · ⊞ 5.3K ·

  • expires on Aug 16, 2026, 5:18 AM UTC-08:00

Metadata

Metadata

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions