Use AWF audit JSONL as source for effective token failure parsing by Copilot · Pull Request #31104 · github/gh-aw

Copilot · 2026-05-08T23:58:04Z

Bug Fix

What was the bug?

Failure handling relied on raw/env outputs for effective-token failure context. While max-effective-tokens was being moved to AWF firewall audit records, effective_tokens and ET rate-limit error signaling could still drift from the canonical runtime values in audit JSONL.

How did you fix it?

Audit log as source of truth
- Added parser logic to extract max effective token budget from AWF firewall audit JSONL entries.
- Supports both schema field shapes: max_effective_tokens and maxEffectiveTokens (including nested objects).
- Added parsing for effective-token error metadata from audit JSONL, including:
  - effective_tokens / effectiveTokens
  - effective_tokens_rate_limit_error / effectiveTokensRateLimitError
  - text-based ET rate-limit indicators in common error fields.
Effective-token logic extracted to dedicated module
- Moved effective-token parsing/computation into actions/setup/js/effective_tokens_context.cjs.
- Updated handle_agent_failure.cjs to consume the new module while preserving behavior and existing fallbacks.
Audit file resolution and compatibility
- Added audit path resolution that prefers sandbox/firewall/audit/log.jsonl.
- Preserves backward compatibility by falling back to sandbox/firewall/audit/audit.jsonl.
Failure context wiring
- Updated failure-path budget resolution to use parsed audit value first, then fall back to GH_AW_MAX_EFFECTIVE_TOKENS when audit data is unavailable.
- Updated effective-token failure context to prefer parsed audit effective_tokens and ET rate-limit signals, with existing env vars as fallback.
- Reused the same audit path resolver in auth-error parsing to keep behavior consistent.
Coverage updates
- Added focused unit tests for missing-file behavior, nested/camelCase extraction, log.jsonl preference over audit.jsonl, and effective-token error info parsing from JSONL.
Branch maintenance from feedback
- Merged latest main into this PR branch and recompiled workflows to refresh generated lock artifacts while keeping the original fix unchanged.

Testing

Ran targeted JS tests for failure handling (handle_agent_failure.test.cjs).
Ran targeted Go tests for touched packages (./pkg/cli/..., ./pkg/workflow/..., ./pkg/parser/...).

Example

const parsedEffectiveTokensErrorInfo = parseEffectiveTokensErrorInfoFromAuditLog();
const effectiveTokens =
  parsedEffectiveTokensErrorInfo.effectiveTokens ||
  process.env.GH_AW_EFFECTIVE_TOKENS ||
  "";

const maxEffectiveTokens =
  parseMaxEffectiveTokensFromAuditLog() ||
  process.env.GH_AW_MAX_EFFECTIVE_TOKENS ||
  "";

const effectiveTokensRateLimitError =
  parsedEffectiveTokensErrorInfo.rateLimitError ||
  process.env.GH_AW_EFFECTIVE_TOKENS_RATE_LIMIT_ERROR === "true";

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot

Pull request overview

Fixes max-effective-tokens reporting in failure handling by using AWF firewall audit JSONL records as the canonical source of the max effective token budget (with env var fallback), and updates compiled workflows to propagate effective-token rate-limit context.

Changes:

Added audit-log parsing in handle_agent_failure.cjs to extract max effective tokens (supports max_effective_tokens and maxEffectiveTokens, including nested shapes) and prefer sandbox/firewall/audit/log.jsonl over audit.jsonl.
Wired failure-path budget resolution to prefer the parsed audit value, and reused the same audit-log path resolver for firewall auth error parsing.
Updated workflow lock files to surface effective_tokens_rate_limit_error outputs and configure/export max effective tokens consistently; added unit tests for audit parsing and path preference.

Show a summary per file

File	Description
actions/setup/js/handle_agent_failure.cjs	Adds audit-log path resolution + max-effective-tokens parsing and uses it in failure reporting.
actions/setup/js/handle_agent_failure.test.cjs	Adds unit coverage for audit parsing (missing file, nested/camelCase extraction, log.jsonl preference).
.github/workflows/video-analyzer.lock.yml	Propagates effective token rate-limit output and configures maxEffectiveTokens / GH_AW_MAX_EFFECTIVE_TOKENS.
.github/workflows/test-workflow.lock.yml	Propagates effective token rate-limit output and configures maxEffectiveTokens / GH_AW_MAX_EFFECTIVE_TOKENS.
.github/workflows/test-project-url-default.lock.yml	Propagates effective token rate-limit output and configures maxEffectiveTokens / GH_AW_MAX_EFFECTIVE_TOKENS.
.github/workflows/test-dispatcher.lock.yml	Propagates effective token rate-limit output and configures maxEffectiveTokens / GH_AW_MAX_EFFECTIVE_TOKENS.
.github/workflows/smoke-opencode.lock.yml	Propagates effective token rate-limit output and configures maxEffectiveTokens / GH_AW_MAX_EFFECTIVE_TOKENS.
.github/workflows/smoke-gemini.lock.yml	Propagates effective token rate-limit output and configures maxEffectiveTokens / GH_AW_MAX_EFFECTIVE_TOKENS.
.github/workflows/smoke-crush.lock.yml	Propagates effective token rate-limit output and configures maxEffectiveTokens / GH_AW_MAX_EFFECTIVE_TOKENS.
.github/workflows/smoke-ci.lock.yml	Propagates effective token rate-limit output and configures maxEffectiveTokens / GH_AW_MAX_EFFECTIVE_TOKENS.
.github/workflows/release.lock.yml	Propagates effective token rate-limit output and configures maxEffectiveTokens / GH_AW_MAX_EFFECTIVE_TOKENS.
.github/workflows/poem-bot.lock.yml	Propagates effective token rate-limit output and configures maxEffectiveTokens / GH_AW_MAX_EFFECTIVE_TOKENS.
.github/workflows/jsweep.lock.yml	Propagates effective token rate-limit output and configures maxEffectiveTokens / GH_AW_MAX_EFFECTIVE_TOKENS.
.github/workflows/issue-triage-agent.lock.yml	Propagates effective token rate-limit output and configures maxEffectiveTokens / GH_AW_MAX_EFFECTIVE_TOKENS.
.github/workflows/hippo-embed.lock.yml	Propagates effective token rate-limit output and configures maxEffectiveTokens / GH_AW_MAX_EFFECTIVE_TOKENS.
.github/workflows/gpclean.lock.yml	Propagates effective token rate-limit output and configures maxEffectiveTokens / GH_AW_MAX_EFFECTIVE_TOKENS.
.github/workflows/firewall.lock.yml	Propagates effective token rate-limit output and configures maxEffectiveTokens / GH_AW_MAX_EFFECTIVE_TOKENS.
.github/workflows/example-permissions-warning.lock.yml	Propagates effective token rate-limit output and configures maxEffectiveTokens / GH_AW_MAX_EFFECTIVE_TOKENS.
.github/workflows/dev.lock.yml	Propagates effective token rate-limit output and configures maxEffectiveTokens / GH_AW_MAX_EFFECTIVE_TOKENS.
.github/workflows/daily-malicious-code-scan.lock.yml	Propagates effective token rate-limit output and configures maxEffectiveTokens / GH_AW_MAX_EFFECTIVE_TOKENS.
.github/workflows/copilot-token-optimizer.lock.yml	Propagates effective token rate-limit output and configures maxEffectiveTokens / GH_AW_MAX_EFFECTIVE_TOKENS.
.github/workflows/codex-github-remote-mcp-test.lock.yml	Propagates effective token rate-limit output and configures maxEffectiveTokens / GH_AW_MAX_EFFECTIVE_TOKENS.
.github/workflows/code-simplifier.lock.yml	Propagates effective token rate-limit output and configures maxEffectiveTokens / GH_AW_MAX_EFFECTIVE_TOKENS.
.github/workflows/changeset.lock.yml	Propagates effective token rate-limit output and configures maxEffectiveTokens / GH_AW_MAX_EFFECTIVE_TOKENS.
.github/workflows/brave.lock.yml	Propagates effective token rate-limit output and configures maxEffectiveTokens / GH_AW_MAX_EFFECTIVE_TOKENS.
.github/workflows/bot-detection.lock.yml	Propagates effective token rate-limit output and configures maxEffectiveTokens / GH_AW_MAX_EFFECTIVE_TOKENS.
.github/workflows/ai-moderator.lock.yml	Propagates effective token rate-limit output and configures maxEffectiveTokens / GH_AW_MAX_EFFECTIVE_TOKENS.
.github/workflows/ace-editor.lock.yml	Propagates effective token rate-limit output and configures maxEffectiveTokens / GH_AW_MAX_EFFECTIVE_TOKENS.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Files reviewed: 70/219 changed files
Comments generated: 1

 function parseFirewallAuthErrors(auditJsonlPath) {
  try {
-    if (!fs.existsSync(auditJsonlPath)) {
+    const resolvedPath = resolveFirewallAuditLogPath(auditJsonlPath);
+    if (!fs.existsSync(resolvedPath)) {
      return [];


pelikhan · 2026-05-09T00:20:07Z

        with:
          script: |
            try {
              const { setupGlobals } = require('${{ runner.temp }}/gh-aw/actions/setup_globals.cjs');


@copilot move to .cjs file in actions/setup/js + parse the effective token error info from JSONL as well

Implemented in 515fd17. Effective-token error info is now parsed directly in actions/setup/js/handle_agent_failure.cjs from AWF firewall audit JSONL (including nested snake_case/camelCase fields and text-based ET rate-limit signals), with existing env vars kept as fallback. Added targeted tests in handle_agent_failure.test.cjs.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

pelikhan · 2026-05-09T00:39:01Z

@copilot move effective token computation in own JavaScript file

pelikhan · 2026-05-09T00:49:41Z

@copilot merge main and recompile

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot · 2026-05-09T00:53:14Z

@copilot move effective token computation in own JavaScript file

Done in 5205148. Effective-token parsing and failure-state computation are now moved into actions/setup/js/effective_tokens_context.cjs, and handle_agent_failure.cjs consumes that module.

pelikhan · 2026-05-09T00:57:01Z

@copilot merge main and recompile

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot · 2026-05-09T01:05:24Z

@copilot merge main and recompile

Done in 3643611. I merged main into this branch and ran make recompile, then validated with targeted JS and Go tests.

Copilot AI and others added 2 commits May 8, 2026 23:56

Plan max effective tokens parsing update

3aed2aa

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Parse max effective tokens from firewall audit log

62ff1b7

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI assigned Copilot and pelikhan May 8, 2026

Copilot AI changed the title ~~Parse max effective tokens from AWF audit log JSONL~~ Use AWF audit JSONL as source for max effective tokens parsing May 8, 2026

Copilot finished work on behalf of pelikhan May 8, 2026 23:58

Copilot created this pull request from a session on behalf of pelikhan May 8, 2026 23:58 View session

Copilot AI requested a review from pelikhan May 8, 2026 23:58

pelikhan marked this pull request as ready for review May 9, 2026 00:03

Copilot AI review requested due to automatic review settings May 9, 2026 00:03

Copilot started reviewing on behalf of pelikhan May 9, 2026 00:04 View session

Copilot AI reviewed May 9, 2026

View reviewed changes

pelikhan reviewed May 9, 2026

View reviewed changes

Copilot started work on behalf of pelikhan May 9, 2026 00:20 View session

Copilot AI and others added 4 commits May 9, 2026 00:27

Parse effective token error info from firewall audit JSONL

7ea7701

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Refine audit parser constants and precedence notes

a6da3e2

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Document effective-token rate-limit pattern matching

ad44ee8

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Harden effective-token rate-limit signal parsing

515fd17

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI changed the title ~~Use AWF audit JSONL as source for max effective tokens parsing~~ Use AWF audit JSONL as source for effective token failure parsing May 9, 2026

Copilot finished work on behalf of pelikhan May 9, 2026 00:31

Copilot AI requested a review from pelikhan May 9, 2026 00:31

Copilot started work on behalf of pelikhan May 9, 2026 00:39 View session

Copilot AI and others added 2 commits May 9, 2026 00:51

Extract effective token failure computation into dedicated module

09b1a7e

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Extract effective token parsing into dedicated helper module

5205148

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot finished work on behalf of pelikhan May 9, 2026 00:53

Copilot started work on behalf of pelikhan May 9, 2026 00:53 View session

Merge origin/main and recompile workflows

3643611

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot finished work on behalf of pelikhan May 9, 2026 01:06

Copilot started work on behalf of pelikhan May 9, 2026 01:06 View session

pelikhan merged commit b527231 into main May 9, 2026
1 check failed

pelikhan deleted the copilot/update-max-effective-tokens-parsing branch May 9, 2026 01:09

Copilot stopped work on behalf of pelikhan due to an error May 9, 2026 01:09
The session was cancelled by the user.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use AWF audit JSONL as source for effective token failure parsing#31104

Use AWF audit JSONL as source for effective token failure parsing#31104
pelikhan merged 9 commits into
mainfrom
copilot/update-max-effective-tokens-parsing

Copilot AI commented May 8, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

pelikhan May 9, 2026

Uh oh!

Copilot AI May 9, 2026

Uh oh!

pelikhan commented May 9, 2026

Uh oh!

pelikhan commented May 9, 2026

Uh oh!

Copilot AI commented May 9, 2026

Uh oh!

pelikhan commented May 9, 2026

Uh oh!

Copilot AI commented May 9, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

Copilot AI commented May 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Bug Fix

What was the bug?

How did you fix it?

Testing

Example

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Copilot's findings

Uh oh!

pelikhan May 9, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI May 9, 2026

Choose a reason for hiding this comment

Uh oh!

pelikhan commented May 9, 2026

Uh oh!

pelikhan commented May 9, 2026

Uh oh!

Copilot AI commented May 9, 2026

Uh oh!

pelikhan commented May 9, 2026

Uh oh!

Copilot AI commented May 9, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Copilot AI commented May 8, 2026 •

edited

Loading