Document and test Copilot token count accumulation behavior

[Copilot]

The Copilot log parser accumulates token counts from multiple API responses (one per turn), but this behavior was undocumented and untested.

Changes

• Added test (copilot_token_parsing_test.go) validating token accumulation:

  // Log with 2 responses: 1613 + 1712 tokens
  expectedTokens := 3325
  metrics := engine.ParseLogMetrics(logContent, false)
  // Verifies: metrics.TokenUsage == 3325

• Documented ParseLogMetrics with accumulation behavior:

  // Example: If a run has 3 turns with token counts [1000, 1500, 800],
  // the total token usage will be 3300 (sum of all turns).
  //
  // This matches the behavior of the JavaScript parser in parse_copilot_log.cjs.

• Added real-world test (copilot_token_parsing_real_test.go) validating parser with actual Copilot CLI logs:

  • Tests with real workflow run logs from GitHub Actions run 20680961415
  • Validates correct accumulation of 50 API responses totaling 2,484,450 tokens
  • Confirms parser handles production-scale logs accurately

This ensures consistency between Go and JavaScript parsers and prevents regression of the accumulation logic.

Original prompt

Reference: https://github.com/githubnext/gh-aw/actions/runs/20681216756/job/59375708286#step:6:1

Review token count parsing for copilot in go

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[pelikhan]

@copilot test logs parser with https://github.com/githubnext/gh-aw/actions/runs/20680961415/job/59375124033

[Copilot]

@copilot test logs parser with https://github.com/githubnext/gh-aw/actions/runs/20680961415/job/59375124033

Added test with the real Copilot log from that workflow run. The parser correctly accumulates all 50 API responses totaling 2,484,450 tokens. Test passes: commit 1a289b4