Add max-concurrency option under engine config with agent concurrency global lock (default: 3, -1 to disable)

[Copilot]

Overview

Adds a new max-concurrency option under the engine configuration to control how many agentic jobs can run concurrently across all workflows and refs using that engine. This provides cost control, resource management, and engine isolation for AI-powered workflows through a global concurrency lock applied at the agent job level only.

Implementation

The feature uses GitHub Actions dual-level concurrency strategy with slot-based distribution and engine ID isolation:

---
on: push
engine:
  id: claude
  max-concurrency: 5  # Allow up to 5 concurrent workflows for this engine
tools:
  github:
    allowed: [list_issues]
---

This generates concurrency configuration at both workflow and agent levels with different strategies:

# At workflow level - Context-specific (original behavior, unchanged)
concurrency:
  group: "gh-aw-${{ github.workflow }}-${{ github.ref }}"

# At agent level - Global max-concurrency limiting
jobs:
  agent:
    runs-on: ubuntu-latest
    permissions: read-all
    concurrency:
      group: "gh-aw-claude-${{ github.run_id % 5 }}"
    steps:
      ...

How It Works

• Placement: Configured as a sibling of max-turns under the engine: section
• Default: 3 concurrent slots (when not specified or set to 0)
• Disabled: Set to -1 to disable agent concurrency (no global limiting)
• Distribution: Uses github.run_id % max-concurrency to assign each run to a slot (0 to max-concurrency-1)
• Workflow-Level Concurrency: Unchanged - Uses context-specific keys (issue numbers, PR numbers, refs) to allow different contexts to run concurrently
• Agent Concurrency: Uses only engine ID and slot number for global limiting across all workflows and refs
• Dual-Level Strategy: Workflow-level maintains context isolation, agent concurrency enforces global AI execution limits
• Standardized Naming: All concurrency groups use the gh-aw- prefix
• Engine Isolation: Different engines can run concurrently without interfering
• Repository-Wide: Limit applies across all workflows and refs for each engine at the agent level

Example Usage

# Restrict to sequential execution (1 at a time)
engine:
  id: copilot
  max-concurrency: 1

# Allow moderate parallelism (default)
engine:
  id: claude
  # max-concurrency not specified → defaults to 3

# Allow higher parallelism for busy repositories
engine:
  id: claude
  max-concurrency: 10

# Disable global limiting (unlimited concurrent executions)
engine:
  id: claude
  max-concurrency: -1

Benefits

✅ Global Cost Control: Prevents runaway costs across all workflows for an engine
✅ Repository-Wide Limiting: Single concurrency limit applies to all workflows and refs via agent concurrency lock
✅ Context Isolation Preserved: Workflow-level concurrency unchanged, allowing different issues/PRs/branches to run concurrently
✅ Dual-Level Strategy: Workflow-level for context isolation, agent concurrency for global AI execution limiting
✅ Standardized Naming: gh-aw- prefix for all concurrency groups ensures consistent identification
✅ Simple and Predictable: Agent concurrency uses only engine ID and slot for global control
✅ Engine Isolation: Different engines can run concurrently without interfering
✅ Per-Engine Configuration: Each engine can have its own concurrency limits
✅ Flexible: Easy to adjust based on repository needs and budget, including ability to disable with -1

Changes Made

• Added MaxConcurrency field to EngineConfig struct
• Added Concurrency field to Job struct for agent concurrency support
• Updated ExtractEngineConfig() to parse max-concurrency from engine configuration
• Fixed default engine handling to ensure EngineConfig always has engine ID populated
• Created GenerateJobConcurrencyConfig(): New function for agent concurrency with global lock pattern and gh-aw- prefix
• Restored GenerateConcurrencyConfig(): Maintains original workflow-level concurrency behavior (context-specific keys)
• Updated buildMainJob() to set agent concurrency on the agentic job using GenerateJobConcurrencyConfig()
• Added support for max-concurrency: -1: Setting to -1 completely disables agent concurrency (no concurrency config emitted at agent level)
• Updated job rendering to include concurrency configuration at agent level (when not disabled)
• Workflow-level concurrency preserved with original context-specific behavior
• Updated JSON schema with max-concurrency under engine configuration (minimum: -1)
• Added comprehensive documentation and examples in frontmatter.md and instructions.md
• Added dedicated concurrency control documentation page at docs/src/content/docs/reference/concurrency.md with complete coverage of dual-level strategy, configuration, slot distribution, global locks, engine isolation, disable option, best practices, and troubleshooting
• Updated terminology: Renamed "job-level concurrency" to "agent concurrency" throughout documentation for clarity
• Updated frontmatter.md to reference the new dedicated concurrency documentation page
• Split test suites: TestGenerateConcurrencyConfig() tests workflow-level (context-specific), TestGenerateJobConcurrencyConfig() tests agent concurrency (max-concurrency)
• Added new tests for custom max-concurrency values, engine isolation, and -1 disable behavior
• Recompiled all 59 workflow files with dual-level concurrency strategy
• Merged latest changes from main branch
• Formatted all Go and JavaScript code

Documentation

A comprehensive dedicated documentation page has been added at /docs/src/content/docs/reference/concurrency.md covering:

• Overview of dual-level concurrency strategy (workflow context-specific + agent global limiting)
• Max-concurrency configuration, defaults, and examples including -1 to disable
• How concurrency groups are generated and slot distribution works
• Distinction between workflow-level (context-specific) and agent concurrency (global lock) strategies
• Engine isolation and how different engines work independently
• Cancellation behavior by trigger type
• Best practices for setting and monitoring limits
• Troubleshooting common issues (queuing, too many runs, cancellation problems)
• Cross-references to related documentation

Testing

All unit tests pass, including:

• 17 concurrency tests split between workflow-level and agent concurrency functions
• New tests for custom max-concurrency values and engine isolation
• Test for -1 disable behavior (validates empty string returned when disabled)
• Workflow-level tests validate original context-specific behavior
• Agent concurrency tests validate global max-concurrency limiting
• Manual validation with test workflows

Fixes the requirement to support max-concurrency control with global engine-level limiting at the agent level while preserving original workflow-level concurrency behavior, with ability to disable via -1.

Original prompt

Update compiler to support a max-concurrency option for the agentic job - accross all workflows (use github actions concurrency controls). Default is 3.

---

💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.

[pelikhan]

@copilot

• include engine id in generated key
• place field at sibling of "max-runs" under "engine"

[Copilot]

@copilot

• include engine id in generated key
• place field at sibling of "max-runs" under "engine"

Done! Changes made in commit 65abddd:

1. ✅ Moved max-concurrency under engine config - Now a sibling of max-turns
2. ✅ Included engine ID in concurrency group - Provides isolation between different engines

New Structure:

engine:
  id: claude
  max-concurrency: 5

Generated Concurrency:

concurrency:
  group: "gh-aw-${{ github.workflow }}-${{ github.event.issue.number }}-claude-${{ github.run_id % 5 }}"

The engine ID (claude, copilot, codex) is now part of the concurrency group, allowing different engines to run concurrently while respecting per-engine limits.

[pelikhan]

@copilot the key can only use runid and engine id as it is a global lock accross all workflows and refs for the agent.

[pelikhan]

@copilot this engine-specific concurrency element should be added to the agentic job

[Copilot]

@copilot the key can only use runid and engine id as it is a global lock accross all workflows and refs for the agent.

Fixed in commit 47a2e8a. The concurrency group now uses only engine ID and run_id slot for a true global lock:

concurrency:
  group: "{engine-id}-${{ github.run_id % max-concurrency }}"

Example output:

concurrency:
  group: "claude-${{ github.run_id % 5 }}"

This creates a global lock across all workflows and refs for each engine, as requested.

[Copilot]

@copilot this engine-specific concurrency element should be added to the agentic job

Done in commit [hash]. The concurrency configuration is now added to the agentic job instead of the workflow level.

Before (workflow level):

name: "Workflow"
concurrency:
  group: "claude-${{ github.run_id % 5 }}"

jobs:
  agent:
    ...

After (job level):

name: "Workflow"

jobs:
  agent:
    runs-on: ubuntu-latest
    concurrency:
      group: "claude-${{ github.run_id % 5 }}"
    steps:
      ...

[pelikhan]

@copilot recompile

[pelikhan]

@copilot prepend with gh-aw-

[Copilot]

Done in commit [hash]. All concurrency groups now have the "gh-aw-" prefix.

Before:

concurrency:
  group: "copilot-${{ github.run_id % 3 }}"

After:

concurrency:
  group: "gh-aw-copilot-${{ github.run_id % 3 }}"

[pelikhan]

@copilot use max-concurrency: -1 to disable emission of concurrency at the agentic job level

[pelikhan]

@copilot set to -1 to remove maximum

[Copilot]

Done in commit fe60ca7. Documentation updated to include -1 option:

• Added "Disabled: -1 (no agent concurrency limiting)" to default values
• Added example showing max-concurrency: -1
• Added explanation in Global Lock Behavior section

[pelikhan]

@copilot rename "job-level" to "agent concurrency"

[Copilot]

Done in commit fe60ca7. Renamed "job-level" to "agent concurrency" throughout the documentation for better clarity.

[Copilot]

@copilot merge main, format, lint, recompile

Done in commit 787be54. Main branch merged, code formatted, and all 59 workflows recompiled successfully.

[Copilot]

@copilot use max-concurrency: -1 to disable emission of concurrency at the agentic job level

Done in commit fe60ca7. Setting max-concurrency: -1 now disables agent concurrency emission entirely. When set to -1, no concurrency configuration is added to the agent job level.

[pelikhan]

@copilot merge main, format, lint, recompile

[pelikhan]

@copilot add concurrency-group field under engine to override the use of the engine id in the generation of the agent job concurrency group field.

[pelikhan]

@copilot format,lint,fix tests