Skip to content

Replace engine.max-concurrency with engine.concurrency field for flexible agent job concurrency control#1305

Merged
pelikhan merged 10 commits intomainfrom
copilot/update-concurrency-field-engine
Oct 7, 2025
Merged

Replace engine.max-concurrency with engine.concurrency field for flexible agent job concurrency control#1305
pelikhan merged 10 commits intomainfrom
copilot/update-concurrency-field-engine

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Oct 7, 2025

Overview

Replaces the engine.max-concurrency field with a new engine.concurrency field that accepts standard GitHub Actions concurrency syntax. This provides more flexibility in controlling agent job concurrency while simplifying the default behavior.

Changes

Old Behavior (max-concurrency)

The previous implementation used slot-based distribution with a default of 3 concurrent jobs:

engine:
  id: claude
  max-concurrency: 5

This generated: gh-aw-claude-${{ github.run_id % 5 }} (distributing workflows across 5 slots)

New Behavior (concurrency)

The new implementation supports full GitHub Actions concurrency syntax with per-engine default concurrency control:

Default behavior (engine-specific):

Copilot has default concurrency enabled:

engine:
  id: copilot

Generates: gh-aw-copilot (single job per engine across all workflows)

Claude, Codex, and Custom engines do NOT have default concurrency:

engine:
  id: claude

Generates: No concurrency configuration (unlimited concurrent jobs)

String format:

engine:
  id: claude
  concurrency: "custom-${{ github.workflow }}"

Generates: concurrency: { group: "custom-${{ github.workflow }}" }

Object format (full control):

engine:
  id: claude
  concurrency:
    group: "custom-${{ github.ref }}"
    cancel-in-progress: true

Benefits

  1. Engine-Specific Defaults: Only Copilot has default concurrency enabled; Claude and Codex allow unlimited concurrency by default
  2. Standard GitHub Actions Syntax: Uses familiar concurrency patterns instead of custom slot-based distribution
  3. Greater Flexibility: Supports any GitHub Actions concurrency pattern (per-workflow, per-branch, custom expressions, etc.)
  4. Simpler Implementation: Removes complex modulo-based slot distribution logic
  5. Consistent Application: Concurrency configuration is applied to both agent and detection jobs

Migration Guide

Workflows using max-concurrency will have different behavior based on engine type:

Copilot workflows:

  • Before: gh-aw-copilot-${{ github.run_id % 3 }} (3 concurrent jobs via slot distribution)
  • After: gh-aw-copilot (single job per engine)

Claude/Codex workflows:

  • Before: gh-aw-claude-${{ github.run_id % 3 }} (3 concurrent jobs via slot distribution)
  • After: No concurrency (unlimited concurrent jobs)

To add concurrency control to Claude/Codex workflows:

engine:
  id: claude
  concurrency:
    group: "gh-aw-claude-${{ github.workflow }}"

Implementation Details

  • Removed MaxConcurrency and ConcurrencyGroup fields from EngineConfig struct
  • Added Concurrency field to store YAML-formatted concurrency configuration
  • Added HasDefaultConcurrency() method to CodingAgentEngine interface
  • Added hasDefaultConcurrency field to BaseEngine struct (Copilot: true, others: false)
  • Updated JSON schema to support both string and object formats
  • Updated GenerateJobConcurrencyConfig() to check engine flag before applying default concurrency
  • Updated buildThreatDetectionJob() to apply concurrency configuration to detection job
  • Recompiled all existing workflows with new per-engine default behavior

Testing

  • ✅ All existing unit tests updated and passing
  • ✅ New integration tests for concurrency extraction and compilation
  • ✅ Manual testing of all supported formats (default, string, object)
  • ✅ Verified concurrency applied to both agent and detection jobs
  • ✅ Verified per-engine default behavior (Copilot has default, Claude/Codex do not)
  • ✅ Documentation updated in both frontmatter.md and concurrency.md

Fixes #[issue-number]

Original prompt

Update "max-concurrency" under the "engine" front matter field to a "concurrency" field that allows to configure the concurrency of the agent job itself. The "concurrency" field has the same schema as the top level concurrency (github action compatible). The default is to limit agents to a single job accross all workflows. so group is gh-aw-<agentic engine id>


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

@pelikhan
Copy link
Copy Markdown
Collaborator

pelikhan commented Oct 7, 2025

@copilot make sure the concurrency section is also applied in the detection job

Copilot AI and others added 3 commits October 7, 2025 15:17
- Remove MaxConcurrency and ConcurrencyGroup fields from EngineConfig
- Add Concurrency field to EngineConfig to store YAML-formatted concurrency config
- Update ExtractEngineConfig to parse concurrency field (string or object format)
- Update GenerateJobConcurrencyConfig to use new field with default gh-aw-{engine-id}
- Update JSON schema to replace max-concurrency with concurrency field
- Update tests to reflect new concurrency behavior
- Add integration tests for concurrency field extraction and compilation

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
- Replace max-concurrency documentation with concurrency field
- Update frontmatter.md with new Agent Job Concurrency section
- Update concurrency.md with comprehensive concurrency documentation
- Remove references to slot-based distribution
- Add examples for string and object concurrency formats
- Document default behavior (single job per engine)
- Update troubleshooting sections

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
- Verified default concurrency pattern: gh-aw-{engine-id}
- Verified string format concurrency conversion
- Verified object format concurrency with cancel-in-progress
- All unit tests pass
- All integration tests pass
- Manual compilation testing successful

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Update max-concurrency to configurable concurrency field Replace engine.max-concurrency with engine.concurrency field for flexible agent job concurrency control Oct 7, 2025
Copilot AI requested a review from pelikhan October 7, 2025 15:31
- Add GenerateJobConcurrencyConfig call in buildThreatDetectionJob
- Detection job now uses same concurrency pattern as agent job
- Maintains consistency across all agentic jobs (agent and detection)

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Oct 7, 2025

@copilot make sure the concurrency section is also applied in the detection job

Done. The concurrency configuration is now applied to the detection job in commit 7c4a2ed. The detection job uses the same GenerateJobConcurrencyConfig() function as the agent job, ensuring consistent concurrency behavior across all agentic jobs.

Both custom and default concurrency patterns are now applied to detection jobs:

  • Default: gh-aw-{engine-id}
  • Custom: Uses the same pattern configured in engine.concurrency

@pelikhan
Copy link
Copy Markdown
Collaborator

pelikhan commented Oct 7, 2025

@copilot add a flag in the agentic engine configuration to enable this default concurrency mode. Claude/Codex should NOT have it enabled, Copilot should have it enabled by default.

- Add HasDefaultConcurrency() method to CodingAgentEngine interface
- Add hasDefaultConcurrency field to BaseEngine struct
- Copilot engine has default concurrency enabled (true)
- Claude, Codex, and Custom engines do NOT have default concurrency (false)
- Update GenerateJobConcurrencyConfig to check engine flag before applying default
- Update tests to reflect new behavior per engine
- Engines without default concurrency return empty string (no job concurrency)

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Oct 7, 2025

@copilot add a flag in the agentic engine configuration to enable this default concurrency mode. Claude/Codex should NOT have it enabled, Copilot should have it enabled by default.

Done. Added HasDefaultConcurrency() flag to engine configuration in commit 4f84b1e.

Default concurrency behavior per engine:

  • ✅ Copilot: Has default concurrency enabled → generates gh-aw-copilot
  • ❌ Claude: Does NOT have default concurrency → no job-level concurrency
  • ❌ Codex: Does NOT have default concurrency → no job-level concurrency
  • ❌ Custom: Does NOT have default concurrency → no job-level concurrency

All engines can still use custom concurrency via the engine.concurrency field.

…ty and error handling

- Simplified the parsing and validation of agent output in create_discussion.js and create_issue.js.
- Enhanced error messages for better debugging and user feedback.
- Streamlined the process of creating discussions and issues, ensuring consistent handling of labels and titles.
- Added checks for repository settings to ensure discussions and issues can be created.
- Improved summary output for staged mode to provide clearer previews of discussions and issues to be created.
@pelikhan pelikhan marked this pull request as ready for review October 7, 2025 16:32
@pelikhan pelikhan merged commit 6fb011d into main Oct 7, 2025
12 checks passed
@pelikhan pelikhan deleted the copilot/update-concurrency-field-engine branch October 7, 2025 16:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants