Skip to content

Enable multiple pipeline testing per task with name-based discovery and optimized pipeline fetching. #21122

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

raujaiswal
Copy link
Contributor

@raujaiswal raujaiswal commented Jun 30, 2025

Context

Enable multiple pipeline testing per task with name-based discovery.
https://mseng.visualstudio.com/AzureDevOps/_workitems/edit/2221293

Task Name

Enhanced the canary test system to support multiple pipelines per task through automatic pipeline discovery and optimized pipeline fetching.


Description

Summarize the changes made in this PR clearly and concisely.

Changes

1. Multi-Pipeline Support

  • Before: 1:1 mapping - one pipeline per task in canary test
  • After: 1:many mapping - automatically discovers and runs all pipelines whose names start with the task name

2. Performance Optimization

  • Before: Multiple fetchPipelines() calls per task
  • After: Single fetchPipelines() call at startup for efficiency

Example
For task WindowsMachineFileCopyV1, the system now automatically runs:

  • WindowsMachineFileCopyV1 (default)
  • WindowsMachineFileCopyV1_LegacyHost (legacy PowerShell)
  • WindowsMachineFileCopyV1_NewFeature (feature testing)

Risk Assessment (Low / Medium / High)

_Assess the level of risk and provide reasoning (e.g., scope, impact, backward compatibility). Low

Unit Tests Added or Updated (Yes / No)

Indicate whether unit tests were added or modified to reflect these changes.

Canary Test Schedule run

Additional Testing Performed

List all other tests performed (manual or automated, including integration, regression, scenario tests, etc.).


Documentation Changes Required (Yes / No)

Indicate whether related documentation needs to be updated.


Checklist

  • Related issue linked (if applicable)
  • Task version was bumped — see versioning guide
  • Verified the task behaves as expected

@raujaiswal raujaiswal requested review from tarunramsinghani and a team as code owners June 30, 2025 06:16
@raujaiswal raujaiswal changed the title Enable multi-pipeline testing for Azure DevOps tasks Enable multi-pipeline testing for Tasks in canary Test Jun 30, 2025
@raujaiswal raujaiswal changed the title Enable multi-pipeline testing for Tasks in canary Test Enable multiple pipeline testing per task with name-based discovery Jun 30, 2025
@raujaiswal raujaiswal changed the title Enable multiple pipeline testing per task with name-based discovery Enable multiple pipeline testing per task with name-based discovery and optimized pipeline fetching. Jun 30, 2025
@raujaiswal
Copy link
Contributor Author

/azp run

@raujaiswal raujaiswal requested a review from Copilot June 30, 2025 07:04
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR refactors the task testing system to support multiple pipelines per task using name-based discovery and optimizes performance by reducing redundant pipeline fetches.

  • Introduces automatic discovery of pipelines that share the task name.
  • Reduces multiple fetch calls to a single pipeline-fetch at startup.
Comments suppressed due to low confidence (1)

ci/ci-test-tasks/test-and-verify-v2/src/index.ts:123

  • [nitpick] The variable name 'allowParrallelRun' appears to have a typo; consider renaming it to 'allowParallelRun' for clarity.
  let allowParrallelRun = true;

} else {
runningTestBuilds.push(...runResult);
console.log(`⚠️ No pipelines found starting with "${task}"`);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

change this to warning and use the original mesage from below - console.log(Cannot build and run tests for task ${taskName} - corresponding test pipeline was not found);

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated.

@tarunramsinghani tarunramsinghani self-requested a review July 1, 2025 08:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants