Long running pipeline #4092

sivakami-projects · 2025-10-21T16:59:32Z

Notes:

Moved to #4098

Copilot

Pull Request Overview

This PR introduces a new Azure DevOps pipeline for running long-term tests on AKS Swift v2 clusters. The pipeline automates the creation and configuration of AKS clusters along with the necessary networking infrastructure.

Key Changes:

Creates a parameterized pipeline template that orchestrates AKS cluster creation and networking setup
Implements bash scripts to provision two AKS clusters, virtual networks, network peerings, storage accounts, and network security groups
Configures parallel execution for AKS cluster creation to optimize deployment time

Reviewed Changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
`.pipelines/swiftv2-long-running/pipeline.yaml`	Main pipeline definition with configurable parameters for subscription, location, resource groups, and VM SKUs
`.pipelines/swiftv2-long-running/template/long-running-pipeline-template.yaml`	Pipeline template defining five sequential jobs for infrastructure provisioning
`.pipelines/swiftv2-long-running/scripts/create_aks.sh`	Script to create two AKS clusters in parallel with default and high-NIC node pools
`.pipelines/swiftv2-long-running/scripts/create_vnets.sh`	Script to provision four virtual networks with multiple subnets
`.pipelines/swiftv2-long-running/scripts/create_peerings.sh`	Script to establish VNet peerings between virtual networks
`.pipelines/swiftv2-long-running/scripts/create_storage.sh`	Script to create two storage accounts with secure configuration
`.pipelines/swiftv2-long-running/scripts/create_nsg.sh`	Script to configure network security group with SSH, VNet, and AKS control plane rules

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

.pipelines/swiftv2-long-running/scripts/create_peerings.sh

.pipelines/swiftv2-long-running/scripts/create_nsg.sh

.pipelines/swiftv2-long-running/scripts/create_aks.sh

.pipelines/swiftv2-long-running/template/long-running-pipeline-template.yaml

.pipelines/swiftv2-long-running/scripts/create_storage.sh

prmathur-microsoft · 2025-10-21T21:48:47Z

.pipelines/swiftv2-long-running/scripts/create_vnets.sh

+VNET_A3="cx_vnet_a3"
+VNET_B1="cx_vnet_b1"
+
+A1_S1="10.10.1.0/24"


Why are these hardcoded.

I think they would need to be anyways, what's the alternative?

pjohnst5

Commenting on a few things

.pipelines/swiftv2-long-running/scripts/create_aks.sh

pjohnst5 · 2025-10-22T16:36:26Z

.pipelines/swiftv2-long-running/scripts/create_vnets.sh

+VNET_A3="cx_vnet_a3"
+VNET_B1="cx_vnet_b1"
+
+A1_S1="10.10.1.0/24"


I think they would need to be anyways, what's the alternative?

.pipelines/swiftv2-long-running/scripts/create_vnets.sh

.pipelines/swiftv2-long-running/scripts/create_peerings.sh

.pipelines/swiftv2-long-running/scripts/create_storage.sh

.pipelines/swiftv2-long-running/scripts/create_nsg.sh

#3421) * cover adding iptables rules for dns in vnet scale cilium case * replace existing iptables rules * modify imds iptables rule * mock iptables in cns and add ut * address linter issues * address linter issue

fix check-shadowing not allowed

Bumps [golang.org/x/time](https://github.com/golang/time) from 0.9.0 to 0.10.0. - [Commits](golang/time@v0.9.0...v0.10.0) --- updated-dependencies: - dependency-name: golang.org/x/time dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

feat: add apiserver FQDN to CNS log metadata in AKS Signed-off-by: Evan Baker <rbtr@users.noreply.github.com>

Bumps [golang.org/x/net](https://github.com/golang/net) from 0.34.0 to 0.35.0. - [Commits](golang/net@v0.34.0...v0.35.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* split dualstack e2e tests by os split dualstack remove windows to debug remove dup remove unknown stage remove windows from cluster name add windows vm size add windows and linux specific parameters make hybrid windows test on windows only dualstack * split overlay e2e tests by os fix win cluster overlay always being linux only * pick: increase load test (control plane scale) timeout * remove if structure since win nodepool should never start off existing on cluster create * add aliases for new cluster types for compatability * address feedback

Bumps [gotest.tools/v3](https://github.com/gotestyourself/gotest.tools) from 3.5.1 to 3.5.2. - [Release notes](https://github.com/gotestyourself/gotest.tools/releases) - [Commits](gotestyourself/gotest.tools@v3.5.1...v3.5.2) --- updated-dependencies: - dependency-name: gotest.tools/v3 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps google.golang.org/protobuf from 1.36.3 to 1.36.4. --- updated-dependencies: - dependency-name: google.golang.org/protobuf dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [github.com/prometheus/common](https://github.com/prometheus/common) from 0.61.0 to 0.62.0. - [Release notes](https://github.com/prometheus/common/releases) - [Changelog](https://github.com/prometheus/common/blob/main/RELEASE.md) - [Commits](prometheus/common@v0.61.0...v0.62.0) --- updated-dependencies: - dependency-name: github.com/prometheus/common dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

…1 to 1.8.2 (#3418) deps: bump github.com/Azure/azure-sdk-for-go/sdk/azidentity Bumps [github.com/Azure/azure-sdk-for-go/sdk/azidentity](https://github.com/Azure/azure-sdk-for-go) from 1.8.1 to 1.8.2. - [Release notes](https://github.com/Azure/azure-sdk-for-go/releases) - [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/release.md) - [Commits](Azure/azure-sdk-for-go@sdk/azidentity/v1.8.1...sdk/azidentity/v1.8.2) --- updated-dependencies: - dependency-name: github.com/Azure/azure-sdk-for-go/sdk/azidentity dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

fix: zapetw core with fields Signed-off-by: Evan Baker <rbtr@users.noreply.github.com>

…3435) Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.20.5 to 1.21.0. - [Release notes](https://github.com/prometheus/client_golang/releases) - [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md) - [Commits](prometheus/client_golang@v1.20.5...v1.21.0) --- updated-dependencies: - dependency-name: github.com/prometheus/client_golang dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

…/tools (#3366) deps: bump google.golang.org/protobuf in /build/tools Bumps google.golang.org/protobuf from 1.36.3 to 1.36.4. --- updated-dependencies: - dependency-name: google.golang.org/protobuf dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.69.2 to 1.70.0. - [Release notes](https://github.com/grpc/grpc-go/releases) - [Commits](grpc/grpc-go@v1.69.2...v1.70.0) --- updated-dependencies: - dependency-name: google.golang.org/grpc dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* cleanup makefile * remove windows user and password from cniv1 cluster * fix windows variable not propagating

ci: add k8s yaml depreciation check to nightly

* set appArmorProfile and remove annotations for cilium 1.16 * set dualstack appArmorProfile * set profile per container * update nightly daemonset

* sort set policy values before validation * address linter issues * sort expected values * address feedback

* ci(cyclonus extended): [NPM] try to fix by bumping kind Signed-off-by: Hunter Gregory <42728408+huntergregory@users.noreply.github.com> * ci: temporarily add push and pull_request triggers to workflow * ci: use the kind setup step in the regular cyclonus gh action * fix: missing cluster_name * ci: revert PR triggers --------- Signed-off-by: Hunter Gregory <42728408+huntergregory@users.noreply.github.com>

* chore: add cilium 1.17 and chart updates for k8s 1.32 * fix: rename config path * fix: make json path * swap kpr to strict in hubble chart

* add status to mtpnc crd * update mtpnc status * update mtpnc status only * fix lint formating errs * add crd manifest * update mtpnc statuses * fix status enum * address pr feedback * fix lint * add missing file

* added logic to fix cns bug for overlay subnet expansion * reverted a line change * fixed spelling * added unit test * fixing go lint * expanded on a comment * updated logic * updated test * updated validate superset logic * updated to return bool instead of error for checking cidr superset * updated logic to check for containment --------- Co-authored-by: Riya <jainriya@microsoft.com> Co-authored-by: Riya <rejain6@gmail.com>

chore: fix cns vulneribility issues Co-authored-by: paulyu <paulyu@microsoft.com>

downgrade easyjson GO package Co-authored-by: paulyu <paulyu@microsoft.com>

* feat: add SkipDefaultRoutes field to network container request and responses * feat: implement ARP proxy setting and custom route addition for VLAN interfaces * feat: enable dual NIC feature support and improve ARP proxy handling in transparent VLAN client * feat: add tests for SkipDefaultRoutes handling in network container requests * feat: remove addition of custom routes * fix: improve ARP proxy error handling * fix: Lint Errors * refactor: streamline ARP proxy setup in AddEndpointRules * fix: update comments for dual NIC support and clarify ARP proxy function --------- Co-authored-by: Mugesh SP <mugeshsp@microsoft.com>

* Adding delete timestamp check * Adding UT and refactoring getmtpnc * Update cns/middlewares/k8sSwiftV2_linux_test.go Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: shreyashastantram <105284415+shreyashastantram@users.noreply.github.com> * Changing the MTPNC error message * Removing unused fields * Revert manifest changes for CRD files * Fixing error message and adding comments * Revert CRD API files to master state * fix: correcting comment assertion --------- Signed-off-by: shreyashastantram <105284415+shreyashastantram@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* update go security dependencies * update retry dependency --------- Co-authored-by: paulyu <paulyu@microsoft.com>

ebtables/ebtables.go

network/network_linux.go

network/networkutils/networkutils_linux.go

ovsctl/ovsctl.go

network/endpoint_windows.go

+func getPnPDeviceID(instanceID string, plc platform.ExecClient) (string, error) {
+	// get device locationPath
+	getLocationPath := fmt.Sprintf("(Get-PnpDeviceProperty -KeyName DEVPKEY_Device_LocationPaths –InstanceId \"%s\").Data[0]", instanceID) //nolint
+	locationPath, err := plc.ExecutePowershellCommand(getLocationPath)


network/endpoint_windows.go

+func disableVFDevice(instanceID string, plc platform.ExecClient) error {
+	// disable device
+	disableVFDevice := fmt.Sprintf("Disable-PnpDevice -InstanceId \"%s\" -confirm:$false", instanceID) //nolint
+	_, err := plc.ExecutePowershellCommand(disableVFDevice)


network/endpoint_windows.go

+func getLocationPath(instanceID string, plc platform.ExecClient) (string, error) {
+	// get device locationPath
+	getLocationPath := fmt.Sprintf("(Get-PnpDeviceProperty -KeyName DEVPKEY_Device_LocationPaths –InstanceId \"%s\").Data[0]", instanceID) //nolint
+	locationPath, err := plc.ExecutePowershellCommand(getLocationPath)


network/endpoint_windows.go

+func getPnpDeviceState(instanceID string, plc platform.ExecClient) (string, string, error) { //nolint
+	// get if device is present
+	getDeviceIsPresent := fmt.Sprintf("(Get-PnpDeviceProperty -InstanceId \"%s\" | Where-Object KeyName -eq DEVPKEY_Device_IsPresent).Data[0]", instanceID) //nolint
+	devpkeyDeviceIsPresent, err := plc.ExecutePowershellCommand(getDeviceIsPresent)


network/endpoint_windows.go

+
+	// DEVPKEY_Device_ProblemCode is not there once device is disabled and dismounted, so need to check if DEVPKEY_Device_ProblemCode exists first
+	getDeviceProblemCodeExist := fmt.Sprintf("(Get-PnpDeviceProperty -InstanceId \"%s\" | Where-Object KeyName -eq DEVPKEY_Device_ProblemCode)", instanceID) //nolint
+	devpkeyDeviceProblemCodeExist, err := plc.ExecutePowershellCommand(getDeviceProblemCodeExist)


network/endpoint_windows.go

+
+	// get device problemCode
+	getDeviceProblemCode := fmt.Sprintf("(Get-PnpDeviceProperty -InstanceId \"%s\" | Where-Object KeyName -eq DEVPKEY_Device_ProblemCode).Data[0]", instanceID) //nolint
+	devpkeyDeviceProblemCode, err := plc.ExecutePowershellCommand(getDeviceProblemCode)


network/network_windows.go

@@ -212,13 +204,13 @@

 		cmd := fmt.Sprintf(routeCmd, "delete", nwInfo.Subnets[1].Prefix.String(),
 			ifName, ipv6DefaultHop)
-		if out, err = nm.plClient.ExecuteCommand(cmd); err != nil {
+		if out, err = nm.plClient.ExecuteRawCommand(cmd); err != nil {


network/network_windows.go

 			logger.Error("Deleting ipv6 route failed", zap.Any("out", out), zap.Error(err))
 		}

 		cmd = fmt.Sprintf(routeCmd, "add", nwInfo.Subnets[1].Prefix.String(),
 			ifName, ipv6DefaultHop)
-		if out, err = nm.plClient.ExecuteCommand(cmd); err != nil {
+		if out, err = nm.plClient.ExecuteRawCommand(cmd); err != nil {


Bumps [golang.org/x/sys](https://github.com/golang/sys) from 0.20.0 to 0.21.0. - [Commits](golang/sys@v0.20.0...v0.21.0) --- updated-dependencies: - dependency-name: golang.org/x/sys dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bumps [golang.org/x/sync](https://github.com/golang/sync) from 0.9.0 to 0.10.0. - [Commits](golang/sync@v0.9.0...v0.10.0) --- updated-dependencies: - dependency-name: golang.org/x/sync dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

sivakami-projects requested a review from a team as a code owner October 21, 2025 16:59

sivakami-projects requested review from Copilot and jpayne3506 October 21, 2025 16:59

Copilot AI reviewed Oct 21, 2025

View reviewed changes

.pipelines/swiftv2-long-running/scripts/create_peerings.sh Outdated Show resolved Hide resolved

.pipelines/swiftv2-long-running/scripts/create_nsg.sh Outdated Show resolved Hide resolved

prmathur-microsoft reviewed Oct 21, 2025

View reviewed changes

pjohnst5 reviewed Oct 22, 2025

View reviewed changes

sivakami-projects requested a review from prmathur-microsoft October 22, 2025 19:13

sivakami-projects self-assigned this Oct 22, 2025

sivakami-projects added the swift-v2 label Oct 22, 2025

sivakami-projects requested a review from pjohnst5 October 22, 2025 21:44

QxBytes and others added 20 commits October 23, 2025 00:01

forwardport: fix: add iptables rules for dns in vnet scale cilium case (

613e368

#3421) * cover adding iptables rules for dns in vnet scale cilium case * replace existing iptables rules * modify imds iptables rule * mock iptables in cns and add ut * address linter issues * address linter issue

ci: fix check-shadowing not allowed (#3432)

0ce1515

fix check-shadowing not allowed

feat: add apiserver FQDN to CNS log metadata (#3382)

f8e4701

feat: add apiserver FQDN to CNS log metadata in AKS Signed-off-by: Evan Baker <rbtr@users.noreply.github.com>

fix: zapetw core With fields (#3433)

f90ed35

fix: zapetw core with fields Signed-off-by: Evan Baker <rbtr@users.noreply.github.com>

ci: unify windows and linux cluster creation methods (#3436)

9ddc5bf

* cleanup makefile * remove windows user and password from cniv1 cluster * fix windows variable not propagating

ci: add k8s yaml deprecation check to nightly (#3379)

34cc959

ci: add k8s yaml depreciation check to nightly

chore: use AppArmorProfile for cilium 1.16 (#3371)

b115b6d

* set appArmorProfile and remove annotations for cilium 1.16 * set dualstack appArmorProfile * set profile per container * update nightly daemonset

test: sort npm set policy values before validation (#3358)

805c51b

* sort set policy values before validation * address linter issues * sort expected values * address feedback

chore: add cilium 1.17 charts (#3413)

b400986

* chore: add cilium 1.17 and chart updates for k8s 1.32 * fix: rename config path * fix: make json path * swap kpr to strict in hubble chart

aggarwal0009 and others added 7 commits October 23, 2025 00:02

Add status to mtpnc crd (#4070)

4f314c8

* add status to mtpnc crd * update mtpnc status * update mtpnc status only * fix lint formating errs * add crd manifest * update mtpnc statuses * fix status enum * address pr feedback * fix lint * add missing file

chore: update dependencies for CVEs (#4093)

9c8aee8

chore: fix cns vulneribility issues Co-authored-by: paulyu <paulyu@microsoft.com>

downgrade easyjson GO package to unblock Release build (#4084)

230b7bb

downgrade easyjson GO package Co-authored-by: paulyu <paulyu@microsoft.com>

Update GO security dependencies (#4096)

5f4da73

* update go security dependencies * update retry dependency --------- Co-authored-by: paulyu <paulyu@microsoft.com>

sivakami-projects requested review from a team, QxBytes, ZetaoZhuang, ashvindeodhar, camrynl, paulyufan2, rbtr, santhoshmprabhu and thatmattlong as code owners October 23, 2025 07:02

sivakami-projects requested a review from vakalapa October 23, 2025 07:02

github-advanced-security bot found potential problems Oct 23, 2025

View reviewed changes

dependabot bot and others added 5 commits October 23, 2025 00:44

create storage account.

051afa8

disallow shared key access.

9d739d9

change pipeline template.

e46e84f

sivakami-projects mentioned this pull request Oct 23, 2025

Long running test pipeline for AKS swiftv2 clusters. #4098

Closed

sivakami-projects closed this Oct 23, 2025

sivakami-projects mentioned this pull request Oct 23, 2025

Swiftv2 pipeline branch #4099

Open

Uh oh!

Long running pipeline #4092

Long running pipeline #4092

Conversation

sivakami-projects commented Oct 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

prmathur-microsoft Oct 21, 2025

Choose a reason for hiding this comment

Uh oh!

pjohnst5 Oct 22, 2025

Choose a reason for hiding this comment

Uh oh!

pjohnst5 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

pjohnst5 Oct 22, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Check failure

Check failure

Check failure

Check failure

Check failure

Check failure

Check failure

Check failure

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

31 participants

sivakami-projects commented Oct 21, 2025 •

edited

Loading