Skip to content

{AKS} Fix remaining CLI Runner live test failures - #33848

Merged
Julie Zhu (yanzhudd) merged 2 commits into
Azure:devfrom
FumingZhang:fix/aks-live-test-followup
Aug 5, 2026
Merged

{AKS} Fix remaining CLI Runner live test failures#33848
Julie Zhu (yanzhudd) merged 2 commits into
Azure:devfrom
FumingZhang:fix/aks-live-test-followup

Conversation

@FumingZhang

@FumingZhang FumingZhang commented Aug 5, 2026

Copy link
Copy Markdown
Member

🤖 PR Validation — ️✔️ All clear

Breaking Changes Tests
️✔️ None ️✔️ 130/130

Related command

az aks

Description

Follow up on the AKS CLI Runner live run a8927490-05a9-4bea-9d6e-13057bfbd6c8 (49 failed / 196 passed / 30 skipped) after the previous stability fixes merged.

This change addresses the remaining actionable failures:

  • Read properties.provisioningState from generic az resource show responses. The previous poll loop read only the top-level property, causing 34 tests to time out with Final state: None.
  • Extend the bounded, live-only retry matcher to the additional observed transient AKS conflicts: managed-cluster operations already in progress, clusters still updating, and extensions still updating. Other errors and expected-failure tests still fail immediately.
  • Select the latest version supporting KubernetesOfficial for LocalDNS and network-isolated tests. Current AKS versions can support both official and LTS plans, so excluding all LTS-capable versions returned no version.
  • Preserve feature-required regions for control-plane metrics and artifact streaming rather than applying the runner-wide region override.
  • Use --no-ssh-key explicitly for Automatic SKU, choose a viable Automatic candidate size/region, and prepare a Container Insights workspace before cluster creation.
  • Prepare a Container Insights workspace for the app-routing + monitoring scenario instead of reusing an uninitialized default workspace.
  • Validate Azure Monitor metrics after asynchronous create postprocessing settles instead of asserting against the pre-postprocessing create response.
  • Keep legacy maintenance scenarios in replay with record_only() because their old and new payload shapes are rejected by the current public live API.
  • Follow the required network-isolated migration sequence: enable cached artifacts, reimage and wait for the system node pool, then change outbound type.

This is test-infrastructure-only and does not change customer-facing command behavior.

Testing Guide

Focused unit/replay tests:

python -m pytest -q \
  src/azure-cli/azure/cli/command_modules/acs/tests/latest/test_aks_provisioning_retry.py \
  src/azure-cli/azure/cli/command_modules/acs/tests/latest/test_custom_preparers.py \
  src/azure-cli/azure/cli/command_modules/acs/tests/latest/test_aks_commands.py::AzureKubernetesServiceScenarioTest::test_aks_maintenancewindow \
  src/azure-cli/azure/cli/command_modules/acs/tests/latest/test_aks_commands.py::AzureKubernetesServiceScenarioTest::test_aks_maintenanceconfiguration

Result: 31 passed, 12 subtests passed.

Live verification used subscription 79a7390d-3a85-432d-9f6f-a11a703c8b83 with AZURE_CLI_TEST_RETRY_PROVISIONING_CHECK=true:

  • Passed live: create/update control-plane metrics, managed namespace, all three LocalDNS scenarios, Azure Monitor metrics update, and app-routing + monitoring with a prepared workspace.
  • Automatic cluster creation reached a healthy Succeeded Automatic cluster with metrics enabled in centralus; the local test later hit an interactive Microsoft Graph token-protection challenge, which the workload-identity CLI Runner does not use.
  • Artifact-streaming scenarios could not be exercised in this subscription because its only permitted eastus VM family has zero quota.
  • Network-isolated verification was blocked before the changed migration sequence because the initial private cached cluster remained Creating for over an hour. The updated sequence is covered by command/replay validation.
  • Azure Container Storage live verification was blocked by the locally installed k8s-extension dependency environment, not the core CLI change.

All changed files pass python -m py_compile and git diff --check.

History Notes

None (test infrastructure only).


This checklist is used to make sure that common guidelines for a pull request are followed.

Handle generic ARM provisioning-state responses, expand targeted transient conflict retries, preserve feature rollout regions, and update dynamic version and Automatic SKU test assumptions.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

@microsoft-github-policy-service microsoft-github-policy-service Bot added Auto-Assign Auto assign by bot ARM az resource/group/lock/tag/deployment/policy/managementapp/account management-group act-identity-squad labels Aug 5, 2026
@a0x1ab

Copy link
Copy Markdown
Member

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 3 pipeline(s).

Prepare monitoring workspaces, preserve replay coverage for unsupported maintenance APIs, use a viable Automatic region, and serialize network-isolated migration reimaging.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@FumingZhang
FumingZhang marked this pull request as ready for review August 5, 2026 05:14
@FumingZhang
FumingZhang requested a review from a team as a code owner August 5, 2026 05:14
Copilot AI lite review requested due to automatic review settings August 5, 2026 05:14
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

@FumingZhang

Copy link
Copy Markdown
Member Author

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 3 pipeline(s).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR is a test-infrastructure follow-up for az aks CLI Runner live-test stability, addressing remaining AKS live-run failures by improving provisioning-state polling, expanding transient-conflict retry matching, and making several scenario setup steps more robust (version selection, region handling, and monitoring workspace preparation).

Changes:

  • Fix provisioning-state polling to also read properties.provisioningState from generic ARM az resource show responses.
  • Expand the transient operation-conflict retry matcher to cover additional observed AKS “in-progress/updating” conflict messages, and add unit coverage for those cases.
  • Add helpers and test updates to pick the latest KubernetesOfficial-supported version, preserve feature-required regions, and create a Log Analytics workspace/Container Insights solution for monitoring-related scenarios.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
src/azure-cli/azure/cli/command_modules/acs/tests/latest/test_aks_provisioning_retry.py Adds unit coverage for nested ARM provisioning state polling, expanded transient-conflict retry messages, and official-version selection.
src/azure-cli/azure/cli/command_modules/acs/tests/latest/test_aks_commands.py Implements nested provisioning-state polling, expands transient-conflict detection, adds official-version/workspace helpers, and updates live/replay scenarios accordingly.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@yanzhudd

Copy link
Copy Markdown
Contributor

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 3 pipeline(s).

@yanzhudd
Julie Zhu (yanzhudd) merged commit 9dd87ab into Azure:dev Aug 5, 2026
50 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

act-identity-squad act-observability-squad AKS az aks/acs/openshift ARM az resource/group/lock/tag/deployment/policy/managementapp/account management-group Auto-Assign Auto assign by bot

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants