From 9e850e534ebf9390ffd150c2438f76eb06488bdb Mon Sep 17 00:00:00 2001 From: "Erich(Renyong) Wang" Date: Wed, 16 Dec 2020 21:58:11 +0800 Subject: [PATCH 1/8] add test matrix for Az.Accounts --- .../test-matrix-for-az-accounts.md | 194 ++++++++++++++++++ 1 file changed, 194 insertions(+) create mode 100644 documentation/testing-docs/test-matrix-for-az-accounts.md diff --git a/documentation/testing-docs/test-matrix-for-az-accounts.md b/documentation/testing-docs/test-matrix-for-az-accounts.md new file mode 100644 index 000000000000..e0183db5c5bd --- /dev/null +++ b/documentation/testing-docs/test-matrix-for-az-accounts.md @@ -0,0 +1,194 @@ +# Test Matrix for Authentication in Az.Accounts + +Az.Accounts and its authentication feature is one of most important piece for Azure PowerShell, to make sure Azure PowerShell be delivered to customers with high quality, we define the test matrix and it should be honored by each release of Az.Accounts. + +**Priority Clarification** + +- `P0`: Run if any authentication related code change in Az.Accounts +- `P1`: Run if upgrading minor version of Azure.Identity and/or MSAL library +- `P2`: Run if upgrading major version of Azure.Identity and/or MSAL library + +## Azure Global Instance + +Azure Global Instance is the most important Azure instance, all tests should run against it. + +### PWSH Platform Matrix + +In theory all the combination of different OS platforms and PWSH versions should be covered, to compromise time effort and platform coverages, we should at least cover the following platforms. (For different versions of pwsh 7, need to cover at least smallest and biggest patch version for each major.minor version, currently it should be 7.0.0, 7.0.3 and 7.1.0. The reason to cover 7.0.0 is the future version of Azure.Core may contain higher version of built-in assemblies of pwsh.) + +- Windows PowerShell 5.1 +- PWSH 7.0.0 on Windows +- PWSH 7.0.x on Windows (latest patch version of PWSH 7.0) +- PWSH 7.1.0 on Windows +- PWSH 7.0.0 on Ubuntu +- PWSH 7.0.x on Ubuntu +- PWSH 7.1.0 on Ubuntu +- PWSH 7.0.x on MacOS +- CloudShell +- Docker Env for Ubuntu + +There's no need to run all tests on each of above platforms, the recommendation is to run most of tests on `Windows PowerShell 5.1` and to run the remaining tests on `PWSH 7.0.x on Windows`; just run smoke test `Connect-AzAccount`/`Connect-AzAccount -DeviceCode` on other platforms. + +### Connect-AzAccount Using Work/School Account + +|Scenario\Auth Method|Interactive|Device Code (`-DeviceCode`)|User Name+Password (`-Credential`)|Access Token (`-AccessToken`)|SP Secret (`-ServicePrincipal -Credential`)|SP Cert (`-ServicePrincipal -CertificateThumbprint`)|System MSI (`-Identity`)|User MSI (`-Identity -AccountId`)|User MSI-Func App published by VS Code (`-Identity -AccountId`)| +|----|----|----|----|----|----|----|----|----|----| +|`No parameter`|P0(SemiAuto)|P0|P0(SemiAuto-No)|P0(SemiAuto-No)|P0(SemiAuto-No)|P0(SemiAuto-No)|P0|P0|P0| +|`-Subscription sub-id`|P0(SemiAuto)|P1|P1(SemiAuto-No)|P1(SemiAuto-No)|P1(SemiAuto-No)|P2(SemiAuto-No)|P1|P1|P1| +|`-Subscription sub-name`|P1(SemiAuto)|P2|P2(SemiAuto-No)|P2(SemiAuto-No)|(SemiAuto-No)|P2(SemiAuto-No)|P2|P2|P2| +|`-Subscription sub-id-in-2nd-tenant`|P0(SemiAuto-No)|P2|P2(SemiAuto-No)|P2(SemiAuto-No)|P2(SemiAuto-No)|P2(SemiAuto-No)|NA|NA|NA| +|`-Tenant tenant-id`|P0(SemiAuto)|P1|P1(SemiAuto-No)|P1(SemiAuto-No)|P1(SemiAuto-No)|P2(SemiAuto-No)|P1|P1|P1| +|`-Tenant 2nd-tenant-id`|P1(SemiAuto-No)|P1|P1(SemiAuto-No)|P1(SemiAuto-No)|P1(SemiAuto-No)|P1(SemiAuto-No)|NA|NA|NA| +|`-Tenant tenant-id -Subscription sub-id`|P0(SemiAuto)|P1|P1(SemiAuto-No)|P1(SemiAuto-No)|P1(SemiAuto-No)|P1(SemiAuto-No)|P1|P1|P1| +|`-Tenant 2nd-tenant-id -Subscription sub-id-in-2nd-tenant`|P1(SemiAuto-No)|P2|P2(SemiAuto-No)|P2(SemiAuto-No)|P2(SemiAuto-No)|P2(SemiAuto-No)|NA|NA|NA| +|`No Parameter` Click back button before inputing password(Negative)|P2|P2|NA|NA|NA|NA|NA|NA|NA| +|`-Subscripiton -sub-id-no-permission`(Negative)|P2|P2|P2|P2|P2|P2|P2|P2|P2| +|`-Tenant -tenant-id-no-permission`(Negative)|P2|P2|P2|P2|P2|P2|P2|P2|P2| +|`-Tenant 1st-tenant-id -Subscription sub-id-in-2nd-tenant`(Negative)|P2|P2|P2|P2|P2|P2|P2|P2|P2| + +**Test Case Automation Status** + +- *SemiAuto* means the test case is available, but it needs manual input during running test. +- *SemiAuto-No* means the test case could be written in SemiAuto way, but it is not there yet. +- Test case without any status means manual, the cost for automation is high. + +### Connect-AzAccount Using MSA Account + +|Scenario\Auth Method|Interactive|Device Code (`-DeviceCode`)| +|----|----|----|----| +|`No parameter`|P0(SemiAuto)|P0| +|`-Subscription sub-id`|P2(SemiAuto)|P2| +|`-Tenant tenant-id`|P2(SemiAuto)|P2| +|`-Tenant tenant-id -Subscription sub-id`|P2(SemiAuto)|P2| + +### Other Authentication Related Tests + +It should be fine to run these test cases in just one platform. + +|Test|Priority| +|---|---| +|Login in Process scope `-Scope`|P1| +|Login with Multi Users|P1| +|Disconnect-AzAccount|P0| +|Disconnect-AzAccount(Service Principal) `-ApplicationId xxx -TenantId xxx`|P2| +|Disconnect-AzAccount(specifying context) `-AzureContext contextObject`|P2| +|Disconnect-AzAccount(Login with multi users, log out one)|P2| +|Save-AzContext and Import-AzContext|P2| +|Get-AzAccessToken|P1| +|Token Cache Fallback(Linux Only)|P1| + +**NOTE** + +1. Special Test Environments + +- ADFS Env (P1) +- SAW Machine (P2) + +1. Azure Government Instances (Will be covered by dedicated teams) + +- Mooncake +- Blackforest +- Fairfax + +1. Special Scenario: + +- Interactive authentication should return warning if connecting to Linux using SSH (P0, Auto) +- Interactive authentication should be successful even the port 8400 is taken by other process first (P0, Auto) +- Token should be auto refreshed for long running operation(> 1 hour) (P2, Manual, please refer `How To Test` section) +- Token cache file should be compatible with az (P2, Manual, please refer `How To Test` section) +- Service Principal authentication should be successful if http proxy is set (P2, Manual, please refer to `How To Test` section) +- FMR scenario(Integrated Windows Auth) (P2, Manual, please refer `How To Test` section) + +1. If possible, we should provide preview/engineering bits to our partners for verifying: (P2) + +- Azure Stack team to verify ADFS scenario +- Azure Function team +- Azure StackHCI team + +## How To Test + +1. How to test LRO + +```powershell +New-AzResourceGroupDeployment -Name xxxx -ResourceGroupName xxxx -TemplateFile path-to-template-file +``` + +You may save json content below as template file, make sure the value of `parameters.userAssignedIdentity.defaultValue` has been set correctly and the user MSI has permission. + +```json +{ + "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", + "contentVersion": "1.0.0.0", + "parameters": { + "userAssignedIdentity": { + "type": "string", + "defaultValue": "/subscriptions/331856d2-64ff-4cde-abe7-b782335b52da/resourceGroups/yourgroupname/providers/Microsoft.ManagedIdentity/userAssignedIdentities/youruseridentityname" + }, + "newGuid": { + "type": "string", + "defaultValue": "[newGuid()]" + } + }, + "variables": { + "scriptName": "sleep11" + }, + "resources": [ + { + "type": "Microsoft.Resources/deploymentScripts", + "apiVersion": "2019-10-01-preview", + "name": "[variables('scriptName')]", + "location": "[resourceGroup().location]", + "identity": { + "type": "UserAssigned", + "userAssignedIdentities": { + "[parameters('userAssignedIdentity')]": { + } + } + }, + "kind": "AzurePowerShell", + "properties": { + "forceUpdateTag": "[parameters('newGuid')]", + "azPowerShellVersion": "4.8", + "timeout": "PT90M", + "retentionInterval": "P1D", + "cleanupPreference": "Always", + "scriptContent": "Start-Sleep -Seconds 5000" + } + } + ], + "outputs": { + "ip": { + "type": "string", + "value": "Slept" + } + } +} +``` + +1. How to test Http Proxy + + a. Start Fiddler + b. Restart Windows PowerShell and run `Connect-AzAccount` + c. You should see http request in Fiddler like `https://login.microsoftonline.com/organizations/oauth2/v2.0/token` + +1. How to test compatability with az + + Expect no error happens: + a. az login + b. Delete the token cache file (C:\Users\AppData\Local.IdentityService\msal.cache) + c. Connect-AzAccount + d. Get-AzSubscription + e. az group list + f. Disconnect-AzAccount + +1. How to test Integrated Windows Auth + + a. Start Fiddler + b. Restart Windows PowerShell and Connect-AzAccount using your corp account + + ```powershell + $cred = Get-Credential + Connect-AzAccount -Credential $cred + ``` + + c. Although failed to login, but the http reqeust `https://msft.sts.microsoft.com/adfs/services/trust/2005/usernamemixed` should be successful. From 1062d98f6e18202d3a33710860ecb75e0ca91479 Mon Sep 17 00:00:00 2001 From: erich-wang Date: Wed, 16 Dec 2020 22:04:29 +0800 Subject: [PATCH 2/8] Update test-matrix-for-az-accounts.md --- .../testing-docs/test-matrix-for-az-accounts.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/documentation/testing-docs/test-matrix-for-az-accounts.md b/documentation/testing-docs/test-matrix-for-az-accounts.md index e0183db5c5bd..ad326b024e59 100644 --- a/documentation/testing-docs/test-matrix-for-az-accounts.md +++ b/documentation/testing-docs/test-matrix-for-az-accounts.md @@ -1,6 +1,6 @@ # Test Matrix for Authentication in Az.Accounts -Az.Accounts and its authentication feature is one of most important piece for Azure PowerShell, to make sure Azure PowerShell be delivered to customers with high quality, we define the test matrix and it should be honored by each release of Az.Accounts. +The authentication functionality in Az.Accounts is one of most important piece for Azure PowerShell, to make sure Azure PowerShell be delivered to customers with high quality, we define the test matrix and it should be honored by each release of Az.Accounts. **Priority Clarification** @@ -55,7 +55,7 @@ There's no need to run all tests on each of above platforms, the recommendation ### Connect-AzAccount Using MSA Account |Scenario\Auth Method|Interactive|Device Code (`-DeviceCode`)| -|----|----|----|----| +|----|----|----| |`No parameter`|P0(SemiAuto)|P0| |`-Subscription sub-id`|P2(SemiAuto)|P2| |`-Tenant tenant-id`|P2(SemiAuto)|P2| @@ -84,13 +84,13 @@ It should be fine to run these test cases in just one platform. - ADFS Env (P1) - SAW Machine (P2) -1. Azure Government Instances (Will be covered by dedicated teams) +2. Azure Government Instances (Will be covered by dedicated teams) - Mooncake - Blackforest - Fairfax -1. Special Scenario: +3. Special Scenario: - Interactive authentication should return warning if connecting to Linux using SSH (P0, Auto) - Interactive authentication should be successful even the port 8400 is taken by other process first (P0, Auto) @@ -99,7 +99,7 @@ It should be fine to run these test cases in just one platform. - Service Principal authentication should be successful if http proxy is set (P2, Manual, please refer to `How To Test` section) - FMR scenario(Integrated Windows Auth) (P2, Manual, please refer `How To Test` section) -1. If possible, we should provide preview/engineering bits to our partners for verifying: (P2) +4. If possible, we should provide preview/engineering bits to our partners for verifying: (P2) - Azure Stack team to verify ADFS scenario - Azure Function team @@ -165,13 +165,13 @@ You may save json content below as template file, make sure the value of `parame } ``` -1. How to test Http Proxy +2. How to test Http Proxy a. Start Fiddler b. Restart Windows PowerShell and run `Connect-AzAccount` c. You should see http request in Fiddler like `https://login.microsoftonline.com/organizations/oauth2/v2.0/token` -1. How to test compatability with az +3. How to test compatability with az Expect no error happens: a. az login @@ -181,7 +181,7 @@ You may save json content below as template file, make sure the value of `parame e. az group list f. Disconnect-AzAccount -1. How to test Integrated Windows Auth +4. How to test Integrated Windows Auth a. Start Fiddler b. Restart Windows PowerShell and Connect-AzAccount using your corp account From 053095659c85a78bc1e232146b74375de915b9e3 Mon Sep 17 00:00:00 2001 From: erich-wang Date: Wed, 16 Dec 2020 22:05:22 +0800 Subject: [PATCH 3/8] Update test-matrix-for-az-accounts.md --- documentation/testing-docs/test-matrix-for-az-accounts.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/documentation/testing-docs/test-matrix-for-az-accounts.md b/documentation/testing-docs/test-matrix-for-az-accounts.md index ad326b024e59..61752a882d93 100644 --- a/documentation/testing-docs/test-matrix-for-az-accounts.md +++ b/documentation/testing-docs/test-matrix-for-az-accounts.md @@ -23,7 +23,9 @@ In theory all the combination of different OS platforms and PWSH versions should - PWSH 7.0.0 on Ubuntu - PWSH 7.0.x on Ubuntu - PWSH 7.1.0 on Ubuntu +- PWSH 7.0.0 on MacOS - PWSH 7.0.x on MacOS +- PWSH 7.1.0 on MacOS - CloudShell - Docker Env for Ubuntu From 52f9759f90527356509610826681df4539b2574d Mon Sep 17 00:00:00 2001 From: "Erich(Renyong) Wang" Date: Wed, 23 Dec 2020 10:21:10 +0800 Subject: [PATCH 4/8] update to address comments --- .../testing-docs/test-matrix-for-az-accounts.md | 16 +++++++--------- 1 file changed, 7 insertions(+), 9 deletions(-) diff --git a/documentation/testing-docs/test-matrix-for-az-accounts.md b/documentation/testing-docs/test-matrix-for-az-accounts.md index 61752a882d93..bc83a62fab22 100644 --- a/documentation/testing-docs/test-matrix-for-az-accounts.md +++ b/documentation/testing-docs/test-matrix-for-az-accounts.md @@ -1,16 +1,16 @@ # Test Matrix for Authentication in Az.Accounts -The authentication functionality in Az.Accounts is one of most important piece for Azure PowerShell, to make sure Azure PowerShell be delivered to customers with high quality, we define the test matrix and it should be honored by each release of Az.Accounts. +The authentication functionality in Az.Accounts is one of the most important pieces in Azure PowerShell. To make sure Azure PowerShell be delivered to customers with high quality, we define the test matrix which should be honored by each release of Az.Accounts when there is any change related to authentication. **Priority Clarification** -- `P0`: Run if any authentication related code change in Az.Accounts -- `P1`: Run if upgrading minor version of Azure.Identity and/or MSAL library -- `P2`: Run if upgrading major version of Azure.Identity and/or MSAL library +- `P0`: Run if any authentication related code change in Az.Accounts, or upgrading Azure.Identity or MSAL library +- `P1`: Run if upgrading minor or major version of Azure.Identity or MSAL library +- `P2`: Run if upgrading major version of Azure.Identity or MSAL library -## Azure Global Instance +## Azure Public Cloud -Azure Global Instance is the most important Azure instance, all tests should run against it. +Azure Public Cloud is the most important Azure instance, all tests should run against it. ### PWSH Platform Matrix @@ -18,14 +18,12 @@ In theory all the combination of different OS platforms and PWSH versions should - Windows PowerShell 5.1 - PWSH 7.0.0 on Windows -- PWSH 7.0.x on Windows (latest patch version of PWSH 7.0) +- PWSH 7.0.x(latest patch version) on Windows - PWSH 7.1.0 on Windows - PWSH 7.0.0 on Ubuntu - PWSH 7.0.x on Ubuntu - PWSH 7.1.0 on Ubuntu -- PWSH 7.0.0 on MacOS - PWSH 7.0.x on MacOS -- PWSH 7.1.0 on MacOS - CloudShell - Docker Env for Ubuntu From 80cf0458a8b2fe82f3f5b8f17d0061e1bf03490a Mon Sep 17 00:00:00 2001 From: "Erich(Renyong) Wang" Date: Wed, 23 Dec 2020 13:52:57 +0800 Subject: [PATCH 5/8] update category description --- .../test-matrix-for-az-accounts.md | 29 ++++++++++++++----- 1 file changed, 21 insertions(+), 8 deletions(-) diff --git a/documentation/testing-docs/test-matrix-for-az-accounts.md b/documentation/testing-docs/test-matrix-for-az-accounts.md index bc83a62fab22..5510611c532a 100644 --- a/documentation/testing-docs/test-matrix-for-az-accounts.md +++ b/documentation/testing-docs/test-matrix-for-az-accounts.md @@ -2,12 +2,6 @@ The authentication functionality in Az.Accounts is one of the most important pieces in Azure PowerShell. To make sure Azure PowerShell be delivered to customers with high quality, we define the test matrix which should be honored by each release of Az.Accounts when there is any change related to authentication. -**Priority Clarification** - -- `P0`: Run if any authentication related code change in Az.Accounts, or upgrading Azure.Identity or MSAL library -- `P1`: Run if upgrading minor or major version of Azure.Identity or MSAL library -- `P2`: Run if upgrading major version of Azure.Identity or MSAL library - ## Azure Public Cloud Azure Public Cloud is the most important Azure instance, all tests should run against it. @@ -26,16 +20,31 @@ In theory all the combination of different OS platforms and PWSH versions should - PWSH 7.0.x on MacOS - CloudShell - Docker Env for Ubuntu +- Windows PowerShell 5.1 on SAW machine -There's no need to run all tests on each of above platforms, the recommendation is to run most of tests on `Windows PowerShell 5.1` and to run the remaining tests on `PWSH 7.0.x on Windows`; just run smoke test `Connect-AzAccount`/`Connect-AzAccount -DeviceCode` on other platforms. +There's no need to run all tests on each of above platforms, the recommendation is: + +1. For `Windows PowerShell 5.1`, run all tests. +2. For `PWSH 7.0.x on Windows`, run tests on columns **Interactive/Device Code/SP Secret** (please refer to test scenario table). +3. For other platforms, just run smoke test `Connect-AzAccount`/`Connect-AzAccount -DeviceCode`. ### Connect-AzAccount Using Work/School Account +**Category Clarification** + +All test scenario are grouped into three different category: P0, P1 and P2. Whether to verify test scenario in one category or not depends on change scope in Az.Accounts. In the following table, `Yes` means test scenario in that category need to be verified , `No` means no need to verify. + +|Category\Change in Az.Accounts|Upgrade major version of Azure.Identity/MSAL lib|Upgrade minor version of Azure.Identity/MSAL lib| Auth related code change in Az.Accounts | +|----|----|----|----| +|P0|Yes|Yes|Yes| +|P1|Yes|Yes|No| +|P2|Yes|No|No| + |Scenario\Auth Method|Interactive|Device Code (`-DeviceCode`)|User Name+Password (`-Credential`)|Access Token (`-AccessToken`)|SP Secret (`-ServicePrincipal -Credential`)|SP Cert (`-ServicePrincipal -CertificateThumbprint`)|System MSI (`-Identity`)|User MSI (`-Identity -AccountId`)|User MSI-Func App published by VS Code (`-Identity -AccountId`)| |----|----|----|----|----|----|----|----|----|----| |`No parameter`|P0(SemiAuto)|P0|P0(SemiAuto-No)|P0(SemiAuto-No)|P0(SemiAuto-No)|P0(SemiAuto-No)|P0|P0|P0| |`-Subscription sub-id`|P0(SemiAuto)|P1|P1(SemiAuto-No)|P1(SemiAuto-No)|P1(SemiAuto-No)|P2(SemiAuto-No)|P1|P1|P1| -|`-Subscription sub-name`|P1(SemiAuto)|P2|P2(SemiAuto-No)|P2(SemiAuto-No)|(SemiAuto-No)|P2(SemiAuto-No)|P2|P2|P2| +|`-Subscription sub-name`|P1(SemiAuto)|P2|P2(SemiAuto-No)|P2(SemiAuto-No)|P2(SemiAuto-No)|P2(SemiAuto-No)|P2|P2|P2| |`-Subscription sub-id-in-2nd-tenant`|P0(SemiAuto-No)|P2|P2(SemiAuto-No)|P2(SemiAuto-No)|P2(SemiAuto-No)|P2(SemiAuto-No)|NA|NA|NA| |`-Tenant tenant-id`|P0(SemiAuto)|P1|P1(SemiAuto-No)|P1(SemiAuto-No)|P1(SemiAuto-No)|P2(SemiAuto-No)|P1|P1|P1| |`-Tenant 2nd-tenant-id`|P1(SemiAuto-No)|P1|P1(SemiAuto-No)|P1(SemiAuto-No)|P1(SemiAuto-No)|P1(SemiAuto-No)|NA|NA|NA| @@ -192,3 +201,7 @@ You may save json content below as template file, make sure the value of `parame ``` c. Although failed to login, but the http reqeust `https://msft.sts.microsoft.com/adfs/services/trust/2005/usernamemixed` should be successful. + +5. How to test on SAW machine + + It would be great if you have real SAW machine. If not, please run PowerShell under [Constrained Language Mode](https://devblogs.microsoft.com/powershell/powershell-constrained-language-mode/) which should have similar effect as on SAW machine. From b3c66be1e3aa627ee3efaec25b3e3ef822444685 Mon Sep 17 00:00:00 2001 From: "Erich(Renyong) Wang" Date: Sun, 3 Jan 2021 17:32:29 +0800 Subject: [PATCH 6/8] refine test auto tag --- .../testing-docs/test-matrix-for-az-accounts.md | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/documentation/testing-docs/test-matrix-for-az-accounts.md b/documentation/testing-docs/test-matrix-for-az-accounts.md index 5510611c532a..068db536c121 100644 --- a/documentation/testing-docs/test-matrix-for-az-accounts.md +++ b/documentation/testing-docs/test-matrix-for-az-accounts.md @@ -42,14 +42,14 @@ All test scenario are grouped into three different category: P0, P1 and P2. Whet |Scenario\Auth Method|Interactive|Device Code (`-DeviceCode`)|User Name+Password (`-Credential`)|Access Token (`-AccessToken`)|SP Secret (`-ServicePrincipal -Credential`)|SP Cert (`-ServicePrincipal -CertificateThumbprint`)|System MSI (`-Identity`)|User MSI (`-Identity -AccountId`)|User MSI-Func App published by VS Code (`-Identity -AccountId`)| |----|----|----|----|----|----|----|----|----|----| -|`No parameter`|P0(SemiAuto)|P0|P0(SemiAuto-No)|P0(SemiAuto-No)|P0(SemiAuto-No)|P0(SemiAuto-No)|P0|P0|P0| -|`-Subscription sub-id`|P0(SemiAuto)|P1|P1(SemiAuto-No)|P1(SemiAuto-No)|P1(SemiAuto-No)|P2(SemiAuto-No)|P1|P1|P1| -|`-Subscription sub-name`|P1(SemiAuto)|P2|P2(SemiAuto-No)|P2(SemiAuto-No)|P2(SemiAuto-No)|P2(SemiAuto-No)|P2|P2|P2| -|`-Subscription sub-id-in-2nd-tenant`|P0(SemiAuto-No)|P2|P2(SemiAuto-No)|P2(SemiAuto-No)|P2(SemiAuto-No)|P2(SemiAuto-No)|NA|NA|NA| -|`-Tenant tenant-id`|P0(SemiAuto)|P1|P1(SemiAuto-No)|P1(SemiAuto-No)|P1(SemiAuto-No)|P2(SemiAuto-No)|P1|P1|P1| -|`-Tenant 2nd-tenant-id`|P1(SemiAuto-No)|P1|P1(SemiAuto-No)|P1(SemiAuto-No)|P1(SemiAuto-No)|P1(SemiAuto-No)|NA|NA|NA| -|`-Tenant tenant-id -Subscription sub-id`|P0(SemiAuto)|P1|P1(SemiAuto-No)|P1(SemiAuto-No)|P1(SemiAuto-No)|P1(SemiAuto-No)|P1|P1|P1| -|`-Tenant 2nd-tenant-id -Subscription sub-id-in-2nd-tenant`|P1(SemiAuto-No)|P2|P2(SemiAuto-No)|P2(SemiAuto-No)|P2(SemiAuto-No)|P2(SemiAuto-No)|NA|NA|NA| +|`No parameter`|P0(SemiAuto)|P0|P0(Auto-No)|P0(SemiAuto-No)|P0(Auto-No)|P0(SemiAuto-No)|P0|P0|P0| +|`-Subscription sub-id`|P0(SemiAuto)|P1|P1(Auto-No)|P1(SemiAuto-No)|P1(Auto-No)|P2(SemiAuto-No)|P1|P1|P1| +|`-Subscription sub-name`|P1(SemiAuto)|P2|P2(Auto-No)|P2(SemiAuto-No)|P2(Auto-No)|P2(SemiAuto-No)|P2|P2|P2| +|`-Subscription sub-id-in-2nd-tenant`|P0(SemiAuto-No)|P2|P2(Auto-No)|P2(SemiAuto-No)|P2(Auto-No)|P2(SemiAuto-No)|NA|NA|NA| +|`-Tenant tenant-id`|P0(SemiAuto)|P1|P1(Auto-No)|P1(SemiAuto-No)|P1(Auto-No)|P2(SemiAuto-No)|P1|P1|P1| +|`-Tenant 2nd-tenant-id`|P1(SemiAuto-No)|P1|P1(Auto-No)|P1(SemiAuto-No)|P1(Auto-No)|P1(SemiAuto-No)|NA|NA|NA| +|`-Tenant tenant-id -Subscription sub-id`|P0(SemiAuto)|P1|P1(Auto-No)|P1(SemiAuto-No)|P1(Auto-No)|P1(SemiAuto-No)|P1|P1|P1| +|`-Tenant 2nd-tenant-id -Subscription sub-id-in-2nd-tenant`|P1(SemiAuto-No)|P2|P2(Auto-No)|P2(SemiAuto-No)|P2(Auto-No)|P2(SemiAuto-No)|NA|NA|NA| |`No Parameter` Click back button before inputing password(Negative)|P2|P2|NA|NA|NA|NA|NA|NA|NA| |`-Subscripiton -sub-id-no-permission`(Negative)|P2|P2|P2|P2|P2|P2|P2|P2|P2| |`-Tenant -tenant-id-no-permission`(Negative)|P2|P2|P2|P2|P2|P2|P2|P2|P2| @@ -59,6 +59,7 @@ All test scenario are grouped into three different category: P0, P1 and P2. Whet - *SemiAuto* means the test case is available, but it needs manual input during running test. - *SemiAuto-No* means the test case could be written in SemiAuto way, but it is not there yet. +- *Auto-No* means the test case could be written in automatic way(by environment variable), but it is not there yet. - Test case without any status means manual, the cost for automation is high. ### Connect-AzAccount Using MSA Account From 1a23fc53c857541d074bd76929ea1e5595c0452c Mon Sep 17 00:00:00 2001 From: "Erich(Renyong) Wang" Date: Tue, 5 Jan 2021 23:29:59 +0800 Subject: [PATCH 7/8] refine test matrix --- .../test-matrix-for-az-accounts.md | 139 ++++++++++-------- 1 file changed, 76 insertions(+), 63 deletions(-) diff --git a/documentation/testing-docs/test-matrix-for-az-accounts.md b/documentation/testing-docs/test-matrix-for-az-accounts.md index 068db536c121..9796d9cab36d 100644 --- a/documentation/testing-docs/test-matrix-for-az-accounts.md +++ b/documentation/testing-docs/test-matrix-for-az-accounts.md @@ -2,54 +2,25 @@ The authentication functionality in Az.Accounts is one of the most important pieces in Azure PowerShell. To make sure Azure PowerShell be delivered to customers with high quality, we define the test matrix which should be honored by each release of Az.Accounts when there is any change related to authentication. -## Azure Public Cloud +## Test Scenario -Azure Public Cloud is the most important Azure instance, all tests should run against it. +Each test scenario is marked with one priority P0, P1, P2 based on two factors: -### PWSH Platform Matrix - -In theory all the combination of different OS platforms and PWSH versions should be covered, to compromise time effort and platform coverages, we should at least cover the following platforms. (For different versions of pwsh 7, need to cover at least smallest and biggest patch version for each major.minor version, currently it should be 7.0.0, 7.0.3 and 7.1.0. The reason to cover 7.0.0 is the future version of Azure.Core may contain higher version of built-in assemblies of pwsh.) - -- Windows PowerShell 5.1 -- PWSH 7.0.0 on Windows -- PWSH 7.0.x(latest patch version) on Windows -- PWSH 7.1.0 on Windows -- PWSH 7.0.0 on Ubuntu -- PWSH 7.0.x on Ubuntu -- PWSH 7.1.0 on Ubuntu -- PWSH 7.0.x on MacOS -- CloudShell -- Docker Env for Ubuntu -- Windows PowerShell 5.1 on SAW machine - -There's no need to run all tests on each of above platforms, the recommendation is: - -1. For `Windows PowerShell 5.1`, run all tests. -2. For `PWSH 7.0.x on Windows`, run tests on columns **Interactive/Device Code/SP Secret** (please refer to test scenario table). -3. For other platforms, just run smoke test `Connect-AzAccount`/`Connect-AzAccount -DeviceCode`. +- The importance of the test scenario, i.e. whether used by customers popularly, P0 means most popular. +- Whether be easily affected by authentication related code change, P0 means most easily be affected. ### Connect-AzAccount Using Work/School Account -**Category Clarification** - -All test scenario are grouped into three different category: P0, P1 and P2. Whether to verify test scenario in one category or not depends on change scope in Az.Accounts. In the following table, `Yes` means test scenario in that category need to be verified , `No` means no need to verify. - -|Category\Change in Az.Accounts|Upgrade major version of Azure.Identity/MSAL lib|Upgrade minor version of Azure.Identity/MSAL lib| Auth related code change in Az.Accounts | -|----|----|----|----| -|P0|Yes|Yes|Yes| -|P1|Yes|Yes|No| -|P2|Yes|No|No| - |Scenario\Auth Method|Interactive|Device Code (`-DeviceCode`)|User Name+Password (`-Credential`)|Access Token (`-AccessToken`)|SP Secret (`-ServicePrincipal -Credential`)|SP Cert (`-ServicePrincipal -CertificateThumbprint`)|System MSI (`-Identity`)|User MSI (`-Identity -AccountId`)|User MSI-Func App published by VS Code (`-Identity -AccountId`)| |----|----|----|----|----|----|----|----|----|----| -|`No parameter`|P0(SemiAuto)|P0|P0(Auto-No)|P0(SemiAuto-No)|P0(Auto-No)|P0(SemiAuto-No)|P0|P0|P0| -|`-Subscription sub-id`|P0(SemiAuto)|P1|P1(Auto-No)|P1(SemiAuto-No)|P1(Auto-No)|P2(SemiAuto-No)|P1|P1|P1| -|`-Subscription sub-name`|P1(SemiAuto)|P2|P2(Auto-No)|P2(SemiAuto-No)|P2(Auto-No)|P2(SemiAuto-No)|P2|P2|P2| -|`-Subscription sub-id-in-2nd-tenant`|P0(SemiAuto-No)|P2|P2(Auto-No)|P2(SemiAuto-No)|P2(Auto-No)|P2(SemiAuto-No)|NA|NA|NA| -|`-Tenant tenant-id`|P0(SemiAuto)|P1|P1(Auto-No)|P1(SemiAuto-No)|P1(Auto-No)|P2(SemiAuto-No)|P1|P1|P1| -|`-Tenant 2nd-tenant-id`|P1(SemiAuto-No)|P1|P1(Auto-No)|P1(SemiAuto-No)|P1(Auto-No)|P1(SemiAuto-No)|NA|NA|NA| -|`-Tenant tenant-id -Subscription sub-id`|P0(SemiAuto)|P1|P1(Auto-No)|P1(SemiAuto-No)|P1(Auto-No)|P1(SemiAuto-No)|P1|P1|P1| -|`-Tenant 2nd-tenant-id -Subscription sub-id-in-2nd-tenant`|P1(SemiAuto-No)|P2|P2(Auto-No)|P2(SemiAuto-No)|P2(Auto-No)|P2(SemiAuto-No)|NA|NA|NA| +|`No Subscrption/Tenant`|P0 (SemiAuto)|P0|P0 (Auto-No)|P0 (SemiAuto-No)|P0 (Auto-No)|P0 (SemiAuto-No)|P0|P0|P0| +|`-Subscription sub-id`|P0 (SemiAuto)|P1|P1 (Auto-No)|P1 (SemiAuto-No)|P1 (Auto-No)|P2 (SemiAuto-No)|P1|P1|P1| +|`-Subscription sub-name`|P1 (SemiAuto)|P2|P2 (Auto-No)|P2 (SemiAuto-No)|P2 (Auto-No)|P2 (SemiAuto-No)|P2|P2|P2| +|`-Subscription sub-id-in-2nd-tenant`|P0 (SemiAuto-No)|P2|P2 (Auto-No)|P2 (SemiAuto-No)|P2 (Auto-No)|P2 (SemiAuto-No)|NA|NA|NA| +|`-Tenant tenant-id`|P0 (SemiAuto)|P1|P1 (Auto-No)|P1 (SemiAuto-No)|P1 (Auto-No)|P2 (SemiAuto-No)|P1|P1|P1| +|`-Tenant 2nd-tenant-id`|P1 (SemiAuto-No)|P1|P1 (Auto-No)|P1 (SemiAuto-No)|P1 (Auto-No)|P1 (SemiAuto-No)|NA|NA|NA| +|`-Tenant tenant-id -Subscription sub-id`|P0 (SemiAuto)|P1|P1 (Auto-No)|P1 (SemiAuto-No)|P1 (Auto-No)|P1 (SemiAuto-No)|P1|P1|P1| +|`-Tenant 2nd-tenant-id -Subscription sub-id-in-2nd-tenant`|P1 (SemiAuto-No)|P2|P2 (Auto-No)|P2 (SemiAuto-No)|P2 (Auto-No)|P2 (SemiAuto-No)|NA|NA|NA| |`No Parameter` Click back button before inputing password(Negative)|P2|P2|NA|NA|NA|NA|NA|NA|NA| |`-Subscripiton -sub-id-no-permission`(Negative)|P2|P2|P2|P2|P2|P2|P2|P2|P2| |`-Tenant -tenant-id-no-permission`(Negative)|P2|P2|P2|P2|P2|P2|P2|P2|P2| @@ -66,14 +37,25 @@ All test scenario are grouped into three different category: P0, P1 and P2. Whet |Scenario\Auth Method|Interactive|Device Code (`-DeviceCode`)| |----|----|----| -|`No parameter`|P0(SemiAuto)|P0| -|`-Subscription sub-id`|P2(SemiAuto)|P2| -|`-Tenant tenant-id`|P2(SemiAuto)|P2| -|`-Tenant tenant-id -Subscription sub-id`|P2(SemiAuto)|P2| +|`No Subscription/Tenant`|P0 (SemiAuto)|P0| +|`-Subscription sub-id`|P2 (SemiAuto)|P2| +|`-Tenant tenant-id`|P2 (SemiAuto)|P2| +|`-Tenant tenant-id -Subscription sub-id`|P2 (SemiAuto)|P2| + +### Connect-AzAccount - Special Test Scenario + +|Test|Priority|Comment| +|----|----|----| +|Interactive authentication should return warning if connecting to Linux using SSH|P0 (Auto)|| +|Interactive authentication should be successful even the port 8400 is taken by other process first|P0 (Auto)|| +|Token should be auto refreshed for long running operation(> 1 hour)|P2|Please refer `How To Test` section| +|Token cache file should be compatible with az|P2|Please refer `How To Test` section| +|Service Principal authentication should be successful if http proxy is set|P2|Please refer `How To Test` section| +|FMR scenario(Integrated Windows Auth)|P2|Please refer `How To Test` section| -### Other Authentication Related Tests +### Other Authentication Test Scenario -It should be fine to run these test cases in just one platform. +It should be fine to run these test cases in just one platform, e.g. Windows PowerShell 5.1. |Test|Priority| |---|---| @@ -87,29 +69,60 @@ It should be fine to run these test cases in just one platform. |Get-AzAccessToken|P1| |Token Cache Fallback(Linux Only)|P1| -**NOTE** +## Test Strategy + +### Azure Environments + +|Azure Environment|Priority|Comment| +|----|-----|----| +|Azure Global Instance|P0|| +|Azure US Instance|P2|Will be covered by dedicated team| +|Azure China Instance|P2|Will be covered by dedicated team| +|Azure German Instance|P2|Will be covered by dedicated team| +|ADFS Environment|P1|Ask Azure Stack team to help| +|SAW Machine|P2|| + +### PWSH Platforms + +In theory all the combination of different OS platforms and PWSH versions should be covered, to compromise time effort and platform coverages, we should at least cover the following platforms. (For different versions of pwsh 7, need to cover at least smallest and biggest patch version for each major.minor version, currently it should be 7.0.0, 7.0.3 and 7.1.0. The reason to cover 7.0.0 is the future version of Azure.Core may contain higher version of built-in assemblies of pwsh.) + +- Windows PowerShell 5.1 +- PWSH 7.0.0 on Windows +- PWSH 7.0.x(latest patch version) on Windows +- PWSH 7.1.0 on Windows +- PWSH 7.0.0 on Ubuntu +- PWSH 7.0.x on Ubuntu +- PWSH 7.1.0 on Ubuntu +- PWSH 7.0.x on MacOS +- CloudShell +- Docker Env for Ubuntu +- Windows PowerShell 5.1 on SAW machine + +There's no need to run all tests on each of above platforms, the recommendation is: + +1. For `Windows PowerShell 5.1`, run all applicable tests. +2. For `PWSH 7.0.x on Windows`, run tests on columns **Interactive/Device Code/SP Secret** (please refer to test scenario table). +3. For other platforms, just run smoke test `Connect-AzAccount`/`Connect-AzAccount -DeviceCode`. + +### Auth Code Change Impact + +When there is auth related code change in Az.Accounts, there's no need to run all test scenario on all platforms and environments. In contrast, we may choose to run different test scenario based on different auth code change impact, so that only affected test cases are covered. + +#### Just Auth Code Change without Version Upgrade of MSAL/Azure.Identity in Az.Accounts -1. Special Test Environments +Only need to verify P0 test scenario. -- ADFS Env (P1) -- SAW Machine (P2) +#### Minor/Patch Version Upgrade of MSAL/Azure.Identity in Az.Accounts -2. Azure Government Instances (Will be covered by dedicated teams) +Only need to verify P0 and P1 test scenario. -- Mooncake -- Blackforest -- Fairfax +#### Major Version Upgrade of MSAL/Azure.Identity in Az.Accounts -3. Special Scenario: +Need to verify P0, P1 and P2 test scenario. -- Interactive authentication should return warning if connecting to Linux using SSH (P0, Auto) -- Interactive authentication should be successful even the port 8400 is taken by other process first (P0, Auto) -- Token should be auto refreshed for long running operation(> 1 hour) (P2, Manual, please refer `How To Test` section) -- Token cache file should be compatible with az (P2, Manual, please refer `How To Test` section) -- Service Principal authentication should be successful if http proxy is set (P2, Manual, please refer to `How To Test` section) -- FMR scenario(Integrated Windows Auth) (P2, Manual, please refer `How To Test` section) +### Partner Teams -4. If possible, we should provide preview/engineering bits to our partners for verifying: (P2) +If possible, we should provide preview/engineering bits to our partners for verifying: (P2) - Azure Stack team to verify ADFS scenario - Azure Function team From ecfe1d51162a6f6c190d6cb56b1e46f0c6da775e Mon Sep 17 00:00:00 2001 From: "Erich(Renyong) Wang" Date: Fri, 8 Jan 2021 16:20:06 +0800 Subject: [PATCH 8/8] refine document --- .../test-matrix-for-az-accounts.md | 51 +++++++++---------- 1 file changed, 25 insertions(+), 26 deletions(-) diff --git a/documentation/testing-docs/test-matrix-for-az-accounts.md b/documentation/testing-docs/test-matrix-for-az-accounts.md index 9796d9cab36d..cf1d9e8dde43 100644 --- a/documentation/testing-docs/test-matrix-for-az-accounts.md +++ b/documentation/testing-docs/test-matrix-for-az-accounts.md @@ -11,7 +11,7 @@ Each test scenario is marked with one priority P0, P1, P2 based on two factors: ### Connect-AzAccount Using Work/School Account -|Scenario\Auth Method|Interactive|Device Code (`-DeviceCode`)|User Name+Password (`-Credential`)|Access Token (`-AccessToken`)|SP Secret (`-ServicePrincipal -Credential`)|SP Cert (`-ServicePrincipal -CertificateThumbprint`)|System MSI (`-Identity`)|User MSI (`-Identity -AccountId`)|User MSI-Func App published by VS Code (`-Identity -AccountId`)| +|Scenario\AuthN Method|Interactive|Device Code (`-DeviceCode`)|User Name+Password (`-Credential`)|Access Token (`-AccessToken`)|SP Secret (`-ServicePrincipal -Credential`)|SP Cert (`-ServicePrincipal -CertificateThumbprint`)|System MSI (`-Identity`)|User MSI (`-Identity -AccountId`)|User MSI-Func App published by VS Code (`-Identity -AccountId`)| |----|----|----|----|----|----|----|----|----|----| |`No Subscrption/Tenant`|P0 (SemiAuto)|P0|P0 (Auto-No)|P0 (SemiAuto-No)|P0 (Auto-No)|P0 (SemiAuto-No)|P0|P0|P0| |`-Subscription sub-id`|P0 (SemiAuto)|P1|P1 (Auto-No)|P1 (SemiAuto-No)|P1 (Auto-No)|P2 (SemiAuto-No)|P1|P1|P1| @@ -35,7 +35,7 @@ Each test scenario is marked with one priority P0, P1, P2 based on two factors: ### Connect-AzAccount Using MSA Account -|Scenario\Auth Method|Interactive|Device Code (`-DeviceCode`)| +|Scenario\AuthN Method|Interactive|Device Code (`-DeviceCode`)| |----|----|----| |`No Subscription/Tenant`|P0 (SemiAuto)|P0| |`-Subscription sub-id`|P2 (SemiAuto)|P2| @@ -51,7 +51,7 @@ Each test scenario is marked with one priority P0, P1, P2 based on two factors: |Token should be auto refreshed for long running operation(> 1 hour)|P2|Please refer `How To Test` section| |Token cache file should be compatible with az|P2|Please refer `How To Test` section| |Service Principal authentication should be successful if http proxy is set|P2|Please refer `How To Test` section| -|FMR scenario(Integrated Windows Auth)|P2|Please refer `How To Test` section| +|FMR scenario(Integrated Windows Authentication)|P2|Please refer `How To Test` section| ### Other Authentication Test Scenario @@ -80,35 +80,34 @@ It should be fine to run these test cases in just one platform, e.g. Windows Pow |Azure China Instance|P2|Will be covered by dedicated team| |Azure German Instance|P2|Will be covered by dedicated team| |ADFS Environment|P1|Ask Azure Stack team to help| -|SAW Machine|P2|| -### PWSH Platforms +### PowerShell Platforms -In theory all the combination of different OS platforms and PWSH versions should be covered, to compromise time effort and platform coverages, we should at least cover the following platforms. (For different versions of pwsh 7, need to cover at least smallest and biggest patch version for each major.minor version, currently it should be 7.0.0, 7.0.3 and 7.1.0. The reason to cover 7.0.0 is the future version of Azure.Core may contain higher version of built-in assemblies of pwsh.) +In theory all the combination of different OS platforms and PowerShell versions should be covered, to compromise time effort and platform coverages, we should at least cover the following platforms. (For different versions of PowerShell 7, need to cover at least smallest and biggest patch version for each major.minor version, currently it should be 7.0.0, 7.0.3 and 7.1.0. The reason to cover 7.0.0 is the future version of Azure.Core may contain higher version of built-in assemblies of PowerShell.) - Windows PowerShell 5.1 -- PWSH 7.0.0 on Windows -- PWSH 7.0.x(latest patch version) on Windows -- PWSH 7.1.0 on Windows -- PWSH 7.0.0 on Ubuntu -- PWSH 7.0.x on Ubuntu -- PWSH 7.1.0 on Ubuntu -- PWSH 7.0.x on MacOS +- PowerShell 7.0.0 on Windows +- PowerShell 7.0.x(latest patch version) on Windows +- PowerShell 7.1.0 on Windows +- PowerShell 7.0.0 on Ubuntu +- PowerShell 7.0.x on Ubuntu +- PowerShell 7.1.0 on Ubuntu +- PowerShell 7.0.x on MacOS - CloudShell - Docker Env for Ubuntu -- Windows PowerShell 5.1 on SAW machine +- Windows PowerShell 5.1 in Constrained Language Mode There's no need to run all tests on each of above platforms, the recommendation is: 1. For `Windows PowerShell 5.1`, run all applicable tests. -2. For `PWSH 7.0.x on Windows`, run tests on columns **Interactive/Device Code/SP Secret** (please refer to test scenario table). +2. For `PowerShell 7.0.x on Windows`, run tests on columns **Interactive/Device Code/SP Secret** (please refer to test scenario table). 3. For other platforms, just run smoke test `Connect-AzAccount`/`Connect-AzAccount -DeviceCode`. -### Auth Code Change Impact +### Authentication Code Change Impact -When there is auth related code change in Az.Accounts, there's no need to run all test scenario on all platforms and environments. In contrast, we may choose to run different test scenario based on different auth code change impact, so that only affected test cases are covered. +When there is authentication related code change in Az.Accounts, there's no need to run all test scenario on all platforms and environments. In contrast, we may choose to run different test scenario based on different authentication code change impact, so that only affected test cases are covered. -#### Just Auth Code Change without Version Upgrade of MSAL/Azure.Identity in Az.Accounts +#### Just Authentication Code Change without Version Upgrade of MSAL/Azure.Identity in Az.Accounts Only need to verify P0 test scenario. @@ -122,7 +121,7 @@ Need to verify P0, P1 and P2 test scenario. ### Partner Teams -If possible, we should provide preview/engineering bits to our partners for verifying: (P2) +If possible, we should provide preview/engineering bits to our partners for verifying their modules: (P2) - Azure Stack team to verify ADFS scenario - Azure Function team @@ -130,7 +129,7 @@ If possible, we should provide preview/engineering bits to our partners for veri ## How To Test -1. How to test LRO +1. How to Test Long Running Operation (LRO) ```powershell New-AzResourceGroupDeployment -Name xxxx -ResourceGroupName xxxx -TemplateFile path-to-template-file @@ -188,23 +187,23 @@ You may save json content below as template file, make sure the value of `parame } ``` -2. How to test Http Proxy +2. How to Test Http Proxy a. Start Fiddler b. Restart Windows PowerShell and run `Connect-AzAccount` c. You should see http request in Fiddler like `https://login.microsoftonline.com/organizations/oauth2/v2.0/token` -3. How to test compatability with az +3. How to Test compatability with az Expect no error happens: a. az login - b. Delete the token cache file (C:\Users\AppData\Local.IdentityService\msal.cache) + b. Delete the token cache file (C:\Users\\AppData\Local.IdentityService\msal.cache) c. Connect-AzAccount d. Get-AzSubscription e. az group list f. Disconnect-AzAccount -4. How to test Integrated Windows Auth +4. How to Test Integrated Windows Authentication a. Start Fiddler b. Restart Windows PowerShell and Connect-AzAccount using your corp account @@ -216,6 +215,6 @@ You may save json content below as template file, make sure the value of `parame c. Although failed to login, but the http reqeust `https://msft.sts.microsoft.com/adfs/services/trust/2005/usernamemixed` should be successful. -5. How to test on SAW machine +5. How to Test in Constrained Language Mode - It would be great if you have real SAW machine. If not, please run PowerShell under [Constrained Language Mode](https://devblogs.microsoft.com/powershell/powershell-constrained-language-mode/) which should have similar effect as on SAW machine. + If you're from MSFT and have SAW machine at hand, you could just run Windows PowerShell on SAW Machine on which constrained language mode is enabled by default. If not, please run PowerShell under [Constrained Language Mode](https://devblogs.microsoft.com/powershell/powershell-constrained-language-mode/).