[HPC-Cache] On-boarding of PowerShell Cmdlets for HPC Cache service. (A…

…zure#527)

* Add initial structure for AZ HPC Cache PowerShell Cmdlets. (Azure#1)

* Add initial structure for AZ HPC Cache PowerShell Cmdlets.

* Adding empty folder layout.

* Update project reference for Test project of HPC Cache (Azure#2)

* Initial commit of clients / first mock cmdlet (Azure#3)

* Initial commit of clients / first mock cmdlet

* Initial commit of clients / first mock cmdlet

* Rename Client, Remove unnecessary files, return skulist in azure cmdlet

* Remove utilities

* Add classes to wrap .NET SDK type object. (Azure#4)

* Create wrapper for SDK management client (Azure#5)

* Fix GetAzSku wrt client changes.

* Restructure base cmdlet and mangement client class.

* Test Framework, Additional Cmdlets, Sku Object, Formatting Fixes (Azure#6)

* More commands and start of test framework

Move all logic out of hpc cache client to the commandlets

More commands and start of test framework

Updated commands using new client

fix rebase

fix spacing

fix comments

remove vars

* Test Framework working, add comments to code

* Help text files

* Updates to cmdlet New-AzHpcCache

* Update cmdlet Get-AzHpcCache

* Update cmdlet Get-AzHpcCacheUsageModels

* Update cmdlet Get-AzHpcCacheSku

* Update cmdlet Remove-AzHpcCache

* Cmdlet Update-AzHpcCache and help file

* Changes made for code review

* Start-AzHpcCache Cmdlet (Azure#8)

* Add support for Start-AzHpcCache.

* Adds resource designer for the project.

* Adds assembly info for the project.

* Remove unused imports

* Remove .keep files as they are no longer required.

* Use resource designer to handle strings in remove hpc cache cmdlet.

* Fix namespace and add project reference to test project.

* Prefix with this for local references for better code readability.

* Fix argument validation.

* Add synopsis for start hpc cache cmdlet help file.

* Update Sku Test

* Fix formatting based on StyleCop

* Change output type of start cmdlet (Azure#12)

* Flush-AzHpcCache cmdlet (Azure#11)

Add FlushAzHpcCache.cs and Flush-AzHpcCache.md
Update Resources files.
Update AzHpcCache.md to include flush markdown file.
Update psd1 file to include new command.

* Add support for stop HPC cache cmdlet. (Azure#13)

* Add support for stop HPC cache cmdlet.

* Fix typos

* Cmdlet: Remove Storage Target

* Cmdlet: Upgrade-AzHpcCache

* Add AsJob parameter option for action commands (Azure#14)

* UpgradeCache and RemoveStorageTarget edits

* Add support for storage target create. (Azure#17)

* Add PS wrapper classes required for storage targets.

* Add help for create storage target.

* Add new storage target cmdlet.

* Update resource designer for storage target.

* Get-AzHpcCacheStorageTarget cmdlet

* Fix help.md files to have correct messages

* PSHpcCache (Azure#16)

* Update how cache is returned

* Update what is sent into PSHpcCache.cs

* Add enumberate Collection

* Update help text and comments (Azure#19)

* Adds support for Set-AzHpcStorageTarget. (Azure#20)

* Add support for storage target update.

* Add help for Set-AzHpcStorageTarget.

* Add hook for update storage target in PSD file.

* Fix typos in help and comments.

* Change UpdateHpcCache to SetHpcCache. (Azure#21)

* Fix UpdateHpcCache to SetHpcCache

* update help

* Use Patch for Update, not Post - createOrUpdate

* Change getST cmdlet to also get individual ST

* Update test framework to use custom context and fixtures. (Azure#24)

* Modify HpcCacheController to use RMTestBase base class and use HpcCacheTestContext.

* Update test class to use HpcCacheCollection and HpcCacheTestFixture.

* Update PS file to test Get-AzHpcCache by resource group and name.

* Add HpcCacheTestContext.

* Add HpcCacheCollection.

* Add HpcCacheTestFixture.

* Add cache helper.

* Add constants.

* Add support to handle ENV vars from constants.

* Add utilities required for fixture and custom context.

* Adds storage account and storage target fixture. (Azure#25)

* Add CLFS storage target and get storage target support.

* Add StorageTargetName constant.

* Add StorageTargetName ENV var.

* Add Retry method and Aggregate exception handler.

* Add storage account fixture.

* Add storage account helper.

* Add example storage target cmdlet test.

* Update help text in SetAzHpcStorageTarget.cs (Azure#26)

Update help text for cmdlet SetAzHpcCacheStorageTarget.

* Fix help.md files and help messages

* Fix for storage target update cmdlet. (Azure#28)

* Add Cache, StorageTarget, and Operations tests

* If incorrect value is entered, return inner exception body message

* For Get and Remove - return inner exception body message

* Fix formatting / Fix Descriptions / Fix Typos  (Azure#32)

* Fix formatting using stylecon

* Flush cmdlet updates

* Get-AzHpcCache cmdlet updates

* Get-AzHpcCacheSku cmdlet updates

* Get-AzHpcCacheStorageTarget cmdlet updates

* Get-AzHpcCacheUsageModels cmdlet updates

* New-AzHpcCache cmdlet updates

* Remove-AzHpcCache cmdlet updates

* Remove-AzHpcCacheStorageCache cmdlet updates

* Set-AzHpcCache cmdlet updates

* Start-AzHpcCache cmdlet updates

* Stop-AzHpcCache cmdlet updates

* Upgrade-AzHpcCache cmdlet updates

* New-AzHpcCacheStorageTarget and Set-AzHpcCacheStorageTarget cmdlet updates

* Fix mockContext for fixtures. (Azure#34)

* Add required packages to test project.

* Add assembly info for test project.

* Fix references for resource manager.

* Remove constructor overloading & fix mockContext, also fix references for resource manager

* Dispose the context in StorageAccountFixture and fix the mockContext.

* Add storage account name constant (Azure#35)

* Add StorageAccountName constant.

* Fix Context calling

* Add recorded tests (Azure#33)

* Add recorded tests

* Rename folder

* Revert "Add storage account name constant (Azure#35)" (Azure#36)

This reverts commit 2ac58a8.

* Revert "Fix mockContext for fixtures. (Azure#34)" (Azure#37)

This reverts commit 768ba77.

* Fix Context (Azure#38)

* Add assebly for tests project. (Azure#39)

* Remove completed TODO check. (Azure#40)

* Add fake help urls to make CI task pass (Azure#41)

* Remove completed TODO check.

* Add fake online documentation URL to make CI task pass.

* Update help files to fix fake URLS. it does not like fake servie & cmdlet name. (Azure#42)

* Fix CI task related issues (Azure#43)

* Remove unused imports

* Fix package references

* Update session records to use correct API versions.

* Package Changes / Resource Name Completer / Change Log and Solution (Azure#44)

* Update ChangeLog

* move packages to test

* Fix solution file, add ResourceGroupCompleter to cmdlets

* move all packages

* Update changelog to preview

* Add inputobj and resourceId params to Start/Stop/Flush/Upgrade. Update CacheName to name (Azure#45)

* Update Flush cmdlet to have InputObj and ResourceId, Update cacheName to Name, update test

* fix flush comments

* Fix StartAzHpcCache to include input obj and resourceId

* Fix StopAzHpcCache to include input obj and resourceId

* Fix UpgradeAzHpcCache to include input obj and resourceId

* test changes

* fix help message for cmdlets

* Add Alias and resource name completer for get cache & get storage target. (Azure#46)

* Add CacheName as an alias & use ResourceNameCompleter.

* Add StorageTargetName as an alias and use ResourceNameCompleter.

* Alias initialization

* Get storage target support pipeline cache input.

* New and Set cmdlets (Azure#47)

* update tests

* update NewAzHpcCache to have should process

* update Set cmdlet to use obj/resource Id and shouldProcess

* remove resourceId from Set

* update resouce designer

* Update help files (Azure#48)

* Fix reference

* Update help files

* Use CheckIn type for tests. (Azure#49)

* Add CacheName and StorageTargetName alias. (Azure#50)

* Update Alias in help

* Add CacheName alias and help text for parameters.

* Add StorageTargetName alias.

* Formatting changes.

* Add description to psd

* Remove whitespaces.

* rename files for linux (Azure#51)

* Add shouldProcess to required cmdlets and update junction parameter. (Azure#52)

* Add shouldProcess parameter

* Modify tests to honor force parameter

* Update help text

* Update usage model cmdlet name.

* Update help for all cmdlets to include whatIf

Co-authored-by: Rebecca Makar <rebeccalmakar@gmail.com>
Co-authored-by: Rebecca Makar <remakar@microsoft.com>

src/StorageCache/HPCCache.Test/Fixtures/HpcCacheCollection.cs

@@ -0,0 +1,26 @@
+// ----------------------------------------------------------------------------------
+//
+// Copyright Microsoft Corporation
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+// http://www.apache.org/licenses/LICENSE-2.0
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+// ----------------------------------------------------------------------------------
+
+namespace Microsoft.Azure.Commands.HPCCache.Test.Fixtures
+{
+    using Xunit;
+
+    /// <summary>
+    /// This class is used by XUnit.
+    /// </summary>
+    [CollectionDefinition("HpcCacheCollection")]
+    public class HpcCacheCollection : ICollectionFixture<HpcCacheTestFixture>
+    {
+    }
+}

src/StorageCache/HPCCache.Test/Fixtures/HpcCacheTestContext.cs

@@ -0,0 +1,324 @@
+// ----------------------------------------------------------------------------------
+//
+// Copyright Microsoft Corporation
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+// http://www.apache.org/licenses/LICENSE-2.0
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+// ----------------------------------------------------------------------------------
+
+namespace Microsoft.Azure.Commands.HPCCache.Test.Fixtures
+{
+    using System;
+    using System.Collections.Generic;
+    using System.Linq;
+    using Microsoft.Azure.Commands.HPCCache.Test.Utilities;
+    using Microsoft.Azure.Management.Authorization;
+    using Microsoft.Azure.Management.Authorization.Models;
+    using Microsoft.Azure.Management.Internal.Resources;
+    using Microsoft.Azure.Management.Internal.Resources.Models;
+    using Microsoft.Azure.Management.Network;
+    using Microsoft.Azure.Management.Network.Models;
+    using Microsoft.Azure.Management.StorageCache;
+    using Microsoft.Azure.Management.StorageCache.Models;
+    using Microsoft.Rest.Azure;
+    using Microsoft.Rest.ClientRuntime.Azure.TestFramework;
+
+    /// <summary>
+    /// Defines the <see cref="HpcCacheTestContext" />.
+    /// </summary>
+    public class HpcCacheTestContext : IDisposable
+    {
+        /// <summary>
+        /// Defines the mockContext.
+        /// </summary>
+        private readonly MockContext mockContext;
+
+        /// <summary>
+        /// Defines the serviceClientCache.
+        /// </summary>
+        private readonly Dictionary<Type, IDisposable> serviceClientCache = new Dictionary<Type, IDisposable>();
+
+        /// <summary>
+        /// Defines the disposedValue.
+        /// </summary>
+        private bool disposedValue = false;
+
+        /// <summary>
+        /// Initializes a new instance of the <see cref="HpcCacheTestContext"/> class.
+        /// </summary>
+        /// <param name="suiteObject">Class object.</param>
+        /// <param name="methodName">Method name of the calling method.</param>
+        public HpcCacheTestContext(
+            string suiteObject,
+            [System.Runtime.CompilerServices.CallerMemberName]
+            string methodName = ".ctor")
+        {
+            this.mockContext = MockContext.Start(suiteObject, methodName);
+            this.RegisterSubscriptionForResource("Microsoft.StorageCache");
+        }
+
+        /// <summary>
+        /// Initializes a new instance of the <see cref="HpcCacheTestContext"/> class.
+        /// </summary>
+        /// <param name="type">Class type.</param>
+        /// <param name="methodName">Method name of the calling method.</param>
+        public HpcCacheTestContext(
+            Type type,
+            [System.Runtime.CompilerServices.CallerMemberName]
+            string methodName = ".ctor")
+        {
+            this.mockContext = MockContext.Start(type.Name, methodName);
+            this.RegisterSubscriptionForResource("Microsoft.StorageCache");
+        }
+
+        /// <summary>
+        /// Get service client.
+        /// </summary>
+        /// <param name="delegationHandler">HTTP delegation handler.</param>
+        /// <typeparam name="TServiceClient">The type of the service client to return.</typeparam>
+        /// <returns>A management client, created from the current context (environment variables).</returns>
+        public TServiceClient GetClient<TServiceClient>()
+            where TServiceClient : class, IDisposable
+        {
+            if (this.serviceClientCache.TryGetValue(typeof(TServiceClient), out IDisposable clientObject))
+            {
+                return (TServiceClient)clientObject;
+            }
+
+            TServiceClient client = this.mockContext.GetServiceClient<TServiceClient>();
+            this.serviceClientCache.Add(typeof(TServiceClient), client);
+            return client;
+        }
+
+        /// <summary>
+        /// Get or create resource group.
+        /// </summary>
+        /// <param name="resourceGroupName">The name of the resource group.</param>
+        /// <param name="location">Location where resource group is to be created.</param>
+        /// <returns>ResourceGroup object.</returns>
+        public ResourceGroup GetOrCreateResourceGroup(string resourceGroupName, string location)
+        {
+            ResourceManagementClient resourceClient = this.GetClient<ResourceManagementClient>();
+            ResourceGroup resourceGroup = this.GetResourceGroupIfExists(resourceGroupName);
+
+            if (resourceGroup == null)
+            {
+                resourceGroup = resourceClient.ResourceGroups.CreateOrUpdate(
+                resourceGroupName,
+                new ResourceGroup
+                {
+                    Location = location,
+                    Tags = new Dictionary<string, string>() { { resourceGroupName, DateTime.UtcNow.ToString("u") } },
+                });
+            }
+
+            return resourceGroup;
+        }
+
+        /// <summary>
+        /// Get or create virtual network.
+        /// </summary>
+        /// <param name="resourceGroup">Object representing a resource group.</param>
+        /// <param name="virtualNetworkName">The name of the virtual network.</param>
+        /// <returns>VirtualNetwork object.</returns>
+        public VirtualNetwork GetOrCreateVirtualNetwork(ResourceGroup resourceGroup, string virtualNetworkName)
+        {
+            NetworkManagementClient networkManagementClient = this.GetClient<NetworkManagementClient>();
+            VirtualNetwork virtualNetwork = null;
+            try
+            {
+                virtualNetwork = networkManagementClient.VirtualNetworks.Get(resourceGroup.Name, virtualNetworkName);
+            }
+            catch (CloudException ex)
+            {
+                if (ex.Body.Code != "ResourceNotFound")
+                {
+                    throw;
+                }
+            }
+
+            if (virtualNetwork == null)
+            {
+                var vnet = new VirtualNetwork()
+                {
+                    Location = resourceGroup.Location,
+                    AddressSpace = new AddressSpace()
+                    {
+                        AddressPrefixes = new List<string> { "10.1.0.0/16" },
+                    },
+                };
+                virtualNetwork = networkManagementClient.VirtualNetworks.CreateOrUpdate(resourceGroup.Name, virtualNetworkName, vnet);
+            }
+
+            return virtualNetwork;
+        }
+
+        /// <summary>
+        /// Get Or create subnet.
+        /// </summary>
+        /// <param name="resourceGroup">Object representing a resource group.</param>
+        /// <param name="virtualNetwork">Object representing a virtual network.</param>
+        /// <param name="subnetName">The name of the subnet.</param>
+        /// <returns>Subnet object.</returns>
+        public Subnet GetOrCreateSubnet(ResourceGroup resourceGroup, VirtualNetwork virtualNetwork, string subnetName)
+        {
+            NetworkManagementClient networkManagementClient = this.GetClient<NetworkManagementClient>();
+            Subnet subNet = null;
+            try
+            {
+                subNet = networkManagementClient.Subnets.Get(resourceGroup.Name, virtualNetwork.Name, subnetName);
+            }
+            catch (CloudException ex)
+            {
+                if (ex.Body.Code != "NotFound")
+                {
+                    throw;
+                }
+            }
+
+            if (subNet == null)
+            {
+                var snet = new Subnet()
+                {
+                    Name = subnetName,
+                    AddressPrefix = "10.1.0.0/24",
+                };
+
+                subNet = networkManagementClient.Subnets.CreateOrUpdate(resourceGroup.Name, virtualNetwork.Name, subnetName, snet);
+            }
+
+            return subNet;
+        }
+
+        /// <summary>
+        /// Get cache if exists.
+        /// </summary>
+        /// <param name="resourceGroup">Object representing a resource group.</param>
+        /// <param name="cacheName">The name of the cache.</param>
+        /// <returns>Cache object if cache exists else null.</returns>
+        public Cache GetCacheIfExists(ResourceGroup resourceGroup, string cacheName)
+        {
+            StorageCacheManagementClient storagecacheManagementClient = this.GetClient<StorageCacheManagementClient>();
+            storagecacheManagementClient.ApiVersion = Constants.DefaultAPIVersion;
+            try
+            {
+                return storagecacheManagementClient.Caches.Get(resourceGroup.Name, cacheName);
+            }
+            catch (CloudErrorException ex)
+            {
+                if (ex.Body.Error.Code == "ResourceNotFound")
+                {
+                    return null;
+                }
+                else
+                {
+                    throw;
+                }
+            }
+        }
+
+        /// <summary>
+        /// Get resource group if exists.
+        /// </summary>
+        /// <param name="resourceGroupName">The name of the resource group.</param>
+        /// <returns>ResourceGroup object if resource group exists else null.</returns>
+        public ResourceGroup GetResourceGroupIfExists(string resourceGroupName)
+        {
+            ResourceManagementClient resourceManagementClient = this.GetClient<ResourceManagementClient>();
+            try
+            {
+                return resourceManagementClient.ResourceGroups.Get(resourceGroupName);
+            }
+            catch (CloudException ex)
+            {
+                if (ex.Body.Code == "ResourceGroupNotFound")
+                {
+                    return null;
+                }
+                else
+                {
+                    throw;
+                }
+            }
+        }
+
+        /// <summary>
+        /// Register subscription for resource.
+        /// </summary>
+        /// <param name="providerName">Resource provider name.</param>
+        public void RegisterSubscriptionForResource(string providerName)
+        {
+            ResourceManagementClient resourceManagementClient = this.GetClient<ResourceManagementClient>();
+            var reg = resourceManagementClient.Providers.Register(providerName);
+            StorageCacheTestUtilities.ThrowIfTrue(reg == null, $"Failed to register provider {providerName}");
+            var result = resourceManagementClient.Providers.Get(providerName);
+            StorageCacheTestUtilities.ThrowIfTrue(result == null, $"Failed to register provier {providerName}");
+        }
+
+        /// <summary>
+        /// Add role assignment by role name.
+        /// </summary>
+        /// <param name="context">Object representing a HpcCacheTestContext.</param>
+        /// <param name="scope">The scope of the role assignment to create.</param>
+        /// <param name="roleName">The role name.</param>
+        /// <param name="assignmentName">The name of the role assignment to create.</param>
+        public void AddRoleAssignment(HpcCacheTestContext context, string scope, string roleName, string assignmentName)
+        {
+            AuthorizationManagementClient authorizationManagementClient = context.GetClient<AuthorizationManagementClient>();
+            var roleDefinition = authorizationManagementClient.RoleDefinitions
+                .List(scope)
+                .First(role => role.RoleName.StartsWith(roleName));
+
+            var newRoleAssignment = new RoleAssignmentCreateParameters()
+            {
+                RoleDefinitionId = roleDefinition.Id,
+                PrincipalId = Constants.StorageCacheResourceProviderPrincipalId,
+
+                // The principal ID assigned to the role.
+                // This maps to the ID inside the Active Directory.
+                // It can point to a user, service principal, or security group.
+                CanDelegate = false,
+            };
+
+            authorizationManagementClient.RoleAssignments.Create(scope, assignmentName, newRoleAssignment);
+        }
+
+        /// <summary>
+        /// This code added to correctly implement the disposable pattern.
+        /// </summary>
+        public void Dispose()
+        {
+            this.Dispose(true);
+        }
+
+        /// <summary>
+        /// Dispose managed state.
+        /// </summary>
+        /// <param name="disposing">true if we should dispose managed state, otherwise false.</param>
+        protected virtual void Dispose(bool disposing)
+        {
+            if (!this.disposedValue)
+            {
+                if (disposing)
+                {
+                    // Dispose clients
+                    foreach (IDisposable client in this.serviceClientCache.Values)
+                    {
+                        client.Dispose();
+                    }
+
+                    // Dispose context
+                    this.mockContext.Dispose();
+                }
+
+                this.disposedValue = true;
+            }
+        }
+    }
+}