appservice-kube extension (Azure#3934)

* add appservice-kube extension

* Retrieving function triggers from the function image

* Retrieving function triggers from the function image

* Fix appservice kube list by subscription

* Fix transient error when loading arguments

* Added az webapp up support

* Handled code review comments

* bump up version to 0.1.1

* Use custom sdk code for app service plan command sdk

* Version 0.1.2

* Private registry image support and fixig windows to unix line feed char issue to retrive triggers from a function app

* Removing debugpy.breaspoint() line

* Update custom.py

Removing trailing white spaces

* Update custom.py

Removing trailing white spaces

* fixing the condition when the image is from the public registry with no user name and password supplied

* Update sdk, add BYO AKS cluster to kube create command

* Add version 0.1.3

* Copying getfunctionsjson.sh as part of appservice-kube extension and correcting the lookup path

* Adding version 0.1.4

* fix

* Update version to 0.1.5

* Update kube environments SDK

* Add version 0.1.6

* Update kube environments SDK (using api version 2020-06-01), delete old parameters from commands

* Add version 0.1.7

* remove azure-cli-core dependency

* Add version 0.1.8

* Add latest changes from cli core (except az webapp up)

* Bugfix: if ftp deployment profile doesn't exist, zip deploy fails

* Bugfix: index.json had weird merge conflict

* Bugfix: webapp config container set wasn't setting

* Version 0.1.9

* Add functionapp config container set

* Start referencing main cli

* Functionapp deployment source config-zip command fix (copy over from main CLI for now)

* Functions CLI should not pull docker image

* Plan creation should drop --kube-environment and --kube-sku

* Change kind to 'linux,kubernetes'

* Help text changes

* Publish extension v 0.1.10

* Version 0.1.11, since broke appservice plan create for non-k8se plans

* Allow specifying custom location by name rather than resource ID

* az appservice kube create command

* Change ASP kind back to K8SE, reserved to None

* Publish 0.1.12 whl

* az webapp create doesn't require plan for k8se

* Change app service plan kind to 'linux,kubernetes' and reserved=true if
linux

* Webapp create should pass with ASP kind=linux if SKU is a kube sku

* az functionapp create without plan

* Fix bug where if webapp/functionapp create called to update, if --plan isnt specified it creates new plan

* fix webapp/functionapp create, if calling create again with new custom location/plan edge cases

* Version 0.1.13

* Version 0.1.14 temporary change to stop doing the list in web/func app creates

* Add better help text for appservice create with custom location

* Update appservice SDK for kube extension to 2020-12-01

* Clean up az kube create

* Update to version 0.1.15

* Fix dotnet5 bug

* Converting skus for kubernetes ASPs to the new values

* 0.1.16 version

* Set number of workers to 1 when creating ASP

* Add webapp scale command

* Version 0.1.17 with az webapp scale

* rename az webapp scale param

* Move restart in from main CLI, since api returns 202 rather than 200. Need SDK update

* Change SDK to have extended location envelope manually - need to see why
generating SDK locally is failing

* Pass ExtendedLocationEnvelope to web/functionapp, appservice plan and kube environment creates

* Version 0.1.18

* Remove webapp up from extension

* Use delete_app_settings function from cli core

* Use webapp validator from core CLI

* Use updated update_app_settings function from core CLI

* 0.1.19 version

* Fix regression when using main CLI validators

* 0.1.20 version

* az appservice kube create: better error message when conflict

* Remove vsts_cd_manager

* Version 0.1.21

* Update index.json

* K8se master kube env create validations (#3)

* az kube create: allow using custom location name. Also get location from custom location rather than resource group

* Require cli core 2.26.0 or later

* Version 0.2.0

* Use ExtendedLocation property ob object level (#4)

* Static ip not required property in kube create

* Version 0.2.1

* Fix NoneType object has no attribute custom_location error

* Bugfix: Wrong custom location being used if multiple custom locations exist in diff rg (#5)

* Fix NoneType object has no attribute custom_location error

* If two custom locations with same name, in different resource group sometimes it uses wrong custom location

* Version 0.2.2

* remove old SDK and start 'az appservice kube' rewrite

* rewrite 'az appservice kube show'

* rewrite 'az appservice kube list'

* rewrite 'az appservice kube delete'

* re-add appservice kube create checks and start appservice kube update impl

* raise exception for 'az appservice kube update'

* add 'az appservice kube wait'

* add kube list pagination

* fix 'az appservice create'

* fix 'az appservice plan update'

* fix 'az webapp create'

* fix az webapp show/restart

* remove 'az webapp config container set'

* remove uneeded TODOs

* format results of 'az appservice kube show/list'

* remove TODOs and fix 'az webapp show -s'

* add misc style fixes

* fix more style issues

* revert src/index.json to master version

* add summary

* fix linter issues

* remove out-of-date test

* fix appservice kube CLI linter issues

* fix ASP sku validation

* fix style

* increment appservice-kube release number and fix service_name.json

* fix hardcoded 'verify_linter.py'

* add testing -- WIP

* update codeowners and fix 'az appservice kube list' bug

* remove unfinished tests

* remove unnecessary branching

* remove unnecessary options lists

* use specific CLI error types

* use more specific error type

* fix more error types

* use HttpResponseError

* fix style

* revert version and erase history

Co-authored-by: Eben Carek <ebcarek@microsoft.com>
Co-authored-by: Satish Ranjan <satishra@microsoft.com>
Co-authored-by: Sanchit Mehta <sanchit.mehta602@gmail.com>
Co-authored-by: Eben Carek <ebengc@gmail.com>
Co-authored-by: SatishRanjan <SatishRanjan@users.noreply.github.com>
Co-authored-by: Calvin Chan <calvinch4n@gmail.com>

.github/CODEOWNERS

@@ -136,6 +136,8 @@
 
 /src/kusto/ @ilayr @orhasban @astauben
 
+/src/appservice-kube/ @ebencarek @calcha @StrawnSC
+
 /src/custom-providers/ @jsntcy
 
 /src/costmanagement/ @kairu-ms @jsntcy

src/appservice-kube/HISTORY.rst

@@ -0,0 +1,9 @@
+.. :changelog:
+
+Release History
+===============
+
+
+0.1.0
+++++++
+* Initial public preview release.

src/appservice-kube/README.rst

@@ -0,0 +1,5 @@
+Microsoft Azure CLI 'appservice-kube' Extension
+==========================================
+
+The appservice-kube extension adds support for controlling App Service Kubernetes Environments.
+See here for more information: https://docs.microsoft.com/en-us/azure/app-service/manage-create-arc-environment?tabs=bash

src/appservice-kube/azext_appservice_kube/__init__.py

@@ -0,0 +1,33 @@
+# --------------------------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License. See License.txt in the project root for license information.
+# --------------------------------------------------------------------------------------------
+
+from azure.cli.core import AzCommandsLoader
+
+from azext_appservice_kube._help import helps  # pylint: disable=unused-import
+
+
+class AppserviceCommandsLoader(AzCommandsLoader):
+
+    def __init__(self, cli_ctx=None):
+        from azure.cli.core.commands import CliCommandType
+        from azure.cli.core.profiles import ResourceType
+        appservice_custom = CliCommandType(operations_tmpl='azext_appservice_kube.custom#{}')
+        super().__init__(cli_ctx=cli_ctx,
+                         custom_command_type=appservice_custom,
+                         resource_type=ResourceType.MGMT_APPSERVICE)
+
+    def load_command_table(self, args):
+        super().load_command_table(args)
+        from azext_appservice_kube.commands import load_command_table
+        load_command_table(self, args)
+        return self.command_table
+
+    def load_arguments(self, command):
+        super().load_arguments(command)
+        from azext_appservice_kube._params import load_arguments
+        load_arguments(self, command)
+
+
+COMMAND_LOADER_CLS = AppserviceCommandsLoader

src/appservice-kube/azext_appservice_kube/_appservice_utils.py

@@ -0,0 +1,21 @@
+# --------------------------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License. See License.txt in the project root for license information.
+# --------------------------------------------------------------------------------------------
+
+from ._client_factory import web_client_factory
+
+
+def _generic_site_operation(cli_ctx, resource_group_name, name, operation_name, slot=None,
+                            extra_parameter=None, client=None):
+    client = client or web_client_factory(cli_ctx)
+    operation = getattr(client.web_apps,
+                        operation_name if slot is None else operation_name + '_slot')
+    if slot is None:
+        return (operation(resource_group_name, name)
+                if extra_parameter is None else operation(resource_group_name,
+                                                          name, extra_parameter))
+
+    return (operation(resource_group_name, name, slot)
+            if extra_parameter is None else operation(resource_group_name,
+                                                      name, extra_parameter, slot))

src/appservice-kube/azext_appservice_kube/_client_factory.py

@@ -0,0 +1,54 @@
+# --------------------------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License. See License.txt in the project root for license information.
+# --------------------------------------------------------------------------------------------
+
+from azure.cli.core.commands.client_factory import get_mgmt_service_client
+from azure.cli.command_modules.appservice._client_factory import web_client_factory
+from azure.cli.core.profiles import ResourceType
+
+
+# pylint: disable=inconsistent-return-statements
+def ex_handler_factory(creating_plan=False, no_throw=False):
+    def _polish_bad_errors(ex):
+        import json
+        from azure.cli.core.azclierror import ValidationError
+        try:
+            detail = json.loads(ex.response.text)['Message']
+            if creating_plan:
+                if 'Requested features are not supported in region' in detail:
+                    detail = ("Plan with linux worker is not supported in current region. For " +
+                              "supported regions, please refer to https://docs.microsoft.com/"
+                              "azure/app-service-web/app-service-linux-intro")
+                elif 'Not enough available reserved instance servers to satisfy' in detail:
+                    detail = ("Plan with Linux worker can only be created in a group " +
+                              "which has never contained a Windows worker, and vice versa. " +
+                              "Please use a new resource group. Original error:" + detail)
+            ex = ValidationError(detail)
+        except Exception:  # pylint: disable=broad-except
+            pass
+        if no_throw:
+            return ex
+        raise ex
+    return _polish_bad_errors
+
+
+def customlocation_client_factory(cli_ctx, **_):
+    return get_mgmt_service_client(cli_ctx, ResourceType.MGMT_CUSTOMLOCATION)
+
+
+def resource_client_factory(cli_ctx, **_):
+    return get_mgmt_service_client(cli_ctx, ResourceType.MGMT_RESOURCE_RESOURCES)
+
+
+def cf_plans(cli_ctx, *_):
+    return web_client_factory(cli_ctx).app_service_plans
+
+
+def cf_compute_service(cli_ctx, *_):
+    return get_mgmt_service_client(cli_ctx, ResourceType.MGMT_COMPUTE)
+
+
+def cf_resource_groups(cli_ctx, subscription_id=None):
+    return get_mgmt_service_client(cli_ctx, ResourceType.MGMT_RESOURCE_RESOURCES,
+                                   subscription_id=subscription_id).resource_groups

src/appservice-kube/azext_appservice_kube/_completers.py

@@ -0,0 +1,53 @@
+# --------------------------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License. See License.txt in the project root for license information.
+# --------------------------------------------------------------------------------------------
+
+from azure.cli.core.decorators import Completer
+from ._utils import _get_location_from_resource_group
+from ._constants import KUBE_DEFAULT_SKU
+
+
+@Completer
+def get_vm_size_completion_list(cmd, prefix, namespace):    # pylint: disable=unused-argument
+    """Return the intersection of the VM sizes allowed by the ACS SDK with those returned by the Compute Service."""
+    from azure.mgmt.containerservice.models import ContainerServiceVMSizeTypes
+
+    location = _get_location(cmd.cli_ctx, namespace)
+    result = get_vm_sizes(cmd.cli_ctx, location)
+    return set(r.name for r in result) & set(c.value for c in ContainerServiceVMSizeTypes)
+
+
+@Completer
+def get_kube_sku_completion_list(cmd, prefix, namespace):   # pylint: disable=unused-argument
+    """
+    Return the VM sizes allowed by AKS, or 'ANY'
+    """
+    return get_vm_size_completion_list(cmd, prefix, namespace) & set(KUBE_DEFAULT_SKU)
+
+
+def get_vm_sizes(cli_ctx, location):
+    from ._client_factory import cf_compute_service
+    return cf_compute_service(cli_ctx).virtual_machine_sizes.list(location)
+
+
+def _get_location(cli_ctx, namespace):
+    """
+    Return an Azure location by using an explicit `--location` argument, then by `--resource-group`, and
+    finally by the subscription if neither argument was provided.
+    """
+    from azure.core.exceptions import HttpResponseError
+    from azure.cli.core.commands.parameters import get_one_of_subscription_locations
+
+    location = None
+    if getattr(namespace, 'location', None):
+        location = namespace.location
+    elif getattr(namespace, 'resource_group_name', None):
+        try:
+            location = _get_location_from_resource_group(cli_ctx, namespace.resource_group_name)
+        except HttpResponseError as err:
+            from argcomplete import warn
+            warn('Warning: {}'.format(err.message))
+    if not location:
+        location = get_one_of_subscription_locations(cli_ctx)
+    return location

src/appservice-kube/azext_appservice_kube/_constants.py

@@ -0,0 +1,84 @@
+# --------------------------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License. See License.txt in the project root for license information.
+# --------------------------------------------------------------------------------------------
+
+
+KUBE_DEFAULT_SKU = "K1"
+KUBE_ASP_KIND = "linux,kubernetes"
+KUBE_APP_KIND = "linux,kubernetes,app"
+KUBE_CONTAINER_APP_KIND = 'linux,kubernetes,app,container'
+KUBE_FUNCTION_APP_KIND = 'linux,kubernetes,functionapp'
+KUBE_FUNCTION_CONTAINER_APP_KIND = 'linux,kubernetes,functionapp,container'
+
+LINUX_RUNTIMES = ['dotnet', 'node', 'python', 'java']
+WINDOWS_RUNTIMES = ['dotnet', 'node', 'java', 'powershell']
+
+NODE_VERSION_DEFAULT = "10.14"
+NODE_VERSION_NEWER = "12-lts"
+NODE_EXACT_VERSION_DEFAULT = "10.14.1"
+NETCORE_VERSION_DEFAULT = "2.2"
+DOTNET_VERSION_DEFAULT = "4.7"
+PYTHON_VERSION_DEFAULT = "3.7"
+NETCORE_RUNTIME_NAME = "dotnetcore"
+DOTNET_RUNTIME_NAME = "aspnet"
+NODE_RUNTIME_NAME = "node"
+PYTHON_RUNTIME_NAME = "python"
+OS_DEFAULT = "Windows"
+STATIC_RUNTIME_NAME = "static"  # not an official supported runtime but used for CLI logic
+NODE_VERSIONS = ['4.4', '4.5', '6.2', '6.6', '6.9', '6.11', '8.0', '8.1', '8.9', '8.11', '10.1', '10.10', '10.14']
+PYTHON_VERSIONS = ['3.7', '3.6', '2.7']
+NETCORE_VERSIONS = ['1.0', '1.1', '2.1', '2.2']
+DOTNET_VERSIONS = ['3.5', '4.7']
+
+LINUX_SKU_DEFAULT = "P1V2"
+FUNCTIONS_VERSIONS = ['2', '3']
+
+# functions version : default node version
+FUNCTIONS_VERSION_TO_DEFAULT_NODE_VERSION = {
+    '2': '~10',
+    '3': '~12'
+}
+# functions version -> runtime : default runtime version
+FUNCTIONS_VERSION_TO_DEFAULT_RUNTIME_VERSION = {
+    '2': {
+        'node': '8',
+        'dotnet': '2',
+        'python': '3.7',
+        'java': '8'
+    },
+    '3': {
+        'node': '12',
+        'dotnet': '3',
+        'python': '3.7',
+        'java': '8'
+    }
+}
+# functions version -> runtime : runtime versions
+FUNCTIONS_VERSION_TO_SUPPORTED_RUNTIME_VERSIONS = {
+    '2': {
+        'node': ['8', '10'],
+        'python': ['3.6', '3.7'],
+        'dotnet': ['2'],
+        'java': ['8']
+    },
+    '3': {
+        'node': ['10', '12'],
+        'python': ['3.6', '3.7', '3.8'],
+        'dotnet': ['3'],
+        'java': ['8']
+    }
+}
+# dotnet runtime version : dotnet linuxFxVersion
+DOTNET_RUNTIME_VERSION_TO_DOTNET_LINUX_FX_VERSION = {
+    '2': '2.2',
+    '3': '3.1'
+}
+
+MULTI_CONTAINER_TYPES = ['COMPOSE', 'KUBE']
+
+OS_TYPES = ['Windows', 'Linux']
+
+CONTAINER_APPSETTING_NAMES = ['DOCKER_REGISTRY_SERVER_URL', 'DOCKER_REGISTRY_SERVER_USERNAME',
+                              'DOCKER_REGISTRY_SERVER_PASSWORD', "WEBSITES_ENABLE_APP_SERVICE_STORAGE"]
+APPSETTINGS_TO_MASK = ['DOCKER_REGISTRY_SERVER_PASSWORD']