Failure: Deploy to ACI

[mkhazen]

Hello
I cloned yesterday evening (4/13/2020) the repo - All worked fine until the deployment of the Model in ACI - I get the following error - it is unable to deploy the environment

2020-04-14T02:15:33.8737563Z ##[section]Starting: Azure ML Model Deploy
2020-04-14T02:15:33.8745958Z ==============================================================================
2020-04-14T02:15:33.8746271Z Task : AzureML Model Deploy
2020-04-14T02:15:33.8746488Z Description : AzureML Deploy Model
2020-04-14T02:15:33.8746688Z Version : 0.8.20
2020-04-14T02:15:33.8746878Z Author : ms-air-aiagility
2020-04-14T02:15:33.8747230Z Help : https://marketplace.visualstudio.com/items?itemName=ms-air-aiagility.private-vss-services-azureml&ssr=false#overview
2020-04-14T02:15:33.8747611Z ==============================================================================
2020-04-14T02:15:34.3223539Z Current Working Directory is : /__w/1/s
2020-04-14T02:15:34.3223879Z Desired Working Directory is : /__w/1/s/diabetes_regression/scoring
2020-04-14T02:15:34.3224183Z Setting Working Directory ......
2020-04-14T02:15:34.3234255Z /__w/1/s/diabetes_regression/scoring /__w/1/s
2020-04-14T02:15:34.3235194Z Current Working Directory : /__w/1/s/diabetes_regression/scoring
2020-04-14T02:15:34.3307592Z ##[warning]Resource file haven't set, can't find loc string for key: AmlCliNotAvailable
2020-04-14T02:15:34.3327400Z [command]/usr/local/envs/mlopspython_ci/bin/az ml -h
2020-04-14T02:15:35.0966305Z az: 'ml' is not in the 'az' command group. See 'az --help'. If the command is from an extension, please make sure the corresponding extension is installed. To learn more about extensions, please visit https://docs.microsoft.com/en-us/cli/azure/azure-cli-extensions-overview
2020-04-14T02:15:35.0966970Z No AzureML CLI installed.
2020-04-14T02:15:35.0989871Z [command]/usr/local/envs/mlopspython_ci/bin/az extension add -n azure-cli-ml
2020-04-14T02:15:52.8194966Z [command]/usr/local/envs/mlopspython_ci/bin/az ml -h
2020-04-14T02:15:54.8364725Z
2020-04-14T02:15:54.8365820Z Group
2020-04-14T02:15:54.8367261Z az ml : Azureml._cli commands.
2020-04-14T02:15:54.8367663Z
2020-04-14T02:15:54.8367808Z Subgroups:
2020-04-14T02:15:54.8368566Z computetarget : Computetarget subgroup commands.
2020-04-14T02:15:54.8368888Z dataset : Commands for managing datasets in Azure Machine Learning Workspace.
2020-04-14T02:15:54.8369242Z datastore : Commands for managing and using datastores with the Azure ML Workspace.
2020-04-14T02:15:54.8369714Z endpoint : Manage machine learning endpoints.
2020-04-14T02:15:54.8369967Z environment : Commands to manage environments.
2020-04-14T02:15:54.8370200Z experiment : Commands to manage experiments.
2020-04-14T02:15:54.8370628Z folder : Folder subgroup commands.
2020-04-14T02:15:54.8370878Z model : Manage machine learning models.
2020-04-14T02:15:54.8371139Z pipeline : Pipeline subgroup commands.
2020-04-14T02:15:54.8371412Z run : Commands for submitting, updating, and monitoring runs.
2020-04-14T02:15:54.8371862Z service : Manage operationalized services.
2020-04-14T02:15:54.8372102Z workspace : Workspace subgroup commands.
2020-04-14T02:15:54.8372255Z
2020-04-14T02:15:54.8372446Z For more specific examples, use: az find "az ml"
2020-04-14T02:15:54.8373019Z
2020-04-14T02:15:54.8373242Z Please let us know how we are doing: https://aka.ms/clihats
2020-04-14T02:15:54.8399320Z [command]/usr/local/envs/mlopspython_ci/bin/az login --service-principal -u *** -p *** --tenant ***
2020-04-14T02:15:57.0783396Z [
2020-04-14T02:15:57.0784022Z {
2020-04-14T02:15:57.0784479Z "cloudName": "AzureCloud",
2020-04-14T02:15:57.0786277Z "homeTenantId": "",
2020-04-14T02:15:57.0786711Z "id": "xxxxxxa",
2020-04-14T02:15:57.0786991Z "isDefault": true,
2020-04-14T02:15:57.0787185Z "managedByTenants": [
2020-04-14T02:15:57.0787338Z {
2020-04-14T02:15:57.0787705Z "tenantId": "xxxxxxx"
2020-04-14T02:15:57.0787904Z }
2020-04-14T02:15:57.0788036Z ],
2020-04-14T02:15:57.0788215Z "name": "Microsoft Azure Internal Consumption",
2020-04-14T02:15:57.0788855Z "state": "Enabled",
2020-04-14T02:15:57.0789443Z "tenantId": "",
2020-04-14T02:15:57.0789779Z "user": {
2020-04-14T02:15:57.0790260Z "name": "***",
2020-04-14T02:15:57.0790462Z "type": "servicePrincipal"
2020-04-14T02:15:57.0790613Z }
2020-04-14T02:15:57.0790720Z }
2020-04-14T02:15:57.0790838Z ]
2020-04-14T02:15:57.0802734Z ##[warning]Resource file haven't set, can't find loc string for key: LoginFailed
2020-04-14T02:15:57.0856107Z [command]/usr/local/envs/mlopspython_ci/bin/az account set --subscription xxxxxxxx
2020-04-14T02:15:59.0069075Z ##[warning]Resource file haven't set, can't find loc string for key: ErrorInSettingUpSubscription
2020-04-14T02:15:59.0072723Z From service connection: resource group = mlopsm-RG workspace = mlopsm-AML-WS
2020-04-14T02:15:59.0096192Z Cannot find result.json for the model profile result.
2020-04-14T02:15:59.0097444Z Command: az ml model deploy -n mlopsm-aci --model diabetes_regression_model.pkl:1 --ic /__w/1/s/diabetes_regression/scoring/inference_config.yml --dc /__w/1/s/diabetes_regression/scoring/deployment_config_aci.yml -g mlopsm-RG -w mlopsm-AML-WS --overwrite
2020-04-14T02:15:59.0178200Z [command]/usr/local/envs/mlopspython_ci/bin/az ml model deploy -n mlopsm-aci --model diabetes_regression_model.pkl:1 --ic /__w/1/s/diabetes_regression/scoring/inference_config.yml --dc /__w/1/s/diabetes_regression/scoring/deployment_config_aci.yml -g mlopsm-RG -w mlopsm-AML-WS --overwrite
2020-04-14T02:16:02.9467185Z {'Azure-cli-ml Version': '1.3.0', 'Error': MlCliError({'Error': 'Error parsing --deploy-config-file. Must be valid JSON or YAML file.', 'Response Content': TypeError("deploy_configuration() got an unexpected keyword argument 'vnet_name'")})}
2020-04-14T02:16:02.9470637Z Output:
2020-04-14T02:16:02.9471782Z ##[error]Error Code: [1]
2020-04-14T02:16:02.9473647Z ##[error]Error: Failed to deploy model(s)
2020-04-14T02:16:02.9502310Z { code: 1,
2020-04-14T02:16:02.9502998Z error: undefined,
2020-04-14T02:16:02.9503916Z stdout: '',
2020-04-14T02:16:02.9505320Z stderr: '{'Azure-cli-ml Version': '1.3.0', 'Error': MlCliError({'Error': 'Error parsing --deploy-config-file. Must be valid JSON or YAML file.', 'Response Content': TypeError("deploy_configuration() got an unexpected keyword argument 'vnet_name'")})}\n' }
2020-04-14T02:16:02.9514703Z Reverting to original directory.....
2020-04-14T02:16:02.9521113Z /__w/1/s
2020-04-14T02:16:03.0094975Z ##[section]Finishing: Azure ML Model Deploy
 

[mouhannadali]

it was working for me 2 days ago and today got the same error

[eedorenko]

Did you change deployment_config_aci.yml? It is complaining on 'vnet_name'.

[martin-weber]

I have the same issue. It was working last Thursday.
The error is in Azure ML Model Deploy:
ERROR: {'Azure-cli-ml Version': '1.3.0', 'Error': MlCliError({'Error': 'Error parsing --deploy-config-file. Must be valid JSON or YAML file.', 'Response Content': TypeError("deploy_configuration() got an unexpected keyword argument 'vnet_name'")})}

[tcare]

AzureML 1.3.0 SDK released and the CLI update seems to have broken the parsing. Trying to see if there's a way to pin the CLI extension version easily.

[tcare]

Doesn't seem like there's an easy way to change the extension version based on how we call the az ml command, so I am trying out an SDK 1.3.0 image to see if it fixes the issue.

[tcare]

The image has been deployed and this should now be fixed with the newest release.