sdk/python/resources/workspace/workspace.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Managing Azure Machine Learning Workspace\n",
    "\n",
    "**Requirements** - In order to benefit from this tutorial, you will need:\n",
    "- A basic understanding of Machine Learning\n",
    "- An Azure account with an active subscription. [Create an account for free](https://azure.microsoft.com/free/?WT.mc_id=A261C142F)\n",
    "- A python environment\n",
    "- Installed Azure Machine Learning Python SDK v2 - [install instructions](../../README.md) - check the getting started section\n",
    "\n",
    "**Learning Objectives** - By the end of this tutorial, you should be able to:\n",
    "- Create Azure Machine Learning workspace from Python SDK\n",
    "- Get the details of a workspace from Python SDK\n",
    "- Load a workspace from Python SDK using parameters\n",
    "- Write details of a workspace into a config file\n",
    "- Load a workspace from Python SDK using a saved config file\n",
    "\n",
    "**Motivations** - The [workspace](https://docs.microsoft.com/en-us/azure/machine-learning/concept-workspace) is the top-level resource for Azure Machine Learning, providing a centralized place to work with all the artifacts you create when you use Azure Machine Learning. Ability to create and manage workspace is a prerequisite for any activity in Azure Machine Learning."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. Create a workspace\n",
    "\n",
    "## 1.1. Import the required libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import required libraries\n",
    "from azure.ai.ml import MLClient\n",
    "from azure.ai.ml.entities import Workspace\n",
    "from azure.identity import DefaultAzureCredential"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.2 Configure where workspace needs to be created.\n",
    "Before creating a workspace, we need identifier parameters - a subscription and resource group. We will use these parameters to define where the Azure Machine Learning workspace."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "name": "subscription_id"
   },
   "outputs": [],
   "source": [
    "# Enter details of your subscription\n",
    "subscription_id = \"<SUBSCRIPTION_ID>\"\n",
    "resource_group = \"<RESOURCE_GROUP>\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `MLClient` from `azure.ml` will be used to create the workspace. We use the default [default azure authentication](https://docs.microsoft.com/en-us/python/api/azure-identity/azure.identity.defaultazurecredential?view=azure-python) for this tutorial. Check the [configuration notebook](../../jobs/configuration.ipynb) for more details on how to configure credentials and connect to a workspace."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "name": "ml_client"
   },
   "outputs": [],
   "source": [
    "# get a handle to the subscription\n",
    "\n",
    "from azure.ai.ml import MLClient\n",
    "from azure.identity import DefaultAzureCredential\n",
    "\n",
    "ml_client = MLClient(DefaultAzureCredential(), subscription_id, resource_group)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.3 Create a basic workspace\n",
    "To create a basic workspace, we will define the following attributes\n",
    "- `name` - Name of the workspace\n",
    "- `location` - The Azure [location](https://azure.microsoft.com/en-us/global-infrastructure/services/?products=machine-learning-service) for workspace. For e.g. eastus, westus etc.\n",
    "- `display_name` - Display name of the workspace\n",
    "- `description` - Description of the workspace\n",
    "- `hbi_workspace` - Flag to define whether the workspace is a High Impact workspace\n",
    "- `tags` - (Optional) Tags to help search/filter on workspace easily\n",
    "\n",
    "Using the `MLClient` created earlier, we will create the workspace. This command will start workspace creation and provide a confirmation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "name": "basic_workspace_name"
   },
   "outputs": [],
   "source": [
    "# Creating a unique workspace name with current datetime to avoid conflicts\n",
    "from azure.ai.ml.entities import Workspace\n",
    "import datetime\n",
    "\n",
    "basic_workspace_name = \"mlw-basic-prod-\" + datetime.datetime.now().strftime(\n",
    "    \"%Y%m%d%H%M\"\n",
    ")\n",
    "\n",
    "ws_basic = Workspace(\n",
    "    name=basic_workspace_name,\n",
    "    location=\"eastus\",\n",
    "    display_name=\"Basic workspace-example\",\n",
    "    description=\"This example shows how to create a basic workspace\",\n",
    "    hbi_workspace=False,\n",
    "    tags=dict(purpose=\"demo\"),\n",
    ")\n",
    "\n",
    "ws_basic = ml_client.workspaces.begin_create(ws_basic).result()\n",
    "print(ws_basic)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.4 Create a workspace with existing resources\n",
    "When an Azure Machine Learning workspace is created, a default `storage account`, `container registry`, `key vault` and `application insights` are created. However, users can decide not to use the defaults and instead use their own assets for these. In this example we will create a workspace where we will pass in details of existing assets for the following:\n",
    "- storage account\n",
    "- container registry\n",
    "- key vault\n",
    "- application insights\n",
    "\n",
    "Using the `MLClient` created earlier, we will create the workspace. This command will start workspace creation and provide a confirmation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "name": "basic_ex_workspace_name"
   },
   "outputs": [],
   "source": [
    "# Creating a unique workspace name with current datetime to avoid conflicts\n",
    "import datetime\n",
    "from azure.ai.ml.entities import Workspace\n",
    "\n",
    "basic_ex_workspace_name = \"mlw-basicex-prod-\" + datetime.datetime.now().strftime(\n",
    "    \"%Y%m%d%H%M\"\n",
    ")\n",
    "\n",
    "# Change the following variables to resource ids of your existing storage account, key vault, application insights\n",
    "# and container registry. Here we reuse the ones we just created for the basic workspace\n",
    "existing_storage_account = (\n",
    "    # e.g. \"/subscriptions/<SUBSCRIPTION_ID>/resourceGroups/<RESOURCE_GROUP>/providers/Microsoft.Storage/storageAccounts/<STORAGE_ACCOUNT>\"\n",
    "    ws_basic.storage_account\n",
    ")\n",
    "existing_container_registry = (\n",
    "    # e.g. \"/subscriptions/<SUBSCRIPTION_ID>/resourceGroups/<RESOURCE_GROUP>/providers/Microsoft.ContainerRegistry/registries/<CONTAINER_REGISTRY>\"\n",
    "    ws_basic.container_registry\n",
    ")\n",
    "existing_key_vault = (\n",
    "    # e.g. \"/subscriptions/<SUBSCRIPTION_ID>/resourceGroups/<RESOURCE_GROUP>/providers/Microsoft.KeyVault/vaults/<KEY_VAULT>\"\n",
    "    ws_basic.key_vault\n",
    ")\n",
    "existing_application_insights = (\n",
    "    # e.g. \"/subscriptions/<SUBSCRIPTION_ID>/resourceGroups/<RESOURCE_GROUP>/providers/Microsoft.insights/components/<APP_INSIGHTS>\"\n",
    "    ws_basic.application_insights\n",
    ")\n",
    "\n",
    "ws_with_existing_resources = Workspace(\n",
    "    name=basic_ex_workspace_name,\n",
    "    location=\"eastus\",\n",
    "    display_name=\"Bring your own dependent resources-example\",\n",
    "    description=\"This sample specifies a workspace configuration with existing dependent resources\",\n",
    "    storage_account=existing_storage_account,\n",
    "    container_registry=existing_container_registry,\n",
    "    key_vault=existing_key_vault,\n",
    "    application_insights=existing_application_insights,\n",
    "    tags=dict(purpose=\"demonstration\"),\n",
    ")\n",
    "\n",
    "ws_with_existing_resources = ml_client.begin_create_or_update(\n",
    "    ws_with_existing_resources\n",
    ").result()\n",
    "\n",
    "print(ws_with_existing_resources)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.5 Create a workspace for use with Azure Private Link\n",
    "When using private link, your workspace cannot use the regular Azure Container Registry tasks compute for image building. Hence, you must set the image_build_compute property to some other CPU compute cluster name to use for Docker image environment building. You can also specify whether the private link workspace should be accessible over the internet using the public_network_access property. After workspace creation, create a private link endpoint for your workspace. For more details, see [Secure Azure Machine Learning workspace resources using virtual networks (VNets)](https://docs.microsoft.com/en-us/azure/machine-learning/how-to-network-security-overview)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "name": "basic_private_link_workspace_name"
   },
   "outputs": [],
   "source": [
    "# Creating a unique workspace name with current datetime to avoid conflicts\n",
    "import datetime\n",
    "from azure.ai.ml.entities import Workspace\n",
    "\n",
    "basic_private_link_workspace_name = (\n",
    "    \"mlw-privatelink-prod-\" + datetime.datetime.now().strftime(\"%Y%m%d%H%M\")\n",
    ")\n",
    "\n",
    "ws_private = Workspace(\n",
    "    name=basic_private_link_workspace_name,\n",
    "    location=\"eastus\",\n",
    "    display_name=\"Private Link endpoint workspace-example\",\n",
    "    description=\"When using private link, you must set the image_build_compute property to a cluster name to use for Docker image environment building. You can also specify whether the workspace should be accessible over the internet.\",\n",
    "    image_build_compute=\"cpu-compute\",\n",
    "    public_network_access=\"Disabled\",\n",
    "    tags=dict(purpose=\"demonstration\"),\n",
    ")\n",
    "\n",
    "ml_client.workspaces.begin_create(ws_private).result()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. Finding and loading workspace\n",
    "Once created, we will examine how to get a list of workspaces and how to load them\n",
    "\n",
    "## 2.1 Configure MLClient\n",
    "To start with we will initialize the `MLClient` with a subscription ID and Resource Group Name. We use the default [default azure authentication](https://docs.microsoft.com/en-us/python/api/azure-identity/azure.identity.defaultazurecredential?view=azure-python) for this tutorial. Check the [configuration notebook](../../jobs/configuration.ipynb) for more details on how to configure credentials and connect to a workspace."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "name": "my_ml_client"
   },
   "outputs": [],
   "source": [
    "from azure.ai.ml import MLClient\n",
    "from azure.ai.ml.entities import Workspace\n",
    "from azure.identity import DefaultAzureCredential\n",
    "\n",
    "# Enter details of your subscription\n",
    "subscription_id = \"<SUBSCRIPTION_ID>\"\n",
    "resource_group = \"<RESOURCE_GROUP>\"\n",
    "\n",
    "my_ml_client = MLClient(DefaultAzureCredential(), subscription_id, resource_group)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.2 Get a list of workspaces in a resource group\n",
    "The `MLClient` can now be used to retrieve a list of workspaces within a group as shown below. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "name": "ws_name"
   },
   "outputs": [],
   "source": [
    "for ws in my_ml_client.workspaces.list():\n",
    "    print(ws.name, \":\", ws.location, \":\", ws.description)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.3 Get details of a specific workspace with name\n",
    "The `MLClient` can also be used to get the details of a specific workspace."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "name": "ws_location"
   },
   "outputs": [],
   "source": [
    "ws = my_ml_client.workspaces.get(\"<AML_WORKSPACE_NAME>\")\n",
    "# uncomment this line after providing a workspace name above\n",
    "# print(ws.location,\":\", ws.resource_group)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3. Load Workspace\n",
    "The `MLClient` can be used to load a workspace for use. \n",
    "\n",
    "## 3.1 Load a specific workspace using parameters\n",
    "In this example we will load a specific workspace using parameters. We use the default [default azure authentication](https://docs.microsoft.com/en-us/python/api/azure-identity/azure.identity.defaultazurecredential?view=azure-python) for this tutorial. Check the [configuration notebook](../../jobs/configuration.ipynb) for more details on how to configure credentials and connect to a workspace."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "name": "ws"
   },
   "outputs": [],
   "source": [
    "from azure.ai.ml import MLClient\n",
    "from azure.ai.ml.entities import Workspace\n",
    "from azure.identity import DefaultAzureCredential\n",
    "\n",
    "ws = MLClient(\n",
    "    DefaultAzureCredential(),\n",
    "    subscription_id=\"<SUBSCRIPTION_ID>\",\n",
    "    resource_group_name=\"<RESOURCE_GROUP>\",\n",
    "    workspace_name=\"<AML_WORKSPACE_NAME>\",\n",
    ")\n",
    "print(ws)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.2 Load workspace from a config file\n",
    "The config details of a workspace can be saved to a file from the Azure Machine Learning [portal](https://ml.azure.com/). Click on the name of the portal on the top right corner to see the link to save the config file.\n",
    "\n",
    "This config file can be used to load a workspace using `MLClient`. If no path is mentioned, path is defaulted to current folder. If no file name is mentioned, file name will be defaulted to `config.json`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "name": "ws_from_config"
   },
   "outputs": [],
   "source": [
    "import json\n",
    "from azure.ai.ml import MLClient\n",
    "from azure.identity import DefaultAzureCredential\n",
    "\n",
    "# create a demo config file \"my_config.json\" to be used in this example\n",
    "# Data to be written\n",
    "wsconfig = {\n",
    "    \"subscription_id\": subscription_id,\n",
    "    \"resource_group\": resource_group,\n",
    "    \"workspace_name\": ws_basic.name,\n",
    "}\n",
    "with open(\"my_config.json\", \"w\") as outfile:\n",
    "    json.dump(wsconfig, outfile)\n",
    "\n",
    "# read the config from the current directory\n",
    "ws_from_config = MLClient.from_config(\n",
    "    DefaultAzureCredential(), file_name=\"my_config.json\"\n",
    ")\n",
    "print(ws_from_config.workspace_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Clean up resources\n",
    "Clean up the workspaces we created"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "name": "delete"
   },
   "outputs": [],
   "source": [
    "ml_client.workspaces.begin_delete(name=ws_basic.name, delete_dependent_resources=True)\n",
    "ml_client.workspaces.begin_delete(\n",
    "    name=ws_with_existing_resources.name, delete_dependent_resources=False\n",
    ")\n",
    "ml_client.workspaces.begin_delete(name=ws_private.name, delete_dependent_resources=True)"
   ]
  }
 ],
 "metadata": {
  "description": {
   "description": "Create Azure ML workspace"
  },
  "interpreter": {
   "hash": "66962d4c952b5ba37638a017d6cc83bab37d76f69b13c17d86b9f71233a0aa71"
  },
  "kernelspec": {
   "display_name": "Python 3.10 - SDK V2",
   "language": "python",
   "name": "python310-sdkv2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}