tutorials/an-introduction/3.pytorch-model-cloud-data.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tutorial: Bring your own data (Part 3 of 3)\n",
    "\n",
    "## Introduction\n",
    "\n",
    "In the previous [Tutorial: Train a model in the cloud](2.pytorch-model.ipynb) article, the CIFAR10 data was downloaded using the builtin `torchvision.datasets.CIFAR10` method in the PyTorch API. However, in many cases you are going to want to use your own data in a remote training run. This article focuses on the workflow you can leverage such that you can work with your own data in Azure Machine Learning. \n",
    "\n",
    "By the end of this tutorial you would have a better understanding of:\n",
    "\n",
    "- How to upload your data to Azure\n",
    "- Best practices for working with cloud data in Azure Machine Learning\n",
    "- Working with command-line arguments\n",
    "\n",
    "---\n",
    "\n",
    "## Your machine learning code\n",
    "\n",
    "By now you have your training script running in Azure Machine Learning, and can monitor the model performance. Let's _parametrize_ the training script by introducing\n",
    "arguments. Using arguments will allow you to easily compare different hyperparmeters.\n",
    "\n",
    "Presently our training script is set to download the CIFAR10 dataset on each run. The python code in [train-with-cloud-data-and-logging.py](src/train-with-cloud-data-and-logging.py) now uses **`argparse` to parametize the script.**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Understanding your machine learning code changes\n",
    "\n",
    "The script `train-with-cloud-data-and-logging.py` has leveraged the `argparse` library to set up the `--data-path`, `--learning-rate`, `--momentum`, and `--epochs` arguments:\n",
    "\n",
    "```python\n",
    "import argparse\n",
    "...\n",
    "parser = argparse.ArgumentParser()\n",
    "parser.add_argument(\"--data-path\", type=str, help=\"Path to the training data\")\n",
    "parser.add_argument(\"--learning-rate\", type=float, default=0.001, help=\"Learning rate for SGD\")\n",
    "parser.add_argument(\"--momentum\", type=float, default=0.9, help=\"Momentum for SGD\")\n",
    "parser.add_argument(\"--epochs\", type=int, default=2, help=\"Number of epochs to train\")\n",
    "args = parser.parse_args()\n",
    "```\n",
    "\n",
    "The script was adapted to update the optimizer to use the user-defined parameters:\n",
    "\n",
    "```python\n",
    "optimizer = optim.SGD(\n",
    "    net.parameters(),\n",
    "    lr=args.learning_rate,     # get learning rate from command-line argument\n",
    "    momentum=args.momentum,    # get momentum from command-line argument\n",
    ")\n",
    "```\n",
    "\n",
    "Similarly the training loop was adapted to update the number of epochs to train to use the user-defined parameters:\n",
    "```python\n",
    "for epoch in range(args.epochs):\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Upload your data to Azure\n",
    "\n",
    "In order to run this script in Azure Machine Learning, you need to make your training data available in Azure. Your Azure Machine Learning workspace comes equipped with a _default_ **Datastore** - an Azure Blob storage account - that you can use to store your training data.\n",
    "\n",
    "> <span style=\"color:purple; font-weight:bold\">! NOTE <br>\n",
    "> Azure Machine Learning allows you to connect other cloud-based datastores that store your data. For more details, see [datastores documentation](./concept-data.md).</span>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install --upgrade torchvision"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.core import Workspace, Dataset\n",
    "from torchvision import datasets\n",
    "\n",
    "ws = Workspace.from_config()\n",
    "\n",
    "datasets.CIFAR10(\".\", download=True)\n",
    "\n",
    "ds = ws.get_default_datastore()\n",
    "ds.upload(\n",
    "    src_dir=\"cifar-10-batches-py\",\n",
    "    target_path=\"tutorials/datasets/cifar-10-batches-py\",\n",
    "    overwrite=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import shutil\n",
    "\n",
    "os.remove(\"cifar-10-python.tar.gz\")\n",
    "shutil.rmtree(\"cifar-10-batches-py\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `target_path` specifies the path on the datastore where the CIFAR10 data will be uploaded."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Submit your machine learning code to Azure Machine Learning\n",
    "\n",
    "As you have done previously, create a new Python control script:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "remote run",
     "batchai",
     "configure run",
     "use notebook widget",
     "get metrics",
     "use datastore"
    ]
   },
   "outputs": [],
   "source": [
    "from azureml.core import (\n",
    "    Experiment,\n",
    "    Environment,\n",
    "    ScriptRunConfig,\n",
    "    Dataset,\n",
    ")\n",
    "from azureml.widgets import RunDetails\n",
    "\n",
    "ds = Dataset.File.from_files(\n",
    "    (ws.get_default_datastore(), \"tutorials/datasets/\")\n",
    ")\n",
    "env = Environment.from_conda_specification(\n",
    "    name=\"pytorch-env-tutorial\", file_path=\"environment.yml\",\n",
    ")\n",
    "\n",
    "src = ScriptRunConfig(\n",
    "    source_directory=\"src\",\n",
    "    script=\"train-with-cloud-data-and-logging.py\",\n",
    "    compute_target=\"cpu-cluster\",\n",
    "    environment=env,\n",
    "    arguments=[\n",
    "        \"--data-path\",\n",
    "        ds.as_mount(),\n",
    "        \"--learning-rate\",\n",
    "        0.003,\n",
    "        \"--momentum\",\n",
    "        0.92,\n",
    "        \"--epochs\",\n",
    "        2,\n",
    "    ],\n",
    ")\n",
    "\n",
    "run = Experiment(\n",
    "    workspace=ws, name=\"an-introduction-train-cloud-data-logging-tutorial\"\n",
    ").submit(src)\n",
    "RunDetails(run).show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Understand the control code\n",
    "\n",
    "The above control code has the following additional code compared to the control code written in [previous tutorial](2.pytorch-model.ipynb)\n",
    "\n",
    "**`ds = Dataset.File.from_files(path=(datastore, 'tutorials/datasets'))`**: A Dataset is used to reference the data you uploaded to the Azure Blob Store. Datasets are an abstraction layer on top of your data that are designed to improve reliability and trustworthiness.\n",
    "\n",
    "\n",
    "**`src = ScriptRunConfig(...)`**: We modified the `ScriptRunConfig` to include a list of arguments that will be passed into training script. We also specified `ds.as_mount()`, which means the directory specified will be _mounted_ to the compute target.\n",
    "\n",
    "## Inspect the 70_driver_log log file\n",
    "\n",
    "In the navigate to the 70_driver_log.txt file - you should see the following output:\n",
    "\n",
    "```\n",
    "Processing 'input'.\n",
    "Processing dataset FileDataset\n",
    "{\n",
    "  \"source\": [\n",
    "    \"('workspaceblobstore', 'datasets/cifar10')\"\n",
    "  ],\n",
    "  \"definition\": [\n",
    "    \"GetDatastoreFiles\"\n",
    "  ],\n",
    "  \"registration\": {\n",
    "    \"id\": \"XXXXX\",\n",
    "    \"name\": null,\n",
    "    \"version\": null,\n",
    "    \"workspace\": \"Workspace.create(name='XXXX', subscription_id='XXXX', resource_group='X')\"\n",
    "  }\n",
    "}\n",
    "Mounting input to /tmp/tmp9kituvp3.\n",
    "Mounted input to /tmp/tmp9kituvp3 as folder.\n",
    "Exit __enter__ of DatasetContextManager\n",
    "Entering Run History Context Manager.\n",
    "Current directory:  /mnt/batch/tasks/shared/LS_root/jobs/dsvm-aml/azureml/tutorial-session-3_1600171983_763c5381/mounts/workspaceblobstore/azureml/tutorial-session-3_1600171983_763c5381\n",
    "Preparing to call script [ train.py ] with arguments: ['--data_path', '$input', '--learning_rate', '0.003', '--momentum', '0.92']\n",
    "After variable expansion, calling script [ train.py ] with arguments: ['--data_path', '/tmp/tmp9kituvp3', '--learning_rate', '0.003', '--momentum', '0.92']\n",
    "\n",
    "Script type = None\n",
    "===== DATA =====\n",
    "DATA PATH: /tmp/tmp9kituvp3\n",
    "LIST FILES IN DATA PATH...\n",
    "['cifar-10-batches-py', 'cifar-10-python.tar.gz']\n",
    "```\n",
    "\n",
    "Notice:\n",
    "\n",
    "1. Azure Machine Learning has mounted the blob store to the compute cluster automatically for you\n",
    "2. The ``ds.as_mount()`` used in the control script resolves to the mount point\n",
    "3. In the machine learning code we include a line to list the directorys under the data directory - you can see the list above"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you're not going to use what you've created here, delete the resources you just created with this quickstart so you don't incur any charges for storage. In the Azure portal, select and delete your resource group."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "outputPrepend"
    ]
   },
   "outputs": [],
   "source": [
    "run.wait_for_completion(show_output=True)"
   ]
  }
 ],
 "metadata": {
  "authors": [
   {
    "name": "samkemp"
   }
  ],
  "celltoolbar": "Edit Metadata",
  "kernelspec": {
   "display_name": "Python 3.8",
   "language": "python",
   "name": "python3.8"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5-final"
  },
  "notice": "Copyright (c) Microsoft Corporation. All rights reserved. Licensed under the MIT License."
 },
 "nbformat": 4,
 "nbformat_minor": 4
}