notebooks/3_classification_from_scratch.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Jupyter Kernel**:\n",
    "\n",
    "* If you are in SageMaker Notebook instance, please make sure you are using **conda_pytorch_latest_p36** kernel\n",
    "* If you are on SageMaker Studio, please make sure you are using **SageMaker JumpStart PyTorch 1.0** kernel\n",
    "\n",
    "**Run All**:\n",
    "\n",
    "* If you are in SageMaker notebook instance, you can go to *Cell tab -> Run All*\n",
    "* If you are in SageMaker Studio, you can go to *Run tab -> Run All Cells*\n",
    "\n",
    "## Training our Classifier from scratch\n",
    "\n",
    "Depending on an application, sometimes image classification is enough. In this notebook, we see how to train and deploy an accurate classifier from scratch on **NEU-CLS** dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import json\n",
    "\n",
    "import numpy as np\n",
    "\n",
    "import sagemaker\n",
    "from sagemaker.s3 import S3Downloader\n",
    "\n",
    "sagemaker_session = sagemaker.Session()\n",
    "sagemaker_config = json.load(open(\"../stack_outputs.json\"))\n",
    "role = sagemaker_config[\"IamRole\"]\n",
    "solution_bucket = sagemaker_config[\"SolutionS3Bucket\"]\n",
    "region = sagemaker_config[\"AWSRegion\"]\n",
    "solution_name = sagemaker_config[\"SolutionName\"]\n",
    "bucket = sagemaker_config[\"S3Bucket\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, we download our **NEU-CLS** dataset from our public S3 bucket"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker.s3 import S3Downloader\n",
    "\n",
    "original_bucket = f\"s3://{solution_bucket}-{region}/{solution_name}\"\n",
    "original_data = f\"{original_bucket}/data/NEU-CLS.zip\"\n",
    "original_sources = f\"{original_bucket}/build/lib/source_dir.tar.gz\"\n",
    "print(\"original data: \")\n",
    "S3Downloader.list(original_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For easiler data processing, depending on the dataset, we unify the class and label names using the scripts from `prepare_data`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "%%time\n",
    "RAW_DATA_PATH= !echo $PWD/raw_neu_cls\n",
    "RAW_DATA_PATH = RAW_DATA_PATH.n\n",
    "DATA_PATH = !echo $PWD/neu_cls\n",
    "DATA_PATH = DATA_PATH.n\n",
    "\n",
    "!mkdir -p $RAW_DATA_PATH\n",
    "!aws s3 cp $original_data $RAW_DATA_PATH\n",
    "\n",
    "!mkdir -p $DATA_PATH\n",
    "!python ../src/prepare_data/neu.py $RAW_DATA_PATH/NEU-CLS.zip $DATA_PATH"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After data preparation, we need to setup some paths that will be used throughtout the notebook"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "prefix = \"neu-cls\"\n",
    "neu_cls_s3 = f\"s3://{bucket}/{prefix}\"\n",
    "sources = f\"{neu_cls_s3}/code/\"\n",
    "train_output = f\"{neu_cls_s3}/output/\"\n",
    "neu_cls_prepared_s3 = f\"{neu_cls_s3}/data/\"\n",
    "!aws s3 sync $DATA_PATH $neu_cls_prepared_s3 --quiet # remove the --quiet flag to view sync outputs\n",
    "s3_checkpoint = f\"{neu_cls_s3}/checkpoint/\"\n",
    "sm_local_checkpoint_dir = \"/opt/ml/checkpoints/\"\n",
    "!aws s3 cp $original_sources $sources"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Visualization\n",
    "\n",
    "Let examine some datasets that we will use later by providing an `ID`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "%matplotlib inline\n",
    "import numpy as np\n",
    "import torch\n",
    "from PIL import Image\n",
    "from torch.utils.data import DataLoader\n",
    "\n",
    "try:\n",
    "    import sagemaker_defect_detection\n",
    "except ImportError:\n",
    "    import sys\n",
    "    from pathlib import Path\n",
    "\n",
    "    ROOT = Path(\"../src\").resolve()\n",
    "    sys.path.insert(0, str(ROOT))\n",
    "\n",
    "from sagemaker_defect_detection import NEUCLS\n",
    "\n",
    "\n",
    "def visualize(image, label, predicted=None):\n",
    "    if not isinstance(image, Image.Image):\n",
    "        image = Image.fromarray(image)\n",
    "\n",
    "    plt.figure(dpi=120)\n",
    "    if predicted is not None:\n",
    "        plt.title(f\"label: {label}, prediction: {predicted}\")\n",
    "    else:\n",
    "        plt.title(f\"label: {label}\")\n",
    "\n",
    "    plt.axis(\"off\")\n",
    "    plt.imshow(image)\n",
    "    return\n",
    "\n",
    "\n",
    "dataset = NEUCLS(DATA_PATH, split=\"train\")\n",
    "ID = 0\n",
    "assert 0 <= ID <= 300\n",
    "image, label = dataset[ID]\n",
    "visualize(image, label)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We train our model with `resnet34` backbone for **50 epochs** and obtains about **99%** test accuracy, f1-score, precision and recall as follows"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "outputPrepend"
    ]
   },
   "outputs": [],
   "source": [
    "%%time\n",
    "import logging\n",
    "from os import path as osp\n",
    "\n",
    "from sagemaker.pytorch import PyTorch\n",
    "\n",
    "NUM_CLASSES = 6\n",
    "BACKBONE = \"resnet34\"\n",
    "assert BACKBONE in [\n",
    "    \"resnet34\",\n",
    "    \"resnet50\",\n",
    "], \"either resnet34 or resnet50. Make sure to be consistent with model_fn in classifier.py\"\n",
    "EPOCHS = 50\n",
    "SEED = 123\n",
    "\n",
    "hyperparameters = {\n",
    "    \"backbone\": BACKBONE,\n",
    "    \"num-classes\": NUM_CLASSES,\n",
    "    \"epochs\": EPOCHS,\n",
    "    \"seed\": SEED,\n",
    "}\n",
    "\n",
    "assert not isinstance(sagemaker_session, sagemaker.LocalSession), \"local session as share memory cannot be altered\"\n",
    "\n",
    "model = PyTorch(\n",
    "    entry_point=\"classifier.py\",\n",
    "    source_dir=osp.join(sources, \"source_dir.tar.gz\"),\n",
    "    role=role,\n",
    "    train_instance_count=1,\n",
    "    train_instance_type=\"ml.g4dn.2xlarge\",\n",
    "    hyperparameters=hyperparameters,\n",
    "    py_version=\"py3\",\n",
    "    framework_version=\"1.5\",\n",
    "    sagemaker_session=sagemaker_session,  # Note: Do not use local session as share memory cannot be altered\n",
    "    output_path=train_output,\n",
    "    checkpoint_s3_uri=s3_checkpoint,\n",
    "    checkpoint_local_path=sm_local_checkpoint_dir,\n",
    "    # container_log_level=logging.DEBUG,\n",
    ")\n",
    "\n",
    "model.fit(neu_cls_prepared_s3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then, we deploy our model which takes about **8 minutes** to complete"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "%%time\n",
    "predictor = model.deploy(\n",
    "    initial_instance_count=1,\n",
    "    instance_type=\"ml.m5.xlarge\",\n",
    "    endpoint_name=sagemaker_config[\"SolutionPrefix\"] + \"-classification-endpoint\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Inference\n",
    "\n",
    "We are ready to test our model by providing some test data and compare the actual labels with the predicted one"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sagemaker_defect_detection import get_transform\n",
    "from sagemaker_defect_detection.utils.visualize import unnormalize_to_hwc\n",
    "\n",
    "ID = 100\n",
    "assert 0 <= ID <= 300\n",
    "test_dataset = NEUCLS(DATA_PATH, split=\"test\", transform=get_transform(\"test\"), seed=SEED)\n",
    "image, label = test_dataset[ID]\n",
    "# SageMaker 1.x doesn't allow_pickle=True by default\n",
    "np_load_old = np.load\n",
    "np.load = lambda *args, **kwargs: np_load_old(*args, allow_pickle=True, **kwargs)\n",
    "outputs = predictor.predict(image.unsqueeze(0).numpy())\n",
    "np.load = np_load_old\n",
    "\n",
    "_, predicted = torch.max(torch.from_numpy(np.array(outputs)), 1)\n",
    "\n",
    "image_unnorm = unnormalize_to_hwc(image)\n",
    "visualize(image_unnorm, label, predicted.item())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Optional: Delete the endpoint and model\n",
    "\n",
    "When you are done with the endpoint, you should clean it up.\n",
    "\n",
    "All of the training jobs, models and endpoints we created can be viewed through the SageMaker console of your AWS account."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictor.delete_model()\n",
    "predictor.delete_endpoint()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.6.10 64-bit",
   "metadata": {
    "interpreter": {
     "hash": "4c1e195df8d07db5ee7a78f454b46c3f2e14214bf8c9489d2db5cf8f372ff2ed"
    }
   },
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.10-final"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}