03_BuildDockerImage.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Build Docker Image"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this notebook, we build a docker image that we will later use with Azure ML service as custom docker image. We will push the image to Azure Container Registry (ACR)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from dotenv import get_key, set_key, find_dotenv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "env_path = find_dotenv(raise_error_if_not_found=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Please fill in the following cell with your information."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "parameters"
    ]
   },
   "outputs": [],
   "source": [
    "image_name = \"YOUR_IMAGE_NAME\"  # e.g. torchdet\n",
    "acr_name = \"YOUR_ACR_NAME\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "set_key(env_path, \"image_name\", image_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create docker file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%writefile dockerfile\n",
    "\n",
    "FROM nvidia/cuda:9.0-cudnn7-devel-ubuntu16.04\n",
    "\n",
    "RUN apt-get update && apt-get install -y --no-install-recommends \\\n",
    "        build-essential \\\n",
    "        cmake \\\n",
    "        git \\\n",
    "        curl \\\n",
    "        ca-certificates && \\\n",
    "    rm -rf /var/lib/apt/lists/*\n",
    "\n",
    "ENV PYTHON_VERSION=3.7\n",
    "RUN curl -o ~/miniconda.sh -O  https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh  && \\\n",
    "    chmod +x ~/miniconda.sh && \\\n",
    "    ~/miniconda.sh -b -p /opt/conda && \\\n",
    "    rm ~/miniconda.sh && \\\n",
    "    /opt/conda/bin/conda create -y --name py$PYTHON_VERSION python=$PYTHON_VERSION scipy scikit-learn && \\\n",
    "    /opt/conda/bin/conda clean -ya && \\\n",
    "    ln -s /opt/conda/etc/profile.d/conda.sh /etc/profile.d/conda.sh && \\\n",
    "    echo \". /opt/conda/etc/profile.d/conda.sh\" >> ~/.bashrc && \\ \n",
    "    echo \"conda activate py$PYTHON_VERSION\" >> ~/.bashrc\n",
    "\n",
    "ENV PATH /opt/conda/envs/py$PYTHON_VERSION/bin:$PATH\n",
    "ENV LD_LIBRARY_PATH /opt/conda/envs/py$PYTHON_VERSION/lib:/usr/local/cuda/lib64/:$LD_LIBRARY_PATH\n",
    "        \n",
    "RUN /opt/conda/bin/conda install --name py$PYTHON_VERSION -c pytorch pytorch=1.1.0 torchvision=0.3.0 cudatoolkit=9.0 \\\n",
    "    matplotlib=3.1.0 Cython=0.29.10\n",
    "    \n",
    "# RUN pip install --upgrade pip && \\\n",
    "RUN pip install azureml-sdk[notebooks] && \\\n",
    "    pip install Pillow==6.1 && \\\n",
    "    /opt/conda/bin/conda clean -yt\n",
    "\n",
    "RUN git clone https://github.com/cocodataset/cocoapi.git && \\\n",
    "    cd cocoapi/PythonAPI && make && cd -"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create ACR"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this section, we create ACR to register our container image later. Make sure you see \"provisioningState\": \"Succeeded\" in the outputs of the following cells before you proceed further."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "acr_resource_group = get_key(env_path, \"resource_group\" )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!az acr create --resource-group {acr_resource_group} --name {acr_name} --sku Basic -o table"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "acr_server_name = \"{}.azurecr.io\".format(acr_name)\n",
    "acr_server_name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "set_key(env_path, \"acr_server_name\", acr_server_name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!az acr login --name {acr_name}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!az acr update -n {acr_name} --admin-enabled true -o table"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Build image and push to ACR"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this section, we build our image locally and push it to ACR we created earlier."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "image_tag = acr_server_name + \"/\" + image_name\n",
    "set_key(env_path, \"image_tag\", image_tag)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!docker build -t {image_tag} ."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!docker push {image_tag}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Save ACR credentials"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will save the ACR credentials and use them in the next notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "acr_username = !az acr credential show -n {acr_name} --query username\n",
    "acr_username = acr_username[0][1:-1]\n",
    "set_key(env_path, \"acr_username\", acr_username)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "acr_password = !az acr credential show -n {acr_name} --query passwords[0].value\n",
    "acr_password = acr_password[0][1:-1]\n",
    "set_key(env_path, \"acr_password\", acr_password)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can now move on to the next notebook to [tune the hyperparameters of the object detection model](04_TuneHyperparameters.ipynb)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [conda env:TorchDetectAML]",
   "language": "python",
   "name": "conda-env-TorchDetectAML-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}