07_stitch_together_the_results.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 7. Stitch together the results\n",
    "At this point in the tutorial we should expect to see the output images of the style transfer inside your specified blob container. Inside the same container, we would also be able to inspect the logs for each job. \n",
    "\n",
    "In this notebook, we'll wrap the tutorial by downloading the output images of the video style transfer job and stitching the newly generated style transfered images back into a video.\n",
    "\n",
    "```md\n",
    "pytorch\n",
    "├── images/\n",
    "│   ├── orangutan/\n",
    "│   ├── output_orangutan_MM_DD_YYYY_HHMMSS/ [<-- this folder of newly generated images will be downloaded from storage]\n",
    "│   ├── sample_content_images/\n",
    "│   ├── sample_output_images/\n",
    "│   └── style_images/\n",
    "├── video/\n",
    "│   ├── orangutan_processed_without_audio.mp4 [<-- this video file will be generated from the individual frames]\n",
    "│   ├── orangutan_processed_with_audio.mp4 [<-- this video file is created by adding audio back]\n",
    "│   ├── orangutan.mp4\n",
    "│   └── orangutan.mp3\n",
    "├── style_transfer_script.py\n",
    "└── style_transfer_script.log\n",
    "```\n",
    "\n",
    "After running the tutorial, your working directory under `/pytorch` should look as shown above with the newly generated files."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "from IPython.display import HTML\n",
    "%load_ext dotenv\n",
    "%dotenv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First we need to get the name of the output folder where all out style transfered frames are stored. We do this by using the Batch AI CLI. We store the name of the output folder as `output_folder_name` to be used in later cells."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We use `--query [-1].name` to mean: 'get the name of the last experiment'\n",
    "exp_names = !az batchai experiment list -g ${RESOURCE_GROUP} --workspace ${WORKSPACE} --query [-1].name\n",
    "\n",
    "# strip quotation marks\n",
    "exp_name = str(exp_names[0])[1:-1]\n",
    "\n",
    "# replace experiment prefix with output folder prefix\n",
    "exp_name_array = str(exp_name).split('_')\n",
    "exp_name_array[0] = \"{0}_{1}\".format(os.getenv('FS_OUTPUT_DIR_PREFIX'), os.getenv('FS_CONTENT_DIR'))\n",
    "output_folder_name = '_'.join(exp_name_array)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using the `output_folder_name` variable, lets download the frames:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%script env output_folder_name=\"$output_folder_name\" bash\n",
    "cd pytorch/images &&\n",
    "    azcopy \\\n",
    "        --source https://${STORAGE_ACCOUNT_NAME}.blob.core.windows.net/${AZURE_CONTAINER_NAME} \\\n",
    "        --destination . \\\n",
    "        --source-key $STORAGE_ACCOUNT_KEY \\\n",
    "        --include $output_folder_name \\\n",
    "        --recursive"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then lets use __ffmpeg__ to stitch independent frames back into a video."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%script env output_folder_name=\"$output_folder_name\" bash\n",
    "cd pytorch &&\n",
    "    ffmpeg \\\n",
    "        -framerate 30 \\\n",
    "        -i images/$output_folder_name/%05d_${VIDEO_NAME}.jpg \\\n",
    "        -c:v libx264 \\\n",
    "        -profile:v high -crf 20 -pix_fmt yuv420p \\\n",
    "        -y \\\n",
    "        video/${VIDEO_NAME}_processed_without_audio.mp4 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the beginning, we also used __ffmpeg__ to extract out the original audio file. Now that we have our newly generated video, attach the audio file to the new video."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "cd pytorch/video &&\n",
    "    ffmpeg -i ${VIDEO_NAME}_processed_without_audio.mp4 -i ${VIDEO_NAME}.mp3 \\\n",
    "        -map 0:0 -map 1:0 \\\n",
    "        -vcodec copy -acodec copy \\\n",
    "        -y \\\n",
    "        ${VIDEO_NAME}_processed_with_audio.mp4"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lets take a look at the final result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "HTML('\\\n",
    "    <video width=\"360\" height=\"360\" controls> \\\n",
    "         <source src=\"pytorch/video/{0}_processed_with_audio.mp4\" type=\"video/mp4\"> \\\n",
    "    </video>'\\\n",
    "    .format(os.getenv('VIDEO_NAME'))\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusion\n",
    "In this notebook, we downloaded the individually generated frames from the style transfer job back to our local machine and stitched it back together using __ffmpeg__. Now we have our neural style transferred video fully generated!\n",
    "\n",
    "Congratulations, you've finished the tutorial and now have an awesome video to share with friends. \n",
    "\n",
    "To wrap up, lets clean up our resources. Because we created everything in a single resource group, we can just delete that resource group to clean up all resources in Azure. \n",
    "\n",
    "Throughout this tutorial, we've also generated a lot of files in our working directory. To restore this directory to its original state, you can use the file `/cleanup_files.sh`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Delete your resource group:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The dotenv extension is already loaded. To reload it, use:\n",
      "  %reload_ext dotenv\n",
      "\u001b[31mResource group 'js_batch_scoring_dl' could not be found.\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "%load_ext dotenv\n",
    "%dotenv\n",
    "! az group delete --name $RESOURCE_GROUP -y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Run the `.cleanup_files` script"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "./cleanup_files.sh"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [conda env:batchscoringdl]",
   "language": "python",
   "name": "conda-env-batchscoringdl-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}