02_download_and_preprocess_the_video.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. Download and preprocess the video\n",
    "In this notebook, we'll download the preprocess the video that we will be applying style transfer to. The output of the tutorial will be the extracted audio file of the video, which will be reused when stitching the video back together, as well as the video separated into individual frames.\n",
    "\n",
    "The video that will be used in this tutorial is also of orangutans (just like the provided sample content images). The video is stored in a publib blob that we will download. However, for this section of the tutorial, you can choose to switch out the video with something of your own choice. Likewise, feel free to switch out the style image instead of using the provided image of a Renior painting.\n",
    "\n",
    "```md\n",
    "pytorch\n",
    "├── images/\n",
    "│   ├── orangutan/ [<-- this folder will contain all individual frames from the video]\n",
    "│   ├── sample_content_images/\n",
    "│   ├── sample_output_images/\n",
    "│   └── style_images/\n",
    "├── video/ [<-- create this new folder to put video content in]\n",
    "│   ├── orangutan.mp4 [<-- this is the downloaded video]\n",
    "│   └── orangutan.mp3 [<-- this is the extracted audio file from the video]\n",
    "├── style_transfer_script.py\n",
    "└── style_transfer_script.log\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Import utilities to help us display images and html embeddings:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.display import HTML\n",
    "import os\n",
    "%load_ext dotenv\n",
    "%dotenv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, create the video folder store your video contents in."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "mkdir pytorch/video"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Download the video that is stored in a public blob storage, located at https://happypathspublic.blob.core.windows.net/videos/orangutan.mp4"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash \n",
    "cd pytorch/video && \n",
    "    wget https://happypathspublic.blob.core.windows.net/videos/orangutan.mp4"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Set the environment variable __VIDEO_NAME__ to the name of the video as this will be used throughout the tutorial for convinience."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "dotenv set VIDEO_NAME orangutan"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lets check out the video so we know what it looks like before hand:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%dotenv\n",
    "HTML('\\\n",
    "    <video width=\"360\" height=\"360\" controls> \\\n",
    "         <source src=\"pytorch/video/{0}.mp4\" type=\"video/mp4\"> \\\n",
    "    </video>'\\\n",
    "    .format(os.getenv('VIDEO_NAME'))\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, use __ffmpeg__ to extract the audio file and save it as orangutan.mp3 under the video directory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash \n",
    "cd pytorch/video &&\n",
    "    ffmpeg -i ${VIDEO_NAME}.mp4 ${VIDEO_NAME}.mp3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, break up the frames of the video into separate individual images. The images will be saved inside a new folder under the `/images` directory, called `/orangutan`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "cd pytorch/images/ &&\n",
    "    mkdir ${VIDEO_NAME} && cd ${VIDEO_NAME} &&\n",
    "    ffmpeg -i ../../video/${VIDEO_NAME}.mp4 %05d_${VIDEO_NAME}.jpg -hide_banner"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To make sure that the frames were successfully extracted, print out the number of images under `/pytorch/images/orangutan`. For the orangutan video, that number should be 823 individual images:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!cd pytorch/images/${VIDEO_NAME} && ls -1 | wc -l"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusion\n",
    "In this notebook, we downloaded the video that we will be applying neural style transfer to, and processed it so that we have the individual frames and audio track as seperate entities. In other scenarios, this can be thought of as preprocessing the data so that it is ready to be scored. \n",
    "\n",
    "Next, we will use the style transfer script from the previous notebook to batch apply style transfer to all extracted frames using Batch AI in Azure. But first, we need to [setup Azure so that we have the appropriate credentials and storage accounts.](./03_setup_azure.ipynb)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [conda env:batchscoringdl]",
   "language": "python",
   "name": "conda-env-batchscoringdl-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}