2D/03_Inference-OpenVINO.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Inference example for trained 2D U-Net model on BraTS.\n",
    "In this tutorial, we will use the Intel® Distribution of OpenVINO™ Toolkit to perform inference. We will load the OpenVINO version of the model (IR) and perform inference on a few validation samples from the Decathlon dataset.\n",
    "\n",
    "This tutorial assumes that you have already downloaded and installed [Intel&reg; OpenVINO&trade;](https://software.intel.com/en-us/openvino-toolkit/choose-download) on your computer. These instructions assume version R5 2018."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Inferencing with the Intel® Distribution of OpenVINO™ Toolkit"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to use Intel® OpenVINO™, we need to do a few steps:\n",
    "\n",
    "1. Use the OpenVINO Model Optimizer to convert the above freezed-model to the OpenVINO Intermediate Representation (IR) format\n",
    "1. Load data\n",
    "1. Validation\n",
    "1. Inference time :)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Like we have seen in the training tutorial, we define the Sorensen Dice score coefficient, to measure of the overlap between the prediction and ground truth masks:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "def calc_dice(target, prediction, smooth=0.0001):\n",
    "    \"\"\"\n",
    "    Sorensen Dice coefficient\n",
    "    \"\"\"\n",
    "    prediction = np.round(prediction)\n",
    "\n",
    "    numerator = 2.0 * np.sum(target * prediction) + smooth\n",
    "    denominator = np.sum(target) + np.sum(prediction) + smooth\n",
    "    coef = numerator / denominator\n",
    "\n",
    "    return coef\n",
    "\n",
    "def calc_soft_dice(target, prediction, smooth=0.0001):\n",
    "    \"\"\"\n",
    "    Sorensen (Soft) Dice coefficient - Don't round predictions\n",
    "    \"\"\"\n",
    "    numerator = 2.0 * np.sum(target * prediction) + smooth\n",
    "    denominator = np.sum(target) + np.sum(prediction) + smooth\n",
    "    coef = numerator / denominator\n",
    "\n",
    "    return coef"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 1: Convert the Tensorflow* saved model to the Intermediate Representation (IR) with Intel® OpenVINO™"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this step, we will use the Intel® OpenVINO™'s `model optimizer` to convert the frozen TensorFlow* model to Intel® OpenVINO™ IR format. Once you have a frozen model, you can use the Intel® OpenVINO™ `model optimizer` to create the Intel® OpenVINO™ version."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's first set up a few environment variables for the bash code snippet to work. The code below will create a **FP32** precision model. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to call `mo_ty.py` directly from this script, it is advisable to source the OpenVINO vars in your `.bashrc` script:\n",
    "`source /opt/intel/openvino/bin/setupvars.sh`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "saved_model_dir = \"./output/2d_unet_decathlon\"\n",
    "\n",
    "batch_size = 1\n",
    "\n",
    "import os\n",
    "\n",
    "# Create output directory for images\n",
    "png_directory = \"inference_examples\"\n",
    "if not os.path.exists(png_directory):\n",
    "    os.makedirs(png_directory)\n",
    "\n",
    "if not os.path.exists(saved_model_dir):\n",
    "    print('File {} doesn\\'t exist. Please make sure you have a trained TF model'.format(saved_model_dir))\n",
    "\n",
    "precision=\"FP32\"\n",
    "openvino_model_dir = os.path.join(\"output\", precision)\n",
    "openvino_model_name = \"2d_unet_decathlon\"\n",
    "\n",
    "!mo_tf.py \\\n",
    "      --saved_model_dir $saved_model_dir \\\n",
    "      --batch $batch_size \\\n",
    "      --data_type $precision \\\n",
    "      --output_dir $openvino_model_dir \\\n",
    "      --model_name $openvino_model_name"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you would like to infer on a Neural Compute Stick, you will need to output a **FP16** precision model. To do this, you simply need to change the `--data_type` from FP32 to FP16:\n",
    "\n",
    "```\n",
    "!mo_tf.py \\\n",
    "      --saved_model_dir $saved_model_dir \\\n",
    "      --batch $batch_size \\\n",
    "      --data_type FP16  \\\n",
    "      --output_dir openvino_model_dir/FP16  \\\n",
    "      --model_name 2d_unet_decathlon\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 2: Load Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_path = \"/data/medical_decathlon/Task01_BrainTumour/\"\n",
    "\n",
    "crop_dim=128  # Original resolution (240)\n",
    "batch_size = 128\n",
    "seed=816\n",
    "train_test_split=0.85"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from dataloader import DatasetGenerator, get_decathlon_filelist\n",
    "\n",
    "trainFiles, validateFiles, testFiles = get_decathlon_filelist(data_path=data_path, seed=seed, split=train_test_split)\n",
    "\n",
    "ds_test = DatasetGenerator(testFiles, \n",
    "                           batch_size=batch_size, \n",
    "                           crop_dim=[crop_dim, crop_dim], \n",
    "                           augment=False, \n",
    "                           seed=seed)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 3: Validation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Inferencing with Intel® OpenVINO™"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from openvino.inference_engine import IECore\n",
    "\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `print_stats` prints layer by layer inference times. This is good for profiling which ops are most costly in your model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "path_to_xml_file = \"{}.xml\".format(os.path.join(openvino_model_dir, openvino_model_name))\n",
    "path_to_bin_file = \"{}.bin\".format(os.path.join(openvino_model_dir, openvino_model_name))\n",
    "print(\"OpenVINO IR: {}, {}\".format(path_to_xml_file, path_to_bin_file))\n",
    "\n",
    "ie = IECore()\n",
    "net = ie.read_network(model=path_to_xml_file, weights=path_to_bin_file)\n",
    "\n",
    "input_layer_name = next(iter(net.input_info))\n",
    "output_layer_name = next(iter(net.outputs))\n",
    "print(\"Input layer name = {}\\nOutput layer name = {}\".format(input_layer_name, output_layer_name))\n",
    "\n",
    "exec_net = ie.load_network(network=net, device_name=\"CPU\", num_requests=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "import time \n",
    "\n",
    "def plot_results(ds):\n",
    "    \n",
    "    img, msk = next(ds.ds)\n",
    "\n",
    "    idx = np.argmax(np.sum(np.sum(msk[:,:,:,0], axis=1), axis=1)) # find the slice with the largest tumor\n",
    "\n",
    "    plt.figure(figsize=(10,10))\n",
    "\n",
    "    plt.subplot(1, 3, 1)\n",
    "    plt.imshow(img[idx, :, :, 0], cmap=\"bone\", origin=\"lower\")\n",
    "    plt.title(\"MRI {}\".format(idx), fontsize=20)\n",
    "\n",
    "    plt.subplot(1, 3, 2)\n",
    "    plt.imshow(msk[idx, :, :], cmap=\"bone\", origin=\"lower\")\n",
    "    plt.title(\"Ground truth\", fontsize=20)\n",
    "\n",
    "    plt.subplot(1, 3, 3)\n",
    "\n",
    "    # Predict using the OpenVINO model\n",
    "    # NOTE: OpenVINO expects channels first for input and output\n",
    "    # So we transpose the input and output\n",
    "    start_time = time.time()\n",
    "    res = exec_net.infer({input_layer_name: np.transpose(img[[idx]], [0,3,1,2])})\n",
    "    prediction = np.transpose(res[output_layer_name], [0,2,3,1])    \n",
    "    print(\"Elapsed time = {:.4f} msecs\".format(1000.0*(time.time()-start_time)))\n",
    "\n",
    "    plt.imshow(prediction[0,:,:,0], cmap=\"bone\", origin=\"lower\")\n",
    "    plt.title(\"Prediction\\nDice = {:.4f}\".format(calc_dice(msk[idx, :, :], prediction)), fontsize=20)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Plot the predictions with Matplotlib and save to PNG files"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_results(ds_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "plot_results(ds_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_results(ds_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_results(ds_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_results(ds_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_results(ds_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_results(ds_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Licensed under the Apache License, Version 2.0 (the \"License\"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. SPDX-License-Identifier: EPL-2.0*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Copyright (c) 2019-2020 Intel Corporation*"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}