examples/notebooks/OpenVINO_TensorFlow_classification_example.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "_E679Mu4ilO-"
   },
   "source": [
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/openvinotoolkit/openvino_tensorflow/blob/master/examples/notebooks/OpenVINO_TensorFlow_classification_example.ipynb)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "1s7OK7vW3put"
   },
   "source": [
    "# **Image Classification with OpenVINO™ integration with TensorFlow**:\n",
    "\n",
    "OpenVINO™ integration with TensorFlow is designed for TensorFlow developers who want to get started with OpenVINO™ in their inferencing applications. This product effectively delivers OpenVINO™ inline optimizations which enhance inferencing performance with minimal code modifications. OpenVINO™ integration with TensorFlow accelerates inference across many AI models on a variety of Intel® silicon such as: \n",
    "*   Intel® CPUs\n",
    "*   Intel® integrated and discrete GPUs\n",
    "\n",
    "**Overview**\n",
    "\n",
    "The following code demonstrates acceleration of InceptionV3 using OpenVINO™ integration with TensorFlow. We compare the performance of InceptionV3 with and without OpenVINO™ integration with TensorFlow.\n",
    "\n",
    "\n",
    "InceptionV3 is a convolutional neural network for assisting in image analysis and object detection. \n",
    "We load a pre-trained version of this network trained on more than a million images \n",
    "from the ImageNet database.\n",
    "\n",
    "* Note: While installing OpenVINO™ integration with TensorFlow on windows follow [these](https://github.com/openvinotoolkit/openvino_tensorflow/blob/master/docs/INSTALL.md#windows) instructions. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Getting Started \n",
    "\n",
    "In this section, we install TensorFlow, OpenVINO integration with TensorFlow, other necessary python packages, and import them."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, let's make sure we're running this notebook on a supported Intel CPU"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Note: This works only on Linux! \n",
    "!lscpu | grep name"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "m0NqXHZ9pg82"
   },
   "source": [
    "## Install OpenVINO™ integration with TensorFlow\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "uOk7Kisgh1Nw",
    "outputId": "e19a48ee-8961-478a-8e01-0b4746be5d99"
   },
   "outputs": [],
   "source": [
    "# Upload the required wheel files, models and images in a google drive folder\n",
    "# Uncomment and run the below command to copy them in your current workspace\n",
    "#!cp /content/drive/MyDrive/TF-OV/working_dir_files/* . \n",
    "\n",
    "import os\n",
    "\n",
    "# Enable this variable for runtime inference optimizations\n",
    "os.environ[\"OPENVINO_TF_CONVERT_VARIABLES_TO_CONSTANTS\"] = \"1\"\n",
    "\n",
    "!python3 -m pip install --upgrade pip\n",
    "!python3 -m pip -q install pillow\n",
    "!python3 -m pip -q install -U tensorflow_hub\n",
    "\n",
    "# Install TensorFlow (v2.9.2) and OpenVINO-TensorFlow (v2.3.0)\n",
    "!python3 -m pip -q install tensorflow==2.9.3\n",
    "!python3 -m pip -q install openvino-tensorflow==2.3.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "tDQDLFx69izd"
   },
   "source": [
    "# Now let's infer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "1EImyzDiiHGW"
   },
   "outputs": [],
   "source": [
    "from __future__ import absolute_import, division, print_function, unicode_literals\n",
    "from IPython.display import HTML\n",
    "\n",
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "import openvino_tensorflow as ovtf\n",
    "import tensorflow_hub as hub\n",
    "\n",
    "from PIL import Image\n",
    "import time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "11b1JLB8wFMi"
   },
   "outputs": [],
   "source": [
    "def preprocess_image(file_name,\n",
    "               input_height=299,\n",
    "               input_width=299,\n",
    "               input_mean=0,\n",
    "               input_std=255):\n",
    "    \"\"\"Reads input image file, resizes it to given input height and width\n",
    "       and returns the pre-processed image\n",
    "    \"\"\"  \n",
    "    image = Image.open(file_name)\n",
    "    resized_image = image.resize((input_height,input_width))\n",
    "    resized_image = np.asarray(resized_image, np.float32)\n",
    "    normalized_image = (resized_image - input_mean) / input_std\n",
    "    result = np.expand_dims(normalized_image, 0)\n",
    "    return result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "ilvFI8KTwGEP"
   },
   "outputs": [],
   "source": [
    "def load_labels(label_file):\n",
    "    \"\"\"Parses the label file, assuming that labels are separated with a newline\n",
    "       in the file and returns the list of labels.\n",
    "    \"\"\"  \n",
    "    label = []\n",
    "    proto_as_ascii_lines = tf.io.gfile.GFile(label_file).readlines()\n",
    "    for l in proto_as_ascii_lines:\n",
    "        label.append(l.rstrip())\n",
    "    return label"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "hA3JTYZMwOWK"
   },
   "outputs": [],
   "source": [
    "def infer_openvino_tensorflow(model_file, file_name , input_height, input_width, input_mean, input_std, label_file):\n",
    "    \"\"\"Takes the tensorflow model and all other input parameters as arguments. \n",
    "       Runs inference with the classification model and prints the predictions.\n",
    "    \"\"\"\n",
    "    print(\"CREATE MODEL - BEGIN\")\n",
    "    if model_file==\"\":\n",
    "        model = hub.load(\"https://tfhub.dev/google/imagenet/inception_v3/classification/4\")\n",
    "    else:\n",
    "        model = tf.saved_model.load(model_file)\n",
    "    print(\"CREATE MODEL - END\")\n",
    "\n",
    "    print(\"PREDICTION - BEGIN\") \n",
    "\n",
    "    img =  tf.convert_to_tensor(preprocess_image(\n",
    "            file_name, input_height=input_height, input_width=input_width, input_mean=input_mean, input_std=input_std))\n",
    "\n",
    "    # Warmup iterations\n",
    "    for _ in range(5):\n",
    "        results = model(img)\n",
    "\n",
    "    # Run\n",
    "    elapsed = 0\n",
    "    num_iterations = 10\n",
    "    for _ in range(num_iterations):\n",
    "        start = time.time()\n",
    "        results = model(img)\n",
    "        elapsed += time.time() - start\n",
    "    print('Average Inference time in ms: %f' % ((elapsed * 1000)/num_iterations))\n",
    "            \n",
    "    print(\"PREDICTION - END\")\n",
    "    results = tf.nn.softmax(results).numpy()\n",
    "    \n",
    "    if label_file:\n",
    "        top_5 = tf.argsort(results, axis=-1, direction=\"DESCENDING\")[0][:5].numpy()\n",
    "        labels = load_labels(label_file)\n",
    "        for i,item in enumerate(top_5):\n",
    "            print(labels[item], results[0][top_5][i])\n",
    "    else:\n",
    "        print(\"No label file provided. Cannot print classification results\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "iG39yQY5sZnS"
   },
   "source": [
    "*   Set all the parameters needed for inference\n",
    "*   Enable OpenVINO™ integration with TensorFlow, and set Backend in just a few simple lines of code to boost performace\n",
    "*   Infer the input image \n",
    "*   Output the top 5 predicted classes, and the inference time with OpenVINO™ integration with TensorFlow enabled"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "_Dr7HWdgZds6",
    "outputId": "ce91c42e-b458-4ce0-e3e3-299e82c8c2a7"
   },
   "outputs": [],
   "source": [
    "file_name = tf.keras.utils.get_file(\n",
    "    'grace_hopper.jpg',\n",
    "    \"https://www.tensorflow.org/images/grace_hopper.jpg\")\n",
    "model_file = \"\"\n",
    "label_file = tf.keras.utils.get_file(\n",
    "    'ImageNetLabels.txt',\n",
    "    'https://storage.googleapis.com/download.tensorflow.org/data/ImageNetLabels.txt')\n",
    "input_height = 299\n",
    "input_width = 299\n",
    "input_mean = 0\n",
    "input_std = 255\n",
    "backend_name = \"CPU\"\n",
    "    \n",
    "\n",
    "#Print list of available backends\n",
    "print('Available Backends:')\n",
    "backends_list = ovtf.list_backends()\n",
    "for backend in backends_list:\n",
    "    print(backend)\n",
    "ovtf.set_backend(backend_name)\n",
    "\n",
    "print(\"OpenVINO TensorFlow is enabled\")\n",
    "infer_openvino_tensorflow(model_file, file_name, input_height, input_width, input_mean, input_std, label_file)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0hNon4fhscSk"
   },
   "source": [
    "*   Disable OpenVINO™ integration with TensorFlow to gauge the achieved performance boost\n",
    "*   Infer the input image again\n",
    "*   Output the top 5 predicted classes, and the inference time with OpenVINO™ integration with TensorFlow disabled\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "nQXt492OASb8",
    "outputId": "7a472560-36bd-4465-9977-ebb813db18f0",
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "ovtf.disable() ## Disabling OVTF\n",
    "print(\"OpenVINO TensorFlow is disabled\")\n",
    "infer_openvino_tensorflow(model_file, file_name, input_height, input_width, input_mean, input_std, label_file )\n",
    "ovtf.enable()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9e7e3e6f",
   "metadata": {},
   "source": [
    "## Notices & Disclaimers\n",
    "\n",
    "Performance varies by use, configuration and other factors. Learn more at www.Intel.com/PerformanceIndex​.  \n",
    "\n",
    "Your costs and results may vary. \n",
    "\n",
    "Intel technologies may require enabled hardware, software or service activation.\n"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "Colab_OpenVINO_TensorFlow_classification_example_mod.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3.7.5 ('venv-tf-py3': venv)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.5"
  },
  "vscode": {
   "interpreter": {
    "hash": "a504c19f32482b72ae61ff9399c8576b450cdcef16ee330bf92782770b2a910b"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}