Keras_EfficientNet_Demo.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Keras EfficientNet Demo.ipynb",
      "version": "0.3.2",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/chiayewken/keras-efficientnets/blob/master/Keras_EfficientNet_Demo.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "RDb8AtkcUyX-"
      },
      "source": [
        "##Keras EfficientNet Demo"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-rrMxoY4pt6j",
        "colab_type": "code",
        "outputId": "3f8c43ed-f5c2-4863-9c16-8066fed31abf",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "!pip install -q --upgrade git+https://github.com/chiayewken/keras-efficientnets.git\n",
        "!git clone -q https://github.com/tensorflow/tpu.git  # for efficientnet preprocessing\n",
        "    \n",
        "# setup path\n",
        "import sys\n",
        "sys.path.append('/content/tpu/models/official/efficientnet')\n",
        "from preprocessing import preprocess_image, IMAGE_SIZE"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "  Building wheel for keras-efficientnets (setup.py) ... \u001b[?25l\u001b[?25hdone\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sqcHstunqx9f",
        "colab_type": "text"
      },
      "source": [
        "###Zero-shot Comparison Against ResNet on CIFAR10 Dataset\n",
        "As a baseline, let's test the quality of the output representations of each network, without any finetuning. \n",
        "\n",
        "A simple linear classifier will then be trained to predict the original labels, using the models' representations as input features.\n",
        "\n",
        "In terms of parameters, the ResNet50 model is more than 4x larger than the EfficientNetB0 (smallest), but the EfficientNet performs almost as well."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Qw6VO57-yFFm",
        "colab_type": "code",
        "outputId": "906b3649-55cb-41fd-bb11-d8ae383ea8eb",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "import tensorflow as tf\n",
        "(x_trn, y_trn), (x_val, y_val) = tf.keras.datasets.cifar10.load_data()\n",
        "\n",
        "BASE_MODEL_TYPE = \"resnet\" #@param [\"efficientnet\", \"resnet\"]\n",
        "print(BASE_MODEL_TYPE)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "resnet\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "6HEL1XIfq0q4",
        "colab_type": "code",
        "collapsed": true,
        "outputId": "2a932028-0565-4b1f-fe9d-917c31c33178",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 6604
        }
      },
      "source": [
        "import tensorflow as tf\n",
        "from keras_efficientnets import EfficientNetB0\n",
        "\n",
        "arch = {\"efficientnet\": EfficientNetB0, \"resnet\": tf.keras.applications.ResNet50}[BASE_MODEL_TYPE]\n",
        "base = arch(\n",
        "    input_shape=(IMAGE_SIZE, IMAGE_SIZE, 3),\n",
        "    include_top=False, \n",
        "    pooling=\"avg\", \n",
        "    weights=\"imagenet\",\n",
        ")\n",
        "base.summary() "
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/resource_variable_ops.py:435: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.\n",
            "Instructions for updating:\n",
            "Colocations handled automatically by placer.\n",
            "Downloading data from https://github.com/fchollet/deep-learning-models/releases/download/v0.2/resnet50_weights_tf_dim_ordering_tf_kernels_notop.h5\n",
            "94658560/94653016 [==============================] - 1s 0us/step\n",
            "__________________________________________________________________________________________________\n",
            "Layer (type)                    Output Shape         Param #     Connected to                     \n",
            "==================================================================================================\n",
            "input_1 (InputLayer)            (None, 224, 224, 3)  0                                            \n",
            "__________________________________________________________________________________________________\n",
            "conv1_pad (ZeroPadding2D)       (None, 230, 230, 3)  0           input_1[0][0]                    \n",
            "__________________________________________________________________________________________________\n",
            "conv1 (Conv2D)                  (None, 112, 112, 64) 9472        conv1_pad[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "bn_conv1 (BatchNormalizationV1) (None, 112, 112, 64) 256         conv1[0][0]                      \n",
            "__________________________________________________________________________________________________\n",
            "activation (Activation)         (None, 112, 112, 64) 0           bn_conv1[0][0]                   \n",
            "__________________________________________________________________________________________________\n",
            "pool1_pad (ZeroPadding2D)       (None, 114, 114, 64) 0           activation[0][0]                 \n",
            "__________________________________________________________________________________________________\n",
            "max_pooling2d (MaxPooling2D)    (None, 56, 56, 64)   0           pool1_pad[0][0]                  \n",
            "__________________________________________________________________________________________________\n",
            "res2a_branch2a (Conv2D)         (None, 56, 56, 64)   4160        max_pooling2d[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "bn2a_branch2a (BatchNormalizati (None, 56, 56, 64)   256         res2a_branch2a[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "activation_1 (Activation)       (None, 56, 56, 64)   0           bn2a_branch2a[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "res2a_branch2b (Conv2D)         (None, 56, 56, 64)   36928       activation_1[0][0]               \n",
            "__________________________________________________________________________________________________\n",
            "bn2a_branch2b (BatchNormalizati (None, 56, 56, 64)   256         res2a_branch2b[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "activation_2 (Activation)       (None, 56, 56, 64)   0           bn2a_branch2b[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "res2a_branch2c (Conv2D)         (None, 56, 56, 256)  16640       activation_2[0][0]               \n",
            "__________________________________________________________________________________________________\n",
            "res2a_branch1 (Conv2D)          (None, 56, 56, 256)  16640       max_pooling2d[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "bn2a_branch2c (BatchNormalizati (None, 56, 56, 256)  1024        res2a_branch2c[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "bn2a_branch1 (BatchNormalizatio (None, 56, 56, 256)  1024        res2a_branch1[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "add (Add)                       (None, 56, 56, 256)  0           bn2a_branch2c[0][0]              \n",
            "                                                                 bn2a_branch1[0][0]               \n",
            "__________________________________________________________________________________________________\n",
            "activation_3 (Activation)       (None, 56, 56, 256)  0           add[0][0]                        \n",
            "__________________________________________________________________________________________________\n",
            "res2b_branch2a (Conv2D)         (None, 56, 56, 64)   16448       activation_3[0][0]               \n",
            "__________________________________________________________________________________________________\n",
            "bn2b_branch2a (BatchNormalizati (None, 56, 56, 64)   256         res2b_branch2a[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "activation_4 (Activation)       (None, 56, 56, 64)   0           bn2b_branch2a[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "res2b_branch2b (Conv2D)         (None, 56, 56, 64)   36928       activation_4[0][0]               \n",
            "__________________________________________________________________________________________________\n",
            "bn2b_branch2b (BatchNormalizati (None, 56, 56, 64)   256         res2b_branch2b[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "activation_5 (Activation)       (None, 56, 56, 64)   0           bn2b_branch2b[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "res2b_branch2c (Conv2D)         (None, 56, 56, 256)  16640       activation_5[0][0]               \n",
            "__________________________________________________________________________________________________\n",
            "bn2b_branch2c (BatchNormalizati (None, 56, 56, 256)  1024        res2b_branch2c[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "add_1 (Add)                     (None, 56, 56, 256)  0           bn2b_branch2c[0][0]              \n",
            "                                                                 activation_3[0][0]               \n",
            "__________________________________________________________________________________________________\n",
            "activation_6 (Activation)       (None, 56, 56, 256)  0           add_1[0][0]                      \n",
            "__________________________________________________________________________________________________\n",
            "res2c_branch2a (Conv2D)         (None, 56, 56, 64)   16448       activation_6[0][0]               \n",
            "__________________________________________________________________________________________________\n",
            "bn2c_branch2a (BatchNormalizati (None, 56, 56, 64)   256         res2c_branch2a[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "activation_7 (Activation)       (None, 56, 56, 64)   0           bn2c_branch2a[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "res2c_branch2b (Conv2D)         (None, 56, 56, 64)   36928       activation_7[0][0]               \n",
            "__________________________________________________________________________________________________\n",
            "bn2c_branch2b (BatchNormalizati (None, 56, 56, 64)   256         res2c_branch2b[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "activation_8 (Activation)       (None, 56, 56, 64)   0           bn2c_branch2b[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "res2c_branch2c (Conv2D)         (None, 56, 56, 256)  16640       activation_8[0][0]               \n",
            "__________________________________________________________________________________________________\n",
            "bn2c_branch2c (BatchNormalizati (None, 56, 56, 256)  1024        res2c_branch2c[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "add_2 (Add)                     (None, 56, 56, 256)  0           bn2c_branch2c[0][0]              \n",
            "                                                                 activation_6[0][0]               \n",
            "__________________________________________________________________________________________________\n",
            "activation_9 (Activation)       (None, 56, 56, 256)  0           add_2[0][0]                      \n",
            "__________________________________________________________________________________________________\n",
            "res3a_branch2a (Conv2D)         (None, 28, 28, 128)  32896       activation_9[0][0]               \n",
            "__________________________________________________________________________________________________\n",
            "bn3a_branch2a (BatchNormalizati (None, 28, 28, 128)  512         res3a_branch2a[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "activation_10 (Activation)      (None, 28, 28, 128)  0           bn3a_branch2a[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "res3a_branch2b (Conv2D)         (None, 28, 28, 128)  147584      activation_10[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "bn3a_branch2b (BatchNormalizati (None, 28, 28, 128)  512         res3a_branch2b[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "activation_11 (Activation)      (None, 28, 28, 128)  0           bn3a_branch2b[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "res3a_branch2c (Conv2D)         (None, 28, 28, 512)  66048       activation_11[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "res3a_branch1 (Conv2D)          (None, 28, 28, 512)  131584      activation_9[0][0]               \n",
            "__________________________________________________________________________________________________\n",
            "bn3a_branch2c (BatchNormalizati (None, 28, 28, 512)  2048        res3a_branch2c[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "bn3a_branch1 (BatchNormalizatio (None, 28, 28, 512)  2048        res3a_branch1[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "add_3 (Add)                     (None, 28, 28, 512)  0           bn3a_branch2c[0][0]              \n",
            "                                                                 bn3a_branch1[0][0]               \n",
            "__________________________________________________________________________________________________\n",
            "activation_12 (Activation)      (None, 28, 28, 512)  0           add_3[0][0]                      \n",
            "__________________________________________________________________________________________________\n",
            "res3b_branch2a (Conv2D)         (None, 28, 28, 128)  65664       activation_12[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "bn3b_branch2a (BatchNormalizati (None, 28, 28, 128)  512         res3b_branch2a[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "activation_13 (Activation)      (None, 28, 28, 128)  0           bn3b_branch2a[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "res3b_branch2b (Conv2D)         (None, 28, 28, 128)  147584      activation_13[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "bn3b_branch2b (BatchNormalizati (None, 28, 28, 128)  512         res3b_branch2b[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "activation_14 (Activation)      (None, 28, 28, 128)  0           bn3b_branch2b[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "res3b_branch2c (Conv2D)         (None, 28, 28, 512)  66048       activation_14[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "bn3b_branch2c (BatchNormalizati (None, 28, 28, 512)  2048        res3b_branch2c[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "add_4 (Add)                     (None, 28, 28, 512)  0           bn3b_branch2c[0][0]              \n",
            "                                                                 activation_12[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "activation_15 (Activation)      (None, 28, 28, 512)  0           add_4[0][0]                      \n",
            "__________________________________________________________________________________________________\n",
            "res3c_branch2a (Conv2D)         (None, 28, 28, 128)  65664       activation_15[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "bn3c_branch2a (BatchNormalizati (None, 28, 28, 128)  512         res3c_branch2a[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "activation_16 (Activation)      (None, 28, 28, 128)  0           bn3c_branch2a[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "res3c_branch2b (Conv2D)         (None, 28, 28, 128)  147584      activation_16[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "bn3c_branch2b (BatchNormalizati (None, 28, 28, 128)  512         res3c_branch2b[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "activation_17 (Activation)      (None, 28, 28, 128)  0           bn3c_branch2b[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "res3c_branch2c (Conv2D)         (None, 28, 28, 512)  66048       activation_17[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "bn3c_branch2c (BatchNormalizati (None, 28, 28, 512)  2048        res3c_branch2c[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "add_5 (Add)                     (None, 28, 28, 512)  0           bn3c_branch2c[0][0]              \n",
            "                                                                 activation_15[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "activation_18 (Activation)      (None, 28, 28, 512)  0           add_5[0][0]                      \n",
            "__________________________________________________________________________________________________\n",
            "res3d_branch2a (Conv2D)         (None, 28, 28, 128)  65664       activation_18[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "bn3d_branch2a (BatchNormalizati (None, 28, 28, 128)  512         res3d_branch2a[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "activation_19 (Activation)      (None, 28, 28, 128)  0           bn3d_branch2a[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "res3d_branch2b (Conv2D)         (None, 28, 28, 128)  147584      activation_19[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "bn3d_branch2b (BatchNormalizati (None, 28, 28, 128)  512         res3d_branch2b[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "activation_20 (Activation)      (None, 28, 28, 128)  0           bn3d_branch2b[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "res3d_branch2c (Conv2D)         (None, 28, 28, 512)  66048       activation_20[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "bn3d_branch2c (BatchNormalizati (None, 28, 28, 512)  2048        res3d_branch2c[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "add_6 (Add)                     (None, 28, 28, 512)  0           bn3d_branch2c[0][0]              \n",
            "                                                                 activation_18[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "activation_21 (Activation)      (None, 28, 28, 512)  0           add_6[0][0]                      \n",
            "__________________________________________________________________________________________________\n",
            "res4a_branch2a (Conv2D)         (None, 14, 14, 256)  131328      activation_21[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "bn4a_branch2a (BatchNormalizati (None, 14, 14, 256)  1024        res4a_branch2a[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "activation_22 (Activation)      (None, 14, 14, 256)  0           bn4a_branch2a[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "res4a_branch2b (Conv2D)         (None, 14, 14, 256)  590080      activation_22[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "bn4a_branch2b (BatchNormalizati (None, 14, 14, 256)  1024        res4a_branch2b[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "activation_23 (Activation)      (None, 14, 14, 256)  0           bn4a_branch2b[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "res4a_branch2c (Conv2D)         (None, 14, 14, 1024) 263168      activation_23[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "res4a_branch1 (Conv2D)          (None, 14, 14, 1024) 525312      activation_21[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "bn4a_branch2c (BatchNormalizati (None, 14, 14, 1024) 4096        res4a_branch2c[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "bn4a_branch1 (BatchNormalizatio (None, 14, 14, 1024) 4096        res4a_branch1[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "add_7 (Add)                     (None, 14, 14, 1024) 0           bn4a_branch2c[0][0]              \n",
            "                                                                 bn4a_branch1[0][0]               \n",
            "__________________________________________________________________________________________________\n",
            "activation_24 (Activation)      (None, 14, 14, 1024) 0           add_7[0][0]                      \n",
            "__________________________________________________________________________________________________\n",
            "res4b_branch2a (Conv2D)         (None, 14, 14, 256)  262400      activation_24[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "bn4b_branch2a (BatchNormalizati (None, 14, 14, 256)  1024        res4b_branch2a[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "activation_25 (Activation)      (None, 14, 14, 256)  0           bn4b_branch2a[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "res4b_branch2b (Conv2D)         (None, 14, 14, 256)  590080      activation_25[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "bn4b_branch2b (BatchNormalizati (None, 14, 14, 256)  1024        res4b_branch2b[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "activation_26 (Activation)      (None, 14, 14, 256)  0           bn4b_branch2b[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "res4b_branch2c (Conv2D)         (None, 14, 14, 1024) 263168      activation_26[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "bn4b_branch2c (BatchNormalizati (None, 14, 14, 1024) 4096        res4b_branch2c[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "add_8 (Add)                     (None, 14, 14, 1024) 0           bn4b_branch2c[0][0]              \n",
            "                                                                 activation_24[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "activation_27 (Activation)      (None, 14, 14, 1024) 0           add_8[0][0]                      \n",
            "__________________________________________________________________________________________________\n",
            "res4c_branch2a (Conv2D)         (None, 14, 14, 256)  262400      activation_27[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "bn4c_branch2a (BatchNormalizati (None, 14, 14, 256)  1024        res4c_branch2a[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "activation_28 (Activation)      (None, 14, 14, 256)  0           bn4c_branch2a[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "res4c_branch2b (Conv2D)         (None, 14, 14, 256)  590080      activation_28[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "bn4c_branch2b (BatchNormalizati (None, 14, 14, 256)  1024        res4c_branch2b[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "activation_29 (Activation)      (None, 14, 14, 256)  0           bn4c_branch2b[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "res4c_branch2c (Conv2D)         (None, 14, 14, 1024) 263168      activation_29[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "bn4c_branch2c (BatchNormalizati (None, 14, 14, 1024) 4096        res4c_branch2c[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "add_9 (Add)                     (None, 14, 14, 1024) 0           bn4c_branch2c[0][0]              \n",
            "                                                                 activation_27[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "activation_30 (Activation)      (None, 14, 14, 1024) 0           add_9[0][0]                      \n",
            "__________________________________________________________________________________________________\n",
            "res4d_branch2a (Conv2D)         (None, 14, 14, 256)  262400      activation_30[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "bn4d_branch2a (BatchNormalizati (None, 14, 14, 256)  1024        res4d_branch2a[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "activation_31 (Activation)      (None, 14, 14, 256)  0           bn4d_branch2a[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "res4d_branch2b (Conv2D)         (None, 14, 14, 256)  590080      activation_31[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "bn4d_branch2b (BatchNormalizati (None, 14, 14, 256)  1024        res4d_branch2b[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "activation_32 (Activation)      (None, 14, 14, 256)  0           bn4d_branch2b[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "res4d_branch2c (Conv2D)         (None, 14, 14, 1024) 263168      activation_32[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "bn4d_branch2c (BatchNormalizati (None, 14, 14, 1024) 4096        res4d_branch2c[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "add_10 (Add)                    (None, 14, 14, 1024) 0           bn4d_branch2c[0][0]              \n",
            "                                                                 activation_30[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "activation_33 (Activation)      (None, 14, 14, 1024) 0           add_10[0][0]                     \n",
            "__________________________________________________________________________________________________\n",
            "res4e_branch2a (Conv2D)         (None, 14, 14, 256)  262400      activation_33[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "bn4e_branch2a (BatchNormalizati (None, 14, 14, 256)  1024        res4e_branch2a[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "activation_34 (Activation)      (None, 14, 14, 256)  0           bn4e_branch2a[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "res4e_branch2b (Conv2D)         (None, 14, 14, 256)  590080      activation_34[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "bn4e_branch2b (BatchNormalizati (None, 14, 14, 256)  1024        res4e_branch2b[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "activation_35 (Activation)      (None, 14, 14, 256)  0           bn4e_branch2b[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "res4e_branch2c (Conv2D)         (None, 14, 14, 1024) 263168      activation_35[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "bn4e_branch2c (BatchNormalizati (None, 14, 14, 1024) 4096        res4e_branch2c[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "add_11 (Add)                    (None, 14, 14, 1024) 0           bn4e_branch2c[0][0]              \n",
            "                                                                 activation_33[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "activation_36 (Activation)      (None, 14, 14, 1024) 0           add_11[0][0]                     \n",
            "__________________________________________________________________________________________________\n",
            "res4f_branch2a (Conv2D)         (None, 14, 14, 256)  262400      activation_36[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "bn4f_branch2a (BatchNormalizati (None, 14, 14, 256)  1024        res4f_branch2a[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "activation_37 (Activation)      (None, 14, 14, 256)  0           bn4f_branch2a[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "res4f_branch2b (Conv2D)         (None, 14, 14, 256)  590080      activation_37[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "bn4f_branch2b (BatchNormalizati (None, 14, 14, 256)  1024        res4f_branch2b[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "activation_38 (Activation)      (None, 14, 14, 256)  0           bn4f_branch2b[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "res4f_branch2c (Conv2D)         (None, 14, 14, 1024) 263168      activation_38[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "bn4f_branch2c (BatchNormalizati (None, 14, 14, 1024) 4096        res4f_branch2c[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "add_12 (Add)                    (None, 14, 14, 1024) 0           bn4f_branch2c[0][0]              \n",
            "                                                                 activation_36[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "activation_39 (Activation)      (None, 14, 14, 1024) 0           add_12[0][0]                     \n",
            "__________________________________________________________________________________________________\n",
            "res5a_branch2a (Conv2D)         (None, 7, 7, 512)    524800      activation_39[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "bn5a_branch2a (BatchNormalizati (None, 7, 7, 512)    2048        res5a_branch2a[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "activation_40 (Activation)      (None, 7, 7, 512)    0           bn5a_branch2a[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "res5a_branch2b (Conv2D)         (None, 7, 7, 512)    2359808     activation_40[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "bn5a_branch2b (BatchNormalizati (None, 7, 7, 512)    2048        res5a_branch2b[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "activation_41 (Activation)      (None, 7, 7, 512)    0           bn5a_branch2b[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "res5a_branch2c (Conv2D)         (None, 7, 7, 2048)   1050624     activation_41[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "res5a_branch1 (Conv2D)          (None, 7, 7, 2048)   2099200     activation_39[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "bn5a_branch2c (BatchNormalizati (None, 7, 7, 2048)   8192        res5a_branch2c[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "bn5a_branch1 (BatchNormalizatio (None, 7, 7, 2048)   8192        res5a_branch1[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "add_13 (Add)                    (None, 7, 7, 2048)   0           bn5a_branch2c[0][0]              \n",
            "                                                                 bn5a_branch1[0][0]               \n",
            "__________________________________________________________________________________________________\n",
            "activation_42 (Activation)      (None, 7, 7, 2048)   0           add_13[0][0]                     \n",
            "__________________________________________________________________________________________________\n",
            "res5b_branch2a (Conv2D)         (None, 7, 7, 512)    1049088     activation_42[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "bn5b_branch2a (BatchNormalizati (None, 7, 7, 512)    2048        res5b_branch2a[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "activation_43 (Activation)      (None, 7, 7, 512)    0           bn5b_branch2a[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "res5b_branch2b (Conv2D)         (None, 7, 7, 512)    2359808     activation_43[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "bn5b_branch2b (BatchNormalizati (None, 7, 7, 512)    2048        res5b_branch2b[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "activation_44 (Activation)      (None, 7, 7, 512)    0           bn5b_branch2b[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "res5b_branch2c (Conv2D)         (None, 7, 7, 2048)   1050624     activation_44[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "bn5b_branch2c (BatchNormalizati (None, 7, 7, 2048)   8192        res5b_branch2c[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "add_14 (Add)                    (None, 7, 7, 2048)   0           bn5b_branch2c[0][0]              \n",
            "                                                                 activation_42[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "activation_45 (Activation)      (None, 7, 7, 2048)   0           add_14[0][0]                     \n",
            "__________________________________________________________________________________________________\n",
            "res5c_branch2a (Conv2D)         (None, 7, 7, 512)    1049088     activation_45[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "bn5c_branch2a (BatchNormalizati (None, 7, 7, 512)    2048        res5c_branch2a[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "activation_46 (Activation)      (None, 7, 7, 512)    0           bn5c_branch2a[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "res5c_branch2b (Conv2D)         (None, 7, 7, 512)    2359808     activation_46[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "bn5c_branch2b (BatchNormalizati (None, 7, 7, 512)    2048        res5c_branch2b[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "activation_47 (Activation)      (None, 7, 7, 512)    0           bn5c_branch2b[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "res5c_branch2c (Conv2D)         (None, 7, 7, 2048)   1050624     activation_47[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "bn5c_branch2c (BatchNormalizati (None, 7, 7, 2048)   8192        res5c_branch2c[0][0]             \n",
            "__________________________________________________________________________________________________\n",
            "add_15 (Add)                    (None, 7, 7, 2048)   0           bn5c_branch2c[0][0]              \n",
            "                                                                 activation_45[0][0]              \n",
            "__________________________________________________________________________________________________\n",
            "activation_48 (Activation)      (None, 7, 7, 2048)   0           add_15[0][0]                     \n",
            "__________________________________________________________________________________________________\n",
            "global_average_pooling2d (Globa (None, 2048)         0           activation_48[0][0]              \n",
            "==================================================================================================\n",
            "Total params: 23,587,712\n",
            "Trainable params: 23,534,592\n",
            "Non-trainable params: 53,120\n",
            "__________________________________________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OOFKgZtAAqkE",
        "colab_type": "code",
        "collapsed": true,
        "outputId": "deb436bb-deec-4838-a007-82488c770bf7",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 307
        }
      },
      "source": [
        "from eval_ckpt_main import MEAN_RGB, STDDEV_RGB\n",
        "\n",
        "class EfficientNetPreprocessLayer(tf.keras.layers.Layer):\n",
        "    def __init__(self, image_size, dtype=tf.float32, **kwargs):\n",
        "        super(EfficientNetPreprocessLayer, self).__init__(**kwargs)\n",
        "        self.image_size = image_size\n",
        "        self.output_dtype = dtype\n",
        "        self.mean_rgb = tf.constant(MEAN_RGB, shape=[1, 1, 3], dtype=dtype)\n",
        "        self.stddev_rgb = tf.constant(STDDEV_RGB, shape=[1, 1, 3], dtype=dtype)\n",
        "        \n",
        "    def preprocess(self, img):\n",
        "        img = preprocess_image(tf.io.encode_jpeg(img), image_size=self.image_size)\n",
        "        img = tf.subtract(img, self.mean_rgb)\n",
        "        img = tf.divide(img, self.stddev_rgb)\n",
        "        return img\n",
        "        \n",
        "    def call(self, inputs, training=None):\n",
        "        return tf.map_fn(self.preprocess, inputs, self.output_dtype)\n",
        "    \n",
        "class ResNetPreprocessLayer(tf.keras.layers.Layer):\n",
        "    def __init__(self, image_size, dtype=tf.float32, **kwargs):\n",
        "        super(ResNetPreprocessLayer, self).__init__(**kwargs)\n",
        "        self.image_size = image_size\n",
        "        \n",
        "    def call(self, inputs, training=None):\n",
        "        x = tf.image.resize_bicubic(inputs, size=(self.image_size, self.image_size))\n",
        "        x = tf.keras.applications.resnet50.preprocess_input(x)\n",
        "        return x\n",
        "    \n",
        "preprocess_layer = {\n",
        "    \"efficientnet\": EfficientNetPreprocessLayer,\n",
        "    \"resnet\": ResNetPreprocessLayer,\n",
        "}[BASE_MODEL_TYPE]\n",
        "    \n",
        "# test\n",
        "with tf.Session() as sess:\n",
        "    a = sess.run(preprocess_layer(IMAGE_SIZE)(tf.constant(x_trn[:10])))\n",
        "    from matplotlib import pyplot as plt\n",
        "    plt.imshow(a[0])\n",
        "    plt.show()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "W0603 10:39:20.469501 140359111931776 image.py:648] Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQUAAAD8CAYAAAB+fLH0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJztnXn83US58L+PFVBZZK/sFCxCRSmL\nWF9kR4XiS4GrCFdlEUWUKqhcBVHk4nsRQbmKIIpQFhdQEYRXUfZF5bIUKC07Baq0QAuKgIBszv1j\ncvrLyckySWaSyTnz7Sf95UwmM08yyZNZn0eUUgQCgUCP17QtQCAQ8IugFAKBQB9BKQQCgT6CUggE\nAn0EpRAIBPoISiEQCPThTCmIyM4icp+IzBWRI1zlEwgE7CIu5imIyDjgfuA9wHzgFmAfpdTd1jML\nBAJWcVVT2BKYq5R6SCn1EnA+MM1RXoFAwCKvdZTuGsAjsd/zgXdmRRYRz6ZVbm4nmaWBDQvi3GqS\n0F+jbWn0re0qcwEB1m9bkGwMi94kmknRjgMmoks2j+eAew3Syxfo1ieVUqsURXOlFAoRkYOAg9rK\nP5+ZdpJ5DsOXPou16detzwKPR/ue6dFcJPH7VryV/1bgHcDNxdFs8CoWXnZTRP5sEs1V82EBsFbs\n95pR2GKUUqcrpbZQSm3hSIYO8wX0i/RIThxh8GWzwV+A7aO03wBcWCOtPBldyG6JWxgTXYDl2xWH\n2fTLU3UzRSllfUPXQB4CJgBLAncAb82Jr/zaVEvb7S3IvL9h+p8wTG/rjtxrS9skpdhKKbZWiuU9\nkCd3Y6bR++tCKUQv+lT0CMSDwFEFcUs+SMOmFLZ3dB1fVHC1grsU/EHBf1hI88QU+bfq0L0e5c1M\nKTgZkiyLfx2NTYnjcRW6Uare7/3RFdKjgZ2sSTO8yK0mzfUwo7FxpuOuP6Cr/Klk/GPQ9+8c4A/o\n6TA3VMz7WPob3r+smI5LZgKbkN5RsDpwtdXcQk0hFdvivMZBmsOG6f35PbBLSvjhwIkl8tsTuMiS\nTC64B5hkO9FQU6jGYzXOfRJYCDwAzGBMmweFUIxpzemWjPBPl8jrcYoVAmiZNi2Rrg2uifK1rhCM\nCUphgDdVOOcIdEGuEp2/AXCgTaFGhKkGcaYz+NjuhB7oMqVM020WcHCJ+HU4FdihobyyCUqhj6pf\n9N9YlWJ0+R0wpyDOCugpPx8CxqMV8hUl8xlfMv4dJeNXZXpD+eQTlIIV/ta2AA2xPnq6n0tOMox3\nProZ8I2K+fyHYbxxwP+rmEc3CUphMXXa/XX6IXxlawanFcxFz//t/f6Fg3x/6yDNNE5Aj1qk8QZg\nD+AC4BVgx4Zk8oPW1j74xedqnNuVpsNFwO7A02TP210CeKlEmh9EK4cb0SMCf68jYMQTFtIw5XJ0\nx+W96Fn52zWYt7+EIUmgei1hBt3oUMy6vnOBX6PXWmxVM48ngI+gX7S6PA+83kI6XcP53JUwJOmO\nf6ILsMsKAWBf9IKnugoB4FFgOQvpgK55jCJ7ty0AEJRCxNGG8e5BK4OufMXOaCifnwGHoNvgNnjB\nUjpd4zy8GIFwtSCq5OIpw8UzrrfHMhaS2FhI1MbmeoHNS6reYqis7bYGZPd9Oyl2P25MHPtS1Xvb\n7irJbiqFYdpcPrAHVZDn8EQad7cke9Z2n4Kdo/zfpODnLclhurlTCqH54A2HoBf47NO2IDkch24+\nnV7h3OS6hI3QJuaSLFshbRtsALw52n8c3W/kK3c6TT2MPrTOj4CPp4TX7Ym2eUs/APyqZhpZ8iSv\nczPsGTurwp5oBXF8izLEmQnEBwxqPRdGow91qvxroVdv3A3cBRwahR+DNr02K9qmhuZD1naCiypi\ntNmooi6yeK2m1/hTS7J3cTtfwess3vNqzYfKNQURWQ1YTSl1m4gsi1bvuwN7Af9QSn2rRFrVhOg8\nRZdd56tQ95ZugjYOaIsF6LX/SeLXuDe6B36UaNSuhtt5Ckqpx5RSt0X7z6LH67psf7xh3u44/Vcr\nnnc8+kG1qRAAvpYR3rP8/w5GRyFci8+Gdqx0NIrIuuiF5zdFQdNFZLaIzBCRFWzkMXwcZRCnTvFU\nmcEuwJE18szjDNJXG76Ktu37JUf5+sRR6Hu8fduC5FJbKYjIMuheqMOUUs8Ap6GX001GrxT6dsZ5\nB4nITBGx5GQhyRTgevTSWh/Zs4E8TL9EvVEF10xG1wgWoU3JT0d3pL2ENqvWRZ4HLgE+AxwQ7Sc5\nCX1/j2tQrhrUnF+wBHAZ8PmM4+sCdzbf0fhkSifOAS47cCpsJh1PTeTV9n2Ib/9jeF982E5VMN6D\ne1ZqcztPQUQEOBO4Ryl1Uix8tVi0PXA9qDqAAlZKCZ8RHdujWXG8QBismWyNf23adwGrti2EAc8A\n30Sb3hs+6ow+vBtd55sD/CsK/jJ69s1k9Bs4D/ikUirX4IC90YeLgd1KxL8c+DrwRzvZl8Lkkn17\naZvkMaqZxmuKlUmffOU1RqMPQzZ5qU4yj6D7H36N1nEboYflVgBuR/cY22RH4MqCOKOsFHqUKdOi\n+2XzWd8JuMpieo3gdvKSn2sfXLcjq8z5ryrvz9tuf3q0bZRzn1YsmdZ9Nco/vl3mwX0pvY2ih6gm\nrmUW9sx+b0t2DSTUEtxi41lZDu0JvDOMopGVJuz7Taa89eAsrssI93kxzrBwkIU0nrGQhn8MmVIo\naqPbYif00g8bnJsS1tZKwVHiR5bS+bCldPxhyJSCDbNiptj40sCgReGZaAvCAffYaKL9xEIafjFk\nSuEDDeZVd3nvyuh27WqJ8B/WTDfQPF9tWwCrhI7GSuxFPe/EeXJORPtXCDSDrWemEx3Do9jR6Jpr\nqOeu/E0UP4TD2XkV6A5BKRgzlXrOPxVmnqQ2rpFHoD1mtC2ANYZIKXzWYdrvQzs/rUqZKupnauQT\nGCQ+d8clBzhOvzmGRCmcCXzXUdoLqe716PeUfxh3r5hXoJ80RaCw49puuBkCpaCAjzlMv+rw4FfQ\nNYwqjOJKzqZ4Y+J31gSyKrTfaW+DjiuFJgphDcxWUS6Pti7U+0J9vUaeF9JvwTdQjjLPxXauhOgs\nHVYKTZpp2IritSZPYde35C1Rujeh1+77SNE9GUVOaVuA2nR0nsLWaFNro8QnqeaExQVln5mmx/CL\n5EvKY/sd8HbOwjDPU/hc2wK0gC8GPaqsGWj/wxMwp3ZNQUTmodePvgq8opTaQkRWBH6OttE4D9hL\nKfVUTholhRjFh8yXr0+de9/kNWTJmSbDI8CaFvM+GE+nqzdaU9heKTU5luERwFVKqYlo8zQWTSov\nbS+pzvB02wJE5CgEo+6EJpV52tBjllKyteK1xw8sp9csrpoP09DeUon+Whx8f8JeUp1h+bYFyCZL\nCWQqh6YUwwpoJbAIM8crvtTEinDfkWtDKSjgchG5VUR664nHx4y1Pg6MT55Uze/Dp4DX1xS3a/yt\nbQEiqs65aJuBRy8Hm4rBxUv7oQbyABv2FdeI/q6KdgG0DfD3RJyn7NhoPF2Z29Arsz3kKF0bW+t2\n/aItRTaTB+SXvl+XwXVW3t7QkGzGabj1+9BDKbUg+rsIuAjYEljY8/8Q/V2Un8rm6Ik/RWT2VVbg\nUbQFJQHWQ3sues5i+gEAbm5bgLLYrC3Yfp6uzghXVnOppRREZOnI4zQisjTwXvSsokuA/aJo+6Ed\nMhRwIPri8uamL1dD2h4nogt+DfpNdM/E/lDn9yyn10FOzDpg90G2y2EW03rSYlo75hxTaI+NFqjZ\ndFgP3WS4A7gLOCoKXwn9xj2ANpy4Yn46m6dUic5NVH3eVbMqd7RhFetflqqOa1uoku5gufrZcPOh\n8PravqammhE2r7WWDF0y8b6F0l9qJ6mXjL8lcAMwzmKes4G3lUxjq0gOX3D1nPje69/U+3EZsLNh\n3KoyyTDPaHTJzdhfhl1WIYBfCmGUaUppvQ/9shcNubtfKBeUQiqnoBchPVzx/PgEq2MrnO/71zPg\njp5BX4XukFkmcfxbziUYgebDxVSbOzUBvax2e7TH5jIzKZ9Ed5zux6C35yJ8VQgunxNfr7lH2+/I\nIuB+YH0GrX+Xwaz58NoaOXSEaWht+4+S5z0MLIn21vQy5Qy5rIzRgMsAvr8cgXZYNdqaYUSaD1X8\n/a2PXs81Hl1LeNWmQCkEhRDwAz9qCm8D5rjORFH84q2CrllsiVYKawHroGsMLgkKIeAPfiiFJRlr\ntjl9P/IUww3Au1xmnkFQCGYKO9AU/jUfeh2veRwC7Fs1g+SM6wuiDNtQCIExNmlbgECEf0qhh0J3\nuPZ8o+yCXq6g0COG56DNt5RmFfonif1bTUHr0JWv40kN5DGrgTwCJvirFEC7VZwDPAhcyuBozDrA\nhk0LNYo0Zf7uSsN4j5M+CzhgA7+VQo/1co69ozEpRpCNafZl2xG9UC2L3sufZSPBhYJY2WJa3aAb\nSuGunGPORy1csXbbAiRI+/K2cXPnA2cnwqq86LYUQ97KxOHEf6XwY+AE4Dh0n0Kcm+lwU/SRtgWI\n8LHqvR/GX/09gSWyDtq4tvfWPL97+DEkmcUc4PvAjWiTe38BPgq8E/gzcGh7onUf3xRBCQ6A62do\n7x+Q0l07EFBnyHP7iud1F3+VwjPA59EKAbTRpR+iHSZtAtwb7QdK8jB6pmZHUYPqTKEbHWvFA6wp\nhgkVzuk2lZsPIvIWEZkV254RkcNE5BgRWRALn1o68YvRfkDTOqNnoYcjg0KoyLptC1CND5KqEHqs\nScIrZGrLocO1owaprBSUUvdFvh4mo40sPo+20Qjw371jSqlLjRO9B3grHfLGvj1j5sO/0bIsRfjY\nd2CIAn5RLP01aYEHJQM6eg8axFZH447Ag0qpP1c6+69oVT8JuNuSRE7pKYJrY2Ffxs/JSB1WBrBY\ndNMrGIiX6qjp8crijAK2lMLewHmx39NFZLaIzBCRFQrPngcssCSJUxbSrlORT2E2YWcIJvR8ndIK\nIZOBBMr4ghg9bPiSXBI9WPhWpdRCERmPtjKi0EW7mlJqwBhB5DgmqtytvbkeTvCVp4AVS56Td19n\nAZtaTG/IULk/jSgekTBV3sN0382MrNhQCtOAQ5RSAwO6IrIu8Bul1MbJY/3xXFpeqkPdr37WvS2T\nrsWHMpnUEsAr9pIvnX/abbCgEFKTTr3lJuUwekrBRvNhH2JNh54TmIg90H4gOsY7cdMM2KBkupYe\nyKyWxMuxY5cbpLMDg60XExGz4qWNLbpiM4dpDxm1ncEA7wEujAWfICJzRGQ2unu+qdU0lngce26N\nfhH9fRqtDB4oca5FhWDCe6K4L+akc1XOseSL/9GUMNNza5Kqdm+zl/6wMwKGW8viywiChXJpv2hr\nY3oJLwBvyIsQmg8Evw+dJuNBLFN1v8K6ULWIi7xtifPyXtu1GRscLq8QAlkEpdAV8kYfkyyF9p2b\nEb23rWJDrhwmka67riV/NXwSydiMlpTtWiKjABCUQoIZbQtQjd6bt3b0959mpy2iepN+OukVl/iW\nt+L9wZL5VcZ8Pm0gwt8FUa1wYNsCkPp6mr6xBVM9TPr8QBuzz3owbLaw66xdtCcBCSn2QD8Ho1vF\n8EMprMqgPdXG8bTh2UI/17h2snXDR4CfFEUamqu1gh/Nh7XQ5XJhUcRhJzyc1vlx2wJ0Dz+UQo89\n0O/FcU1n7GktIWAHRTUvfiOKX0qhx5E0+NH0WCGMQMWhzChELXZjJO6nDfxUCj2cF6JPCmE0n9gH\ngVObzLDji0ebwG+lAA4L0CeF0BmrMk74NMXDmwptRGWlnHS+lXJOJkOwutwVfow+jDwXDQaFh3WA\n7dBr8suQdhtTzTfmRhgtglJonWbe/qBjxijUAWnTF0YI/5sPSVdxo0DKG7xvFHze4KFATTKbHL2A\nt7UgVIv4rRSEITend/ZgUMaExnOi/b3zklsSuL6uTIEB5TC7JUFawj+lcCZjK16Gnv1yj5bqBxO0\n0ZRtGbh365SWKwAptYYRwUgpRAZYF4nInbGwFUXkChF5IPq7QhQuInKyiMyNjLcW27x5gDFF8PFq\nFzIUqNTdYgoU6LwKogQ0q7ctQAuY1hTOBnZOhB0BXKWUmoi2yXNEFL4L2on8RLRh1tMKU3/GUIqh\nQqX+LD1KNhI1qvboMzI+IrUFI6WglLoe+FsieBpjTd1zGBtsnwacqzQ3Assn7DYGchRCEX3GyYNC\naIRPloq9M/BTN4I0RJ0+hfFKqcei/ccZe17XoN/+xfwoLADUUQgQW0waFEJj/MA4pgCXoZdmmp/l\nG1Y6GpU29Fiu1itykIjMFJGZ8IQNMbpHSYVwX28nKAQPSRrl/RTZVnD9po5SWNhrFkR/ex+xBcQc\nAKMdwg34f1JKna6U2kIbknRtGMwXBl//Mpp0QwgKwVs2TAnLm5TtL3VmNF6CHlM7Pvp7cSx8uoic\nj3ag8HSsmTHCpDcbAsPCv1LCnkPblk8OwM0CbkTXLl4EVgbWB94ObOJQRjOMlIKInIeeer6yiMwH\nvoZWBr8QkQPRhsD2iqJfCkwF5qI9UR9gWWYLfKHh/O7v/1my2QDREE7FWsI/gGWqnRpIMhNINZL+\nOtKNYyb9Vs5Efz9/T79bg6WBjYEpaH/Nu9LWNKIR9fvQdB18cAJC2bsui/8rlx3AVsAfS+YX6Ecy\nf+QdmIN+0dO4EfhdFOdZYDlgArqmMAU9ol+G+eiFddcA9wJ/RxvWWxbtB/WNwKVGfh/Cgijn1FcI\nQC099qfqpwZKoYA9gQ+ivSnmMSXaqrIA+B7wXYzNdxsSlIJTWqiFtV/xG35yzVC7NjR6NNqZuzuC\nUmiKOrWEgH+sQgsj6f8X+I3zXPxbEDWMNKUJdsw+9I2GRBgZep50niV/at6+wMG2MnWvECAoBYfU\nm5MQx7g74crsQ3+omHeggGXQfXxZNuTOQQ8dpR1brgV5DQhKwTq7Yq1zsQwFGQTvadX5H1cJP02/\nkvgTg8sO+2jGg1kYkrRK9gSlOnfZaDiyIIP2S9k+8Vvi8vpam0SamnEtaYIr+mbJfizbrCUMq8Hi\n5KsxlLO/UwvOfWmOqFLYt7Gc6hZh4cO+aXqew6oM8hgtxVB2cpM5I6oUzimOUooMw4pNcFs72frK\n6CiG+7MO1GZElYJjWupcHCWFkDdfcGgVQ+rQpgI+ZzWroBRqc2v/z9fZSzn34T7WXj5d5GcFx4dS\nMZyGXioxwEnY/CQEpVCbxLLYFxxkkVyVOw/4an/QKNUSeoziNbMxObOcFfDrlPBxwFeMsxhhpbCr\nhTTcDEFC4kv3+UQeCZvtI/lyRLR67b8DzkKbJ5Ya2+Yl8819v6cxOEvqFcqslxjReQqLc65xbj1b\ni0UMLNWdyIBZBpv5dZ20knRSFrmBljgd+IRBvNIySJin0BguOxZPjP5e4yLx4SE5BHuty8xcd1gk\nax5ZPJJzrAaFSiHDEcyJInJv5OzlIhFZPgpfV0ReEJFZ0WbJpG38DvmysPPMvl/OvthfjP6mLLrp\nai3hfgZr0LboKYdtLabZOvEb9cNY+Jroi/1q2knVMakpnM3gjOwrgI2VUm9Hl/GRsWMPKqUmR5uF\n9WHJR+ZV+u9SUT90HnVWBIyyK6tyrE5/ib0lJc7IeAqsy8HoG3VyLOxYshdkVZjJVqgU0hzBKKUu\nV0q9Ev28Ea2zHGDi7vfDVH+kdgHeU+E8gEP0H9XOF9v3WkJcCZSx2uu9YtiobQEiDgWOc5O0jT6F\nj6H7YXtMEJHbReQ6Edk66yQzvw93ZoRnpgrsUPKcy6k26H8K4P/L2TQ2vvhe1xrubVuAGEe5SbaW\nUhCRo9DjHT0/WY8BayulNkUPpP1MRFJXjbvz+3AN+pEqM8vrq1R6vR1phOXdJOuM1L4B0+G4gnQP\nsS3ssOFAe1ZWCiKyP/B+4MORhyiUUi8qpf4a7d8KPAhsYEHOCnwH+H7Jc0zfclUqdlme7u381VEG\nNdmXjPf6pbTAAgrifx/4bFkBO0uW5rQ7jbmISkpBRHZG94vvppR6Pha+ioiMi/bXQ4+uP2RD0PIc\nCXy6wnm/LzgeKYQm2g0eOBhKe0R/nBVpqZoZZcyv+V6NZP3nvRRr0u+QW72yXFswGZI8D2185i0i\nMj9y/nIK2qD8FYmhx22A2SIyC7gAOFgplfRW7Zhx6Be3ai/M+3KO+WN99WFL6VSq3dcZT8xz4Xx0\ndnre9jFUZgP0VV1R4Vyhege5Qer+z2gsehzWQS8GaICGRhr6rngl9NjPl9A+ufrFqZfH07jvwMgT\nsqhoE+dOBm6vKU4VBvpKzGKiv52HoN3HufTRZfixkqGZ0aiA5CDGKYwNvs5rRoSWhh4Xr7f65uCh\nql/Pxee1qRBMSFzgrJrJ1eb4vINppTE9Cm/IaV/mWF85fJkeWMD17WXddosh7t5pK2q7e5LsnQhL\nV/qcnWSSCC2WxcVZB9pu3ER3xZJvwA7UFFriSdpXCNC/FPuGwcOVHsdSVeCKvMFOMq2/b3E+khbo\nk4B2CEohi6jn/8Z2pRjE6jPo6IE20aJdfJcGJk38NC1W5wlKIY3YQ/3O9qTIRnJ/touJQqg5Vdif\n602tOrSE9P3JOmxCUAo5tD4uY7KmPsKkzAfjpC3qqnHVJqdeQvmpwseUF8UmrT8HdSmpRYNS8Iy+\n8jsjJ+KbbeT2o8TvadWTMn1zqmTxnxXOGWXmxvYrTB0MSqGrPDgYVK1a/Wr09zOk2/eziMV6vz9N\nCF+IaeX4B2M9Snd9dGRIMmBK+SG715Q+Y4CWOhZbHZ4cW6HiF2k35N+jzbAMQk2hy5ydHpxW9p3/\nsno3/fnbDtJ8PXAe8ChwNeZeoOyOnQelkOSytgUowQHZh+Ynfi/hVJACbL25LSmG9PTfaDmX/YDn\ngb2B1YDtMfMCZX8yzYgohY8Cb8JIu7/XtSzNkDSF9VL8R+Hs95LkWcu37ce9mYWCBukeZDmns0vG\nfz+uZtd1pE/hF8AE4B0Vzo0X6+Ho1+PIjLiDZ7Y6HHUP+WP6Vae13oLdt+hSdG03afTqLLRdLhf0\n5FeDQYng1OP1WdZiWu8vOK6A5YBnY78jLHok69GRmsJvgZsqnJe2LPXLpVJo1TTYhgXHt0oP/nNG\ndKcKbkcGl1S7Ughxenldkx5cdYV3Kt+wkUgaqxrEeYZUC6wOPJJ1RCm8QDXNnPS3Vp3WlMPkjPAc\nU7lrm6Tb+Rk5CXagXwMkDU6cSLqm6G27F6T/S8p+T0owo9ppjsqwqt+HY0RkQcy/w9TYsSNFZK6I\n3CcieRZLSvBBdEdMWUpmPyUlbAn6tIHVL08KA+V8e0rgT3DmCGRoWI/+wvpifnQuJl9p7GVZvhXR\nZogWm2BfX/89A91SKMKlUldK5W5oa0qbAXfGwo4BDk+JOwm4A22YawJ6is244jw2V9rAmYvtjIT1\n+6nZca+KC2WwxS7CJoWFUrAZp+vsno/KZuRtQW+l/j2aXU5r1ng2YKZJtEp+H3KYBpyvtAHXh9ET\nLrc0PNcRB6I7GHv8NjvqddFf02pArMpgs+agapybZaSr8/MUusjSjKmFUqwOvHvsp0I3fxSN1BDr\n9ClMj9zGzRCRFaKwNegXez6pDs8w9Ptgi94dLSidY6n29nikGDZlbK7d12i5o3RUOQtdgP+ok8if\n6FsocnhmROtUVQqnAeuju8Eeo8L0LuXM70NLOFIMJjYJ386YyjsA7ZZNKHBxs39N4QIMfO9uQxfC\n/rbSv4QCG3BOqKQUlFILlVKvKqX+hV5q12siLADWikVdMwobDRwohskMNlBfTPy+IxZ/Q2ChScLn\nWBRyZHnr2K5CV9OsYzanJsk/gN9UzLGq34fVYj/3YMy/2yXA3iKylIhMQE/evrmibJ3GZZV9yYLj\nOb0mmtCeqMlCtPeLicBK9TqBHPA8ujtjV+CfFc6v6vfhBBGZIyKz0ZO0PweglLoLPf3wbrRXlUOU\nUq9mJD2cxF64tty/TUWPyAVscC6wCf3jk29C+8naCNQi4GX0t9EV5WZNvT62vxT93utN6IDfh45i\ncFt/QL5vFBtkVgpCbSGDEjdGncSgS7cNgfssyrM4s1oxdwSuHh6/D8PLwbh/N3t9EIE8qk5JuzAl\n7F6c+Yg3IMuC31Ul0ghKwRVnmUd1rRiWJOXrcZfjTL3mbOzMTc1akXYktR3AvFzttF3q5QoEpeCO\nkouBmqjN9ymGSQ1k6AVpc5ZzDFGYsltRhGeLImQjaE1e4aE4r3quiwlKwSNCM98mn8XpHc30FlWT\npMgle4x/aZhsHh2xpzA6uLbhoBgF5ePLFd4A/B/z6GliP1w+178zNvJVxYdwqCmMII/2dtofeHJA\nkwph9YLj7zJPKlfsz5unA6zAWEOpyrB4UAquKJphlIPrxzo+82y4FEPTNQQLk3VPJV/sNcCNkdhs\nQvPBFTXHARttRih0D9W/O8zQOQ0qBFsFYyLyvpbyKkGoKbigI1/fPjH3YXCRxf7Ny1SNNvoQahay\nqcgtTHkISsE2FceX02hqmDLz8T6rKIIjlkXPJDZiW4eCpHABwBfqpVGqYJv/wgSlYJNFdLZBFq8g\npA5rKUq8qDU5EfieaeTrHQqSwr8BfKvaua9QUiEcWi2fmgSlYIMp6JcmxyzEAQzWzv/TIOk2KsYf\nIKOC8Bh6CZ5rDkab5Syk4bszpWy1KbaWXijhkSfKR32nRF72CErBBgYOT7ZPCTvaMHlfRt2B/iV4\nI4Wq5timdFNBje22RFAKdTGsSdZ1GdCWYtizpXz94nflT5EL2+s7eImx2kkFgwpBKdTF0HvYPRay\nWqs4inXS1gH6Qc3OPmMuBHbWu6a1eYGo88EQlfszLfaKeRGWiu2/ntJflKp+H34e8/kwT0RmReHr\nisgLsWM/KCdOBynhtivNnH8ZC17z8awp0SonNZCHos94StJ0QpLFiy7PrpdlBn+IHf5rngxlwlMw\nqSmczWJVqVFKfUgpNVkpNRn4Ff0flAd7x5RSBxtJsbmhtD7ya/OozzLoDdPEMGuSqgank9t/AQ9Z\nzqc79HliydhSELS7vm+gO0NTV2CXWYVp1mzYij6j724x8yHBusScwcTCBW3SfWJevML0N9+8u85J\nlqzmmGMloxuTshk6fulR2XFog/b9AAAMCUlEQVRIWbka364teKNvaUmuMwvkim8X55ZtUTmXLxtL\nzmAK2BpYqJR6IBY2QURuF5HrRGTrUqnZdlveBC8xuO7FwCplZvWvR141MMWNXUE0e9S0HWKPbcl/\n5wqtjjnivwzjKfqMMnwqP6YRlgq8rlLYh367Do8BayulNkUv7fqZiKR6xutzBvNE5AxmCrBBTYna\n4Eb6q5Gvjf5WaRtMxaxwZfBnri2hrVMiCTC+hGwCPFci/kiS1yA7BbKaJt+3LUdVA+9gVltMaRag\nH/2FwJo5510LbFGYfq/54FX11NK2m8ENrnPdLtK+LjrnNg/uX+e2ZRWkVV1yzjkxv+yyyC/jtHzd\nNx92Au5VSs3vBYjIKiIyLtpfD20YP091pqNqSOUbl6C/sFkuxL5Pvbq+6Xnrlkhz2yjdzUpLE0id\nilnwQOe4hMs6M98gk2Tsm1HV7wPA3gyahNsGmB0NUV4AHKyUMnVO24/CfMpfLovQE+lPp5prDEss\nS3r1/RALaZuU+8M02H09yiRtHxzlJJeHkwElDDwV4Yffhy22UMzM8PvwT2pMrV0d3c0RZy3gL1UT\n9BuTohzucUZP+DH6i/Y+tHePHArKLO1wahFKUQwFDIvfh9dRozmRVAigR1BLzDjqEoKe4RJomY+i\nv+X15u59NiWsWKfXn73kv1LoYbVC8yLwFZsJ+sM2hNpAVyh4pr8b2/8tOcVquby7oxSggmJ4MueY\n6XiyDxwLrAR81fwUATZKhOWMheczif6OkE7bbfODEs2GrwDvz4pY2Gwoj/99CqknlEldka37VgCq\n9YM2wyXAtERY0+WVdbPbf246S8l+BLMagtFLMSR9CrXpmUBNW4bxFG1ZtylGGFQImd+LQFeo0LFY\njN32QzeVQqU7d1rGiScD19USxz5Zhfz/G5VCEywqNEWpx1oGdqzRTaUA8PGqJyoGjX0eVk8Wq2QV\nclpfdBP8Ct17GacFu+PDQM5bPy8jfMO0QIcKAbqsFH4EXFr15GvRDrV2BFbFH2+rx+Qc+27OMddc\nh+7knIK2+35Oi7IMJ+tkhN+Xecab3QhCZ20PR+zCmPYt7TRvOeBK2xLVxMSUa1t8EO1bqqTH00Ah\n5ZsNewIPOpEFuq4U4ryRsbs7dOP0G7ctADALvYxlqaKIgRLkLY4ceIwXB1zkRJYe3W0+5KEYspl9\nu7ac/x/QbbXrgZuBF9oVZ4gwnjriuB8hzvDUFJK8G4bH77qZVTt3XABchh7CfRm4DW0gLNAszTzM\nw1lTiNOpOTZZrs3XbVKIBH9EK4Gnot8L8XvC13DgYKKiMcOvFKBDiiHNtfmpjUvRz6toG2y9J3MC\n8Jb2xBngDsamX7fjUSmdnkzHjAVNH4z1q1LpNUM3pznXyqyZbOpxAPAoeuivKQeOefwa3a/wCno5\n8NR2xeljF+D3ibA2n+kPAz9LCVd9f1KOLGZH4Op4gCz+ry5G05wL+xREZC3gXLQ1PwWcrpT6rois\nCPwcXbedB+yllHpKRAQ9qD4V7Xlwf6XUbVWvwjqd6Gc4q20BEuwebT6SZmyj9wg2PeEr78GKptv3\nZt0njuwAXJWZQrPzQkyaD68AX1BKTULPXjlERCYBRwBXKaUmoq/niCj+LmgzbBPR/pNOsy51XXoW\n7AJDQJYPq0NpVvtXsLYb4+rco/uXlqYOhUpBKfVY70uvlHoW7QFtDfRqnZ4KO4exT8k04NzItuSN\nwPIispp1yW3QUw5lfHcEPCTPFlkTiqFkHp7XVEt1NIrIumhPZzcB45VSPdNGjzNmLHwNtHmjHvOj\nMH+ZQag5dJo/AUvmHP+yw7yPKRlf+v74iLFSEJFl0J2lhymlnokfU0qVrpCn+n1oGwXMbVuIQDVe\nJHtIt67P7zyqTE33WzEYKQURWQKtEH6qlOo14hb2mgXR30VR+AL6HSSvScpYm1LqdKXUFkqpLVhl\nlary22d9xpoVqUvUAv6SNqTbw8Ub6OlbXRMTE+8CnAnco5SKu/q9BNgv2t8PuDgWvq9opgBPx5oZ\n3eIeQqdk58grrCkW88lrrpjgb23BpKawFdo87Q4xF/NTgeOB94jIA2jHMMdH8S9Fr5yZi17g/Gn7\nYrdAUA5DwE3YeQs/jZ7uXZfIeLBnimH0Ji/ZYnP07N+Ah5i8ZVWf++Ow6+Ale1LTYsSa1rAzeSmQ\nwa2xfc80fcCEeKGZKgiHBb06ehJrKs3OuBuNtQ+uab+yFahF0Qt3hkGcmnjU6xaUgi1Cn0PHSXvp\n/xiFf8J99uOKIthwOmpGUAq26SmH2W0LMqrcUuNcQfsC6a1w3NqKREa8UhThlCakAIJScMfbCDWH\nVrim5vl/tyKFVRY/R80YGA5KwTWhWdEwwzwkdFcjuQSl0BQ95eCTKYKh5N62BShB+nCkim1PJ6I2\n8YXxQimML44yPPyWUHtwysNtC1CL5GOxXGost+4DvVAKa7YtQFuojK2ZWuKQ8kxxlEx2QhfAUwz6\n8bTNYC2h8DuxOIJb94FeKAUIH84+JpGtMPK2u9EjV0kX9AEDjgCuiPaXR5ugUzRlXj/v+U8/pnBl\nkckbpQBBMdRmI/TI1d1kK46yXJmTVnJrcATPLtuRvbz6N4yt9bOFSvtjckYi8r5RQJb1qWp4pRRA\nX+Lr2hZimFFkNVTHmMnYi75jibSvp6OavWgYczeLefVrgjfWSGKMPbDZUeWdUgDtf2iJtoUYZp4m\n3eDwZ9DP1eY10z+pOIo/rGQYz8b8hcGXtkyqZq+8Qk+S6bExZRWGl0oB4KW2BRh29gE+EPu9HXCy\npbQ/ZymdRtjeMF6lb3o6zn30zga+iLaZPKf02WGV5CgTX5WXZ7RoqGnBL+ZDTWRyF9rDQnm8rSkE\n38aOOR+4Ifb7AbRVdBv8t6V0GuEmw3h1hjqbZhP0hJhTgLeXPttbpRCaD47ZJyXsZPSzVJfPW0ij\nMZ40jHechbzqLb82Pzu+Gm9OdKb53AYvlcLxxVECdfhYzjG3k+U6jBufnua+JE3JUh3mIyh+mGMT\neQJ4DnO17SMr0235ofvX0HX5we01rKOUKjSd7oVSABCRmSb243yl6/JD96+h6/KDH9fgZfMhEAi0\nR1AKgUCgD5+UwultC1CTrssP3b+GrssPHlyDN30KgUDAD3yqKQQCAQ9oXSmIyM4icp+IzBWRI9qW\nxxQRmScicyI3ejOjsBVF5AoReSD6u0LbcsYRkRkiskhE7oyFpcoc+QI9OSqX2SKyWXuSL5Y1Tf5j\nRGRBwqVh79iRkfz3icj72pF6DBFZS0SuEZG7ReQuETk0CverDJRSrW1oa/cPopeILAncAUxqU6YS\nss8DVk6EnQAcEe0fAXyzbTkT8m0DbAbcWSQz2prk79CzYaYAN3kq/zHA4SlxJ0XP01LAhOg5G9ey\n/KsBm0X7ywL3R3J6VQZt1xS2BOYqpR5SSr2EnpHv2g6WS6YxZg7nHGD3FmUZQCl1PfC3RHCWzNOA\nc5XmRmB5EVmtGUnTyZA/i2nA+UqpF5VSD6MdHm/pTDgDlFKPKaVui/afRfs1XwPPyqBtpbAG8Ejs\n9/worAso4HIRuVVEDorCxiuleg7AHqcbNmmzZO5S2UyPqtczYk02r+UXkXWBTdErsrwqg7aVQpd5\nt1JqM2AX4BAR2SZ+UOn6X6eGdrooM9powPrAZLRHxm+3K04xIrIMetnDYUqpvuWXPpRB20phAbBW\n7PeadGRlv1JqQfR3EXARumq6sFe9i/4uak9CY7Jk7kTZKKUWKqVeVUr9C/gRY00EL+UXkSXQCuGn\nSqmecUWvyqBtpXALMFFEJojIksDewCUty1SIiCwtIsv29oH3AneiZd8virYf9i1+uiBL5kuAfaMe\n8CnA07Eqrjck2th7oMsBtPx7i8hSIjIBmAjc3LR8cUREgDOBe5RScaN1fpVBm72xsR7W+9G9w0e1\nLY+hzOuhe7bvQJu4OSoKXwm4Cm2y5EpgxbZlTch9HrqK/TK6fXpglszoHu9To3KZA2zhqfw/juSb\njX6JVovFPyqS/z5gFw/kfzdj7odnRdtU38ogzGgMBAJ9tN18CAQCnhGUQiAQ6CMohUAg0EdQCoFA\noI+gFAKBQB9BKQQCgT6CUggEAn0EpRAIBPr4X6u58PhAAh9KAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rxo0DhiQDJOP",
        "colab_type": "code",
        "collapsed": true,
        "outputId": "ffc3c64d-1462-4266-d794-30684e2cdd7c",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 242
        }
      },
      "source": [
        "def get_features_model(base, x, image_size, preprocess_layer):\n",
        "    inp = tf.keras.layers.Input(shape=x.shape[1:])\n",
        "    x = preprocess_layer(image_size)(inp)\n",
        "    x = base(x)\n",
        "    return tf.keras.Model(inp, x)\n",
        "\n",
        "features_model = get_features_model(base, x_trn, IMAGE_SIZE, preprocess_layer)\n",
        "features_model.summary()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "input_5 (InputLayer)         (None, 32, 32, 3)         0         \n",
            "_________________________________________________________________\n",
            "res_net_preprocess_layer_23  (None, 224, 224, 3)       0         \n",
            "_________________________________________________________________\n",
            "resnet50 (Model)             (None, 2048)              23587712  \n",
            "=================================================================\n",
            "Total params: 23,587,712\n",
            "Trainable params: 23,534,592\n",
            "Non-trainable params: 53,120\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cl9M3pa5siL9",
        "colab_type": "code",
        "collapsed": true,
        "outputId": "ece2f027-6f3c-4c49-d436-4dcee93d2758",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 52
        }
      },
      "source": [
        "x_features_trn = features_model.predict(x_trn, batch_size=128, verbose=True)\n",
        "x_features_val = features_model.predict(x_val, batch_size=128, verbose=True)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "50000/50000 [==============================] - 170s 3ms/sample\n",
            "10000/10000 [==============================] - 34s 3ms/sample\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tvNJRqWPs_Tg",
        "colab_type": "code",
        "outputId": "0ead2a1c-8f8f-489a-a243-fc7bd3d8d153",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "from sklearn import linear_model, metrics\n",
        "clf = linear_model.RidgeClassifier()\n",
        "clf.fit(x_features_trn, y_trn.squeeze())\n",
        "print(metrics.accuracy_score(y_val, clf.predict(x_features_val)))\n",
        "# EfficientNetB0 (4mil params): 0.8747\n",
        "# ResNet50 (23mil params): 0.9168"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "0.9168\n"
          ],
          "name": "stdout"
        }
      ]
    }
  ]
}