tutorial/part_3_run_the_experiment.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "ename": "ModuleNotFoundError",
     "evalue": "No module named 'google.colab'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mModuleNotFoundError\u001b[0m                       Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-1-6c6e0aa62b66>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0;32mfrom\u001b[0m \u001b[0mgoogle\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcolab\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mdrive\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      2\u001b[0m \u001b[0mdrive\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmount\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'/content/gdrive/'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mforce_remount\u001b[0m\u001b[0;34m=\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mModuleNotFoundError\u001b[0m: No module named 'google.colab'"
     ]
    }
   ],
   "source": [
    "from google.colab import drive\n",
    "drive.mount('/content/gdrive/', force_remount= True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Uw4lRlTQ5e47"
   },
   "source": [
    "### Addtional Resutls for our PMC Journal Paper: [Privacy and Utility Preserving Sensor-Data Transformations](https://arxiv.org/pdf/1911.05996.pdf)\n",
    "\n",
    "#### (1) Brief Description of the Resutls\n",
    "To show that the compound architecture, proposed in the paper, can generalize across datasets, we repeat the same experiment as Table 9 (in the [paper](https://arxiv.org/pdf/1911.05996.pdf)) on another dataset ([MobiAct](https://bmi.teicrete.gr/en/the-mobifall-and-mobiact-datasets-2/)) by keeping the same architecture for RAE and AAE.\n",
    "\n",
    "In this experiment we consider all kind of **falls** as **sensitive** activities, assuming that they can be considered as symptoms of some diseases. We also consider being **steady** as a **neutral** activity, which in this dataset it means either **sitting** or **standing**. \n",
    "\n",
    "We see summary of the results for two different settings for utility-privacy parameters in the below image, that show almost similar results to what we have on the [MotionSense](https://www.kaggle.com/malekzadeh/motionsense-dataset) dataset in Table 9 in the paper.\n",
    "\n",
    "Accuracy of recognizing the **required** activities are almost same before and after transformations, while accuracy in detecting falls is dropped from 99.6\\% to less than 4.5\\%. Moreover, we can reduce the adversary's accuracy in detecting gender from 97.35\\% to 66.8\\%, which is close to the random guess in this dataset that is 74.5\\%.\n",
    "\n",
    "Note that in this dataset we have 41 males and 14 females. So, randomly guessing a subject as male is 74.5\\% accurate.($\\frac{41 \\text{ males}}{55 \\text{ males and females}}=74.5$).}\n",
    "\n",
    "| <img src=\"additional_mobi_act_pmc_paper.png\" class=\"img-responsive\"> |\n",
    "|:---:|\n",
    "| Reproducing results of Table 9 in the paper (MotionSens dataset) on another dataset (MobiActdataset). |\n",
    "\n",
    "* Note that this notebook just use all the already trained models. If you want to train your own models for each stage, please look at other files in this repository.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "B6A72LVl22Nt"
   },
   "source": [
    "#### (2) Import required libraries "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 64
    },
    "colab_type": "code",
    "id": "llnhh0F0KLiJ",
    "outputId": "d08d50da-f0c3-4429-d799-b8660c4e6fb5"
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import tensorflow as tf\n",
    "import tensorflow.keras as keras\n",
    "from scipy import stats\n",
    "pd.set_option('display.float_format', lambda x: '%.4f' % x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "DqUBl8qX3BzJ"
   },
   "source": [
    "#### (3) Dataset\n",
    "(Dataset is available here: [The MobiFall and MobiAct datasets](https://bmi.teicrete.gr/en/the-mobifall-and-mobiact-datasets-2/))\n",
    "\n",
    "We choose 6 activities in the dataset.\n",
    "\n",
    "* **STR**: satir-stepping, including both staris down or stairs up\"\n",
    "* **WAL**: walking\n",
    "* **JOG**: jogging\n",
    "* **JUM**: jumpping\n",
    "* **STD**: being steady: either sitting or standing \n",
    "* **FALL**: falling (Suddenly from standing position to the floor)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "OGhE_W4swYkP"
   },
   "source": [
    "#### Name and Code of each Class"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 52
    },
    "colab_type": "code",
    "id": "lXvf_Vp4_4mi",
    "outputId": "b2009f68-2e97-4520-e47f-0edb13b473b3"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 : STR | 1 : WAL | 2 : JOG | 3 : JUM | 4 : STD | 5 : FALL | \n",
      "0 : Female | 1 : Male | "
     ]
    }
   ],
   "source": [
    "act_list = [\"STR\",\"WAL\",\"JOG\",\"JUM\",\"STD\",\"FALL\"]\n",
    "gender_labels = [\"Female\", \"Male\"]\n",
    "for i, a in enumerate(act_list):\n",
    "    print(i,\":\",a, end=\" | \")\n",
    "print()\n",
    "for i, a in enumerate(gender_labels):\n",
    "    print(i,\":\",a, end=\" | \")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "611b6GDtwc3i"
   },
   "source": [
    "#### Loading Train and Test Data\n",
    "(Note: you first need to prepare dataset using the first part of the tutorial in the same folder)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "2g0dtCE9_fpc"
   },
   "outputs": [],
   "source": [
    "x_train = np.load(\"x_train.npy\", allow_pickle=True)\n",
    "x_test = np.load(\"x_test.npy\", allow_pickle=True)\n",
    "y_train = np.load(\"y_train.npy\",allow_pickle=True)\n",
    "y_test = np.load(\"y_test.npy\",allow_pickle=True)\n",
    "y_train = y_train.astype(int)\n",
    "y_test = y_test.astype(int)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "colab_type": "code",
    "id": "Imn-IZQhKLmg",
    "outputId": "7bfe65d3-1918-47a1-912a-887a620b09eb"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((176760, 128, 9, 1), (176760, 6), (37117, 128, 9, 1), (37117, 6))"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nb_classes = len(np.unique(y_test[:,0]))\n",
    "batch_size = 64\n",
    "\n",
    "Y_train = keras.utils.to_categorical(y_train[:,0], nb_classes)\n",
    "Y_test = keras.utils.to_categorical(y_test[:,0], nb_classes)\n",
    "x_train = x_train.reshape((x_train.shape[0], x_train.shape[1], x_train.shape[2] ,1))\n",
    "x_test = x_test.reshape((x_test.shape[0], x_test.shape[1], x_test.shape[2] ,1))\n",
    "x_train.shape, Y_train.shape, x_test.shape, Y_test.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "oGOJKQRDr_Xj"
   },
   "outputs": [],
   "source": [
    "np.set_printoptions(suppress=True)\n",
    "\n",
    "from sklearn.metrics import confusion_matrix\n",
    "from sklearn.metrics import accuracy_score\n",
    "from sklearn.metrics import f1_score\n",
    "\n",
    "## src: https://gist.github.com/hitvoice/36cf44689065ca9b927431546381a3f7\n",
    "def cm_analysis(y_true, y_pred, labels, ymap=None, figsize=(10,10)):\n",
    "    cm = confusion_matrix(y_true, y_pred, labels=range(len(labels)))\n",
    "    cm_sum = np.sum(cm, axis=1, keepdims=True)\n",
    "    cm_perc = cm / cm_sum.astype(float) * 100\n",
    "    print(labels)\n",
    "    print(cm_perc.round(1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "JM4tNfMixZi_"
   },
   "source": [
    "#### Original Data (without any transformation)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "fTbMvrq5xgpp"
   },
   "source": [
    "##### Activity Recognition"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 400
    },
    "colab_type": "code",
    "id": "ptZ92_7PXeh4",
    "outputId": "a9bfbb76-7a67-4c20-f62b-593a970b24a0"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1160/1160 [==============================] - 170s 146ms/step\n",
      "['STR', 'WAL', 'JOG', 'JUM', 'STD', 'FALL']\n",
      "[[99.   0.3  0.   0.   0.7  0. ]\n",
      " [ 0.4 98.3  0.   0.   1.1  0.2]\n",
      " [ 0.7  0.1 94.5  4.8  0.   0. ]\n",
      " [ 1.7  0.   5.2 93.   0.1  0. ]\n",
      " [ 0.1  1.   0.   0.  98.7  0.2]\n",
      " [ 0.   0.   0.   0.   0.2 99.8]]\n",
      "f1:  97.24\n",
      "acc:  97.74\n"
     ]
    }
   ],
   "source": [
    "model = keras.models.load_model('_best_FCN_.hdf5')\n",
    "preds = model.predict(x_test, verbose=1)\n",
    "preds = preds.argmax(axis=1)\n",
    "cm_analysis(y_test[:,0], preds, act_list, figsize=(16,16))\n",
    "\n",
    "print(\"f1: \", np.round(f1_score(y_test[:,0], preds, average='macro')*100,2))\n",
    "print(\"acc: \", np.round(accuracy_score(y_test[:,0], preds)*100,2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "BYi1SkosMdK8"
   },
   "source": [
    "## Replacement"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "dKRM-Ew9x6SP"
   },
   "source": [
    "As we said, we do not want the app to infer when falls happen, while we want them to infer the other four moving activites ('STR', 'WAL', 'JOG', 'JUM') as accurate as possible."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 245
    },
    "colab_type": "code",
    "id": "G5VcEUDPMutF",
    "outputId": "561d6dbe-afe6-4a8c-c3ba-6eb13149a79c"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1160/1160 [==============================] - 119s 103ms/step\n",
      "1160/1160 [==============================] - 172s 148ms/step\n",
      "['STR', 'WAL', 'JOG', 'JUM', 'STD', 'FALL']\n",
      "[[98.8  0.3  0.   0.   0.9  0. ]\n",
      " [ 0.9 97.6  0.   0.   1.2  0.3]\n",
      " [ 1.6  0.1 93.4  4.8  0.   0. ]\n",
      " [ 2.4  0.   4.5 93.   0.1  0. ]\n",
      " [ 0.3  0.9  0.   0.  98.6  0.2]\n",
      " [ 0.3  0.   0.   0.  95.8  3.9]]\n"
     ]
    }
   ],
   "source": [
    "rae = keras.models.load_model(\"_RAE_model.hdf5\")\n",
    "rep_x_test = rae.predict(x_test, verbose=1)\n",
    "\n",
    "# model = keras.models.load_model('_best_FCN_.hdf5')\n",
    "preds = model.predict(rep_x_test, verbose=1)\n",
    "preds = preds.argmax(axis=1)\n",
    "cm_analysis(y_test[:,0], preds, act_list, figsize=(16,16))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "VcCUAKGoycAq"
   },
   "source": [
    "### How about gender information?\n",
    "So, we can hid falls and they are inferred as being steady, while we share the moving activites ('STR', 'WAL', 'JOG', 'JUM') with minimum distortion. \n",
    "The question is: what if somebody could infer our gender from them, when they are not supposed to infer it?!\n",
    "\n",
    "* First, let's see how's the accuracy of gender classifier on the output of RAE: So, we choose the required activites and give them to the already trained gender classifier."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 52
    },
    "colab_type": "code",
    "id": "Nq5k9Mm7ZWgI",
    "outputId": "522a343d-3e00-4e51-bb9b-665592420327"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Required--> STR; WAL; JOG; JUM; (107320, 128, 9, 1) (107320,) (107320,)\n",
      "(21340, 128, 9, 1) (21340,) (21340,)\n"
     ]
    }
   ],
   "source": [
    "print(\"Required\", end=\"--> \")\n",
    "wl = [0,1,2,3]\n",
    "for i in wl:\n",
    "    print(act_list[i], end=\"; \")\n",
    "\n",
    "w_train_data = x_train[np.isin(y_train[:,0],wl)]\n",
    "w_act_train_labels = y_train[np.isin(y_train[:,0],wl)][:,0]\n",
    "w_gen_train_labels = y_train[np.isin(y_train[:,0],wl)][:,1]\n",
    "print(w_train_data.shape,w_act_train_labels.shape, w_gen_train_labels.shape)\n",
    "\n",
    "w_test_data = x_test[np.isin(y_test[:,0],wl)]\n",
    "w_act_test_labels = y_test[np.isin(y_test[:,0],wl)][:,0]\n",
    "w_gen_test_labels = y_test[np.isin(y_test[:,0],wl)][:,1]\n",
    "print(w_test_data.shape,w_act_test_labels.shape, w_gen_test_labels.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 104
    },
    "colab_type": "code",
    "id": "Q-YsuCW1gIcL",
    "outputId": "efe4a510-9ed9-40df-bfdc-ff18510d5605"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "667/667 [==============================] - 96s 143ms/step\n",
      "['F', 'M']\n",
      "[[96.2  3.8]\n",
      " [ 2.3 97.7]]\n",
      "f1:  96.51\n",
      "acc:  97.31\n"
     ]
    }
   ],
   "source": [
    "eval_gen = keras.models.load_model('_best_gen_FCN_.hdf5')\n",
    "preds_ = eval_gen.predict(w_test_data, verbose=1)\n",
    "preds = (preds_ > 0.5).astype(int)[:,0]\n",
    "cm_analysis(w_gen_test_labels, preds, ['F','M'], figsize=(16,16))\n",
    "print(\"f1: \", np.round(f1_score(w_gen_test_labels, preds, average='macro')*100,2))\n",
    "print(\"acc: \", np.round(accuracy_score(w_gen_test_labels, preds)*100,2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 52
    },
    "colab_type": "code",
    "id": "7MZ2PteTgsUd",
    "outputId": "f4c7f78c-423d-4d1f-c543-aef57e90e3aa"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "667/667 [==============================] - 65s 98ms/step\n"
     ]
    }
   ],
   "source": [
    "w_data_train_rep = w_train_data.copy()\n",
    "# w_data_train_rep = rae.predict(w_data_train_rep, verbose=1)\n",
    "w_data_test_rep = w_test_data.copy()\n",
    "w_data_test_rep = rae.predict(w_data_test_rep, verbose=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 104
    },
    "colab_type": "code",
    "id": "b4g5NBs8y9W-",
    "outputId": "46eb5796-97e2-456e-e88b-2da3d0cb6f83"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "667/667 [==============================] - 95s 142ms/step\n",
      "['F', 'M']\n",
      "[[96.9  3.1]\n",
      " [ 4.  96. ]]\n",
      "f1:  95.21\n",
      "acc:  96.24\n"
     ]
    }
   ],
   "source": [
    "preds = eval_gen.predict(w_data_test_rep, verbose=1)\n",
    "preds = (preds > 0.5).astype(int)[:,0]\n",
    "cm_analysis(w_gen_test_labels, preds, ['F','M'], figsize=(16,16))\n",
    "print(\"f1: \", np.round(f1_score(w_gen_test_labels, preds, average='macro')*100,2))\n",
    "print(\"acc: \", np.round(accuracy_score(w_gen_test_labels, preds)*100,2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "6YSUegGsz08_"
   },
   "source": [
    "Thus, if the adversary look at the non-sensitive activities such as walking, they can accurately infer the gender. And as we see, even if adversaries do not use the raw data and just look at the output of RAE, they still can infer gender with high accuracy."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "ww2Ooaa52rq_"
   },
   "source": [
    "## Anonymization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Go9WeDEq0StN"
   },
   "source": [
    "Now, we build the AAE and give it the output of the RAE with the goal of hiding the gender."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "WCKf3Msngsph"
   },
   "outputs": [],
   "source": [
    "class Enc_Reg:\n",
    "    l2p = 0.001\n",
    "    @staticmethod\n",
    "    def early_layers(inp, fm, hid_act_func):\n",
    "        # Start\n",
    "        x = Conv2D(32, fm, padding=\"same\", kernel_regularizer=regularizers.l2(Enc_Reg.l2p), activation=hid_act_func)(inp)\n",
    "        x = MaxPooling2D(pool_size=(2, 1))(x)\n",
    "        x = Dropout(0.25)(x)\n",
    "        \n",
    "        # 1\n",
    "        x = Conv2D(32, fm, padding=\"same\", kernel_regularizer=regularizers.l2(Enc_Reg.l2p), activation=hid_act_func)(x)\n",
    "        x = MaxPooling2D(pool_size=(2, 1))(x)\n",
    "        x = Dropout(0.25)(x)\n",
    "\n",
    "        return x\n",
    "    \n",
    "    @staticmethod\n",
    "    def late_layers(inp, num_classes, fm, act_func, hid_act_func):\n",
    "        # 2\n",
    "        x = Conv2D(32, fm, padding=\"same\", kernel_regularizer=regularizers.l2(Enc_Reg.l2p), activation=hid_act_func)(inp)\n",
    "        x = MaxPooling2D(pool_size=(2, 1))(x)\n",
    "        x = Dropout(0.25)(x)\n",
    "        \n",
    "        # End\n",
    "        x = Flatten()(x)\n",
    "        x = Dense(64, kernel_regularizer=regularizers.l2(Enc_Reg.l2p), activation=hid_act_func)(x)\n",
    "        x = Dropout(0.5)(x)\n",
    "        x = Dense(16, kernel_regularizer=regularizers.l2(Enc_Reg.l2p), activation=hid_act_func)(x)\n",
    "        x = Dropout(0.5)(x)\n",
    "        x = Dense(num_classes, activation=act_func)(x)\n",
    "\n",
    "        return x\n",
    "   \n",
    "    @staticmethod\n",
    "    def build(height, width, num_classes, name, fm, act_func,hid_act_func):\n",
    "        inp = Input(shape=(height, width, 1))\n",
    "        early = Enc_Reg.early_layers(inp, fm, hid_act_func=hid_act_func)\n",
    "        late  = Enc_Reg.late_layers(early, num_classes, fm, act_func=act_func, hid_act_func=hid_act_func)\n",
    "        model = Model(inputs=inp, outputs=late ,name=name)\n",
    "        return model\n",
    "\n",
    "\n",
    "class Dec_Reg:\n",
    "    l2p = 0.001\n",
    "    @staticmethod\n",
    "    def early_layers(inp, fm, hid_act_func):\n",
    "        # Start\n",
    "        x = Conv2D(32, fm, padding=\"same\", kernel_regularizer=regularizers.l2(Dec_Reg.l2p), activation=hid_act_func)(inp)\n",
    "        x = MaxPooling2D(pool_size=(2, 1))(x)\n",
    "        x = Dropout(0.25)(x)\n",
    "        \n",
    "        # 1\n",
    "        x = Conv2D(32, fm, padding=\"same\", kernel_regularizer=regularizers.l2(Dec_Reg.l2p), activation=hid_act_func)(x)\n",
    "        x = MaxPooling2D(pool_size=(2, 1))(x)\n",
    "        x = Dropout(0.25)(x)\n",
    "\n",
    "\n",
    "        return x\n",
    "    \n",
    "    @staticmethod\n",
    "    def late_layers(inp, num_classes, fm, act_func, hid_act_func):\n",
    "        # 2\n",
    "        x = Conv2D(32, fm, padding=\"same\", kernel_regularizer=regularizers.l2(Dec_Reg.l2p), activation=hid_act_func)(inp)\n",
    "        x = MaxPooling2D(pool_size=(2, 1))(x)\n",
    "        x = Dropout(0.25)(x)\n",
    "        \n",
    "        # 3\n",
    "        x = Conv2D(32, fm, padding=\"same\", kernel_regularizer=regularizers.l2(Dec_Reg.l2p), activation=hid_act_func)(x)\n",
    "        x = MaxPooling2D(pool_size=(2, 1))(x)\n",
    "        x = Dropout(0.25)(x)\n",
    "        \n",
    "        #4\n",
    "        x = Conv2D(32, fm, padding=\"same\", kernel_regularizer=regularizers.l2(Dec_Reg.l2p), activation=hid_act_func)(x)\n",
    "        x = MaxPooling2D(pool_size=(2, 1))(x)\n",
    "        x = Dropout(0.25)(x)\n",
    "        \n",
    "        \n",
    "        # End\n",
    "        x = Flatten()(x)\n",
    "        x = Dense(128, kernel_regularizer=regularizers.l2(Dec_Reg.l2p), activation=hid_act_func)(x)\n",
    "        x = Dropout(0.5)(x)\n",
    "        x = Dense(32, kernel_regularizer=regularizers.l2(Dec_Reg.l2p), activation=hid_act_func)(x)\n",
    "        x = Dropout(0.5)(x)\n",
    "        x = Dense(num_classes, activation=act_func)(x)\n",
    "\n",
    "        return x\n",
    "   \n",
    "    @staticmethod\n",
    "    def build(height, width, num_classes, name, fm, act_func,hid_act_func):\n",
    "        inp = Input(shape=(height, width, 1))\n",
    "        early = Dec_Reg.early_layers(inp, fm, hid_act_func=hid_act_func)\n",
    "        late  = Dec_Reg.late_layers(early, num_classes, fm, act_func=act_func, hid_act_func=hid_act_func)\n",
    "        model = Model(inputs=inp, outputs=late ,name=name)\n",
    "        return model\n",
    "\n",
    "\n",
    "class Encoder:\n",
    "    l2p = 0.0001\n",
    "    @staticmethod\n",
    "    def layers(x, fm, act_func, hid_act_func):\n",
    "        x = Conv2D(64, fm, activation=hid_act_func, kernel_regularizer=regularizers.l2(Encoder.l2p), padding='same')(x)\n",
    "        x = BatchNormalization()(x)\n",
    "        \n",
    "        x = Conv2D(64, fm, activation=hid_act_func, kernel_regularizer=regularizers.l2(Encoder.l2p), padding='same')(x)\n",
    "        x = BatchNormalization()(x)\n",
    "        x = MaxPooling2D(pool_size=(2,1))(x)\n",
    "\n",
    "        x = Conv2D(64, fm, activation=hid_act_func,kernel_regularizer=regularizers.l2(Encoder.l2p), padding='same')(x)\n",
    "        x = BatchNormalization()(x)\n",
    "        x = MaxPooling2D(pool_size=(2,1))(x)\n",
    "\n",
    "        x = Conv2D(1, fm, activation=act_func, padding='same')(x) \n",
    "        y = BatchNormalization()(x)\n",
    "\n",
    "        return y\n",
    "   \n",
    "    @staticmethod\n",
    "    def build(height, width, fm, act_func, hid_act_func):\n",
    "        inp = Input(shape=(height, width,1))\n",
    "        enc = Encoder.layers(inp, fm, act_func=act_func, hid_act_func=hid_act_func)\n",
    "        model = Model(inputs=inp, outputs=enc ,name=\"Encoder\")\n",
    "        return model\n",
    "\n",
    "class Decoder:\n",
    "    l2p = 0.0001\n",
    "    @staticmethod\n",
    "    def layers(y, height, width, fm, act_func, hid_act_func):\n",
    "        \n",
    "        x = Conv2DTranspose(64, fm, strides = (1, 1), activation=hid_act_func,kernel_regularizer=regularizers.l2(Decoder.l2p), padding='same')(y)\n",
    "        x = BatchNormalization()(x)\n",
    "        x = Conv2DTranspose(64, fm,  strides = (2, 1), activation=hid_act_func,kernel_regularizer=regularizers.l2(Decoder.l2p), padding='same')(x)\n",
    "        x = BatchNormalization()(x)\n",
    "        x = Conv2DTranspose(64, fm, strides = (2, 1), activation=hid_act_func,kernel_regularizer=regularizers.l2(Decoder.l2p), padding='same')(x)\n",
    "        x = BatchNormalization()(x)\n",
    "        \n",
    "        xh = Conv2D(1, fm, activation=act_func, padding='same')(x)\n",
    "        return xh\n",
    "   \n",
    "    @staticmethod\n",
    "    def build(height, width, fm , act_func, hid_act_func):\n",
    "        inp = Input(shape=(height, width,1))\n",
    "        dec  = Decoder.layers(inp,height, width, fm, act_func=act_func, hid_act_func=hid_act_func)\n",
    "        model = Model(inputs=inp, outputs=dec ,name=\"Decoder\")\n",
    "        return model        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 345
    },
    "colab_type": "code",
    "id": "e9LKF6Q6hZkW",
    "outputId": "d1169282-5371-4289-d534-af81e19a002b"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"anon\"\n",
      "__________________________________________________________________________________________________\n",
      "Layer (type)                    Output Shape         Param #     Connected to                     \n",
      "==================================================================================================\n",
      "input_5 (InputLayer)            [(None, 128, 9, 1)]  0                                            \n",
      "__________________________________________________________________________________________________\n",
      "Encoder (Functional)            (None, 32, 9, 1)     375365      input_5[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "Decoder (Functional)            (None, 128, 9, 1)    375361      Encoder[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "GenReg (Functional)             (None, 1)            337665      Decoder[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "ActReg (Functional)             (None, 4)            337764      Decoder[0][0]                    \n",
      "==================================================================================================\n",
      "Total params: 1,426,155\n",
      "Trainable params: 749,956\n",
      "Non-trainable params: 676,199\n",
      "__________________________________________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "import keras.backend as K\n",
    "def gen_equ_loss_func(y_true, y_pred):\n",
    "    loss = K.mean(K.abs(0.5 - y_pred))\n",
    "    return loss\n",
    "\n",
    "def build_AAE(loss_weights):\n",
    "    id_class_numbers = 1\n",
    "    act_class_numbers = 4\n",
    "    #fm = (2,3)\n",
    "    #reps_id = Enc_Reg.build(height, width//4, id_class_numbers, name =\"EncReg\", fm=fm, act_func=\"sigmoid\",hid_act_func=\"relu\")\n",
    "    fm = (5,9)\n",
    "    rcon_id = Dec_Reg.build(height, width, id_class_numbers, name =\"GenReg\", fm=fm, act_func=\"sigmoid\",hid_act_func=\"relu\")\n",
    "    # print(rcon_id.summary())\n",
    "    rcon_task = Dec_Reg.build(height, width, act_class_numbers, name =\"ActReg\", fm=fm, act_func=\"softmax\",hid_act_func=\"relu\")\n",
    "    # print(rcon_task.summary())\n",
    "    #reps_id.compile( loss=\"binary_crossentropy\", optimizer='adam', metrics=['acc'])\n",
    "    rcon_id.compile( loss=\"binary_crossentropy\", optimizer='adam', metrics=['acc'])\n",
    "    rcon_task.compile( loss=\"categorical_crossentropy\", optimizer='adam', metrics=['acc'])\n",
    "\n",
    "    #reps_id.trainable = False\n",
    "    rcon_id.trainable = False\n",
    "    rcon_task.trainable = False\n",
    "\n",
    "    enc_to_reps = Encoder.build(height, width, fm=fm, act_func=\"linear\", hid_act_func=\"relu\")\n",
    "    # print(enc_to_reps.summary())\n",
    "    reps_to_dec = Decoder.build(height//4, width, fm=fm, act_func=\"linear\", hid_act_func=\"relu\")\n",
    "    # print(reps_to_dec.summary())\n",
    "    enc_to_reps.compile( loss=\"mean_squared_error\", optimizer='adam', metrics=['mse'])\n",
    "    reps_to_dec.compile( loss=\"mean_squared_error\", optimizer='adam', metrics=['mse'])\n",
    "\n",
    "    x = Input(shape=(height, width,1))\n",
    "    z = enc_to_reps(x)\n",
    "    #idz = reps_id(z)\n",
    "    xh = reps_to_dec(z)\n",
    "    idxh = rcon_id(xh)\n",
    "    txh = rcon_task(xh)\n",
    "\n",
    "\n",
    "\n",
    "    anon_model = Model(inputs = x,\n",
    "                       outputs = [xh,\n",
    "                                  #idz,\n",
    "                                  idxh,\n",
    "                                  txh\n",
    "                                 ],\n",
    "                       name =\"anon\") \n",
    "    anon_model.compile(loss = [\"mean_squared_error\",\n",
    "                               #\"binary_crossentropy\",\n",
    "                                gen_equ_loss_func,\n",
    "                               \"categorical_crossentropy\"\n",
    "                              ],\n",
    "                       loss_weights = loss_weights,                 \n",
    "                       optimizer = \"adam\",\n",
    "                       metrics = [\"acc\"])\n",
    "    #enc_to_reps.set_weights(enc_dec_tmp.layers[1].get_weights()) \n",
    "    #reps_to_dec.set_weights(enc_dec_tmp.layers[2].get_weights()) \n",
    "\n",
    "\n",
    "    return anon_model, rcon_task, rcon_id\n",
    "\n",
    "from tensorflow.keras.layers import *\n",
    "from tensorflow.keras import Model\n",
    "from tensorflow.keras import regularizers\n",
    "height = w_data_train_rep.shape[1]\n",
    "width = w_data_train_rep.shape[2]\n",
    "fm = (5,9)\n",
    "loss_weights=[2, 1, 4]            \n",
    "anon_model, rcon_task, rcon_id = build_AAE(loss_weights)\n",
    "anon_model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "fkTCMCbm6DEu"
   },
   "source": [
    "Here we can see the results of anonymization by AAE (hiding the gender in this case) for three different settings of trade-off parameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 259
    },
    "colab_type": "code",
    "id": "wer1NoUHv5oX",
    "outputId": "1360c7be-9cb0-4d96-80de-d5cdb007c8b5"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1160/1160 [==============================] - 358s 309ms/step\n",
      "1160/1160 [==============================] - 158s 136ms/step\n",
      "['STR', 'WAL', 'JOG', 'JUM', 'STD', 'FALL']\n",
      "[[98.5  0.5  0.   0.   0.9  0. ]\n",
      " [ 1.5 97.1  0.   0.   1.1  0.3]\n",
      " [ 3.3  0.1 92.1  4.5  0.   0. ]\n",
      " [ 4.6  0.   4.4 91.   0.   0. ]\n",
      " [ 0.5  1.   0.   0.  95.8  2.7]\n",
      " [ 0.7  0.   0.   0.  95.6  3.7]]\n",
      "1160/1160 [==============================] - 155s 134ms/step\n",
      "['F', 'M']\n",
      "[[61.1 38.9]\n",
      " [13.8 86.2]]\n",
      "f1:  73.59\n",
      "acc:  79.71\n"
     ]
    }
   ],
   "source": [
    "anon_model.load_weights('gender_anon_1.h5')\n",
    "rep_anon_test = anon_model.predict(rep_x_test, verbose = 1)[0]\n",
    "\n",
    "\n",
    "eval_act = keras.models.load_model('_best_FCN_.hdf5')\n",
    "preds = eval_act.predict(rep_anon_test, verbose=1)\n",
    "preds = preds.argmax(axis=1)\n",
    "cm_analysis(y_test[:,0], preds, act_list, figsize=(16,16))\n",
    "\n",
    "\n",
    "eval_gen = keras.models.load_model('_best_gen_FCN_.hdf5')\n",
    "preds = eval_gen.predict(rep_anon_test, verbose=1)\n",
    "preds = (preds > 0.5).astype(int)[:,0]\n",
    "cm_analysis(y_test[:,1], preds, ['F','M'], figsize=(16,16))\n",
    "print(\"f1: \", np.round(f1_score(y_test[:,1], preds, average='macro')*100,2))\n",
    "print(\"acc: \", np.round(accuracy_score(y_test[:,1], preds)*100,2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 259
    },
    "colab_type": "code",
    "id": "DRiCBsaN5w64",
    "outputId": "84206b43-742c-4c51-958d-d92a269454bb"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1160/1160 [==============================] - 350s 302ms/step\n",
      "1160/1160 [==============================] - 154s 133ms/step\n",
      "['STR', 'WAL', 'JOG', 'JUM', 'STD', 'FALL']\n",
      "[[99.3  0.3  0.   0.   0.4  0. ]\n",
      " [ 3.5 95.1  0.   0.   1.1  0.3]\n",
      " [ 4.   0.1 92.2  3.7  0.   0. ]\n",
      " [ 6.   0.   4.9 89.   0.   0. ]\n",
      " [ 1.4  0.9  0.   0.  96.3  1.4]\n",
      " [ 1.   0.   0.   0.  95.3  3.7]]\n",
      "['F', 'M']\n",
      "[[69.2 30.8]\n",
      " [30.2 69.8]]\n",
      "f1:  65.67\n",
      "acc:  69.61\n"
     ]
    }
   ],
   "source": [
    "anon_model.load_weights('gender_anon_2.h5')\n",
    "\n",
    "rep_anon_test = anon_model.predict(rep_x_test, verbose = 1)[0]\n",
    "\n",
    "eval_act = keras.models.load_model('_best_FCN_.hdf5')\n",
    "preds = eval_act.predict(rep_anon_test, verbose=1)\n",
    "preds = preds.argmax(axis=1)\n",
    "cm_analysis(y_test[:,0], preds, act_list, figsize=(16,16))\n",
    "\n",
    "\n",
    "eval_gen = keras.models.load_model('_best_gen_FCN_.hdf5')\n",
    "preds = eval_gen.predict(rep_anon_test)\n",
    "preds = (preds > 0.5).astype(int)[:,0]\n",
    "cm_analysis(y_test[:,1], preds, ['F','M'], figsize=(16,16))\n",
    "print(\"f1: \", np.round(f1_score(y_test[:,1], preds, average='macro')*100,2))\n",
    "print(\"acc: \", np.round(accuracy_score(y_test[:,1], preds)*100,2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 259
    },
    "colab_type": "code",
    "id": "r_Qnbhdm1mCP",
    "outputId": "6af69928-79f4-416b-9bf9-71dc82a53a9f"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1160/1160 [==============================] - 467s 403ms/step\n",
      "1160/1160 [==============================] - 152s 131ms/step\n",
      "['STR', 'WAL', 'JOG', 'JUM', 'STD', 'FALL']\n",
      "[[99.1  0.6  0.   0.   0.4  0. ]\n",
      " [ 4.2 94.3  0.   0.   1.1  0.4]\n",
      " [ 2.5  0.1 93.4  4.   0.   0. ]\n",
      " [ 5.4  0.   5.2 89.4  0.   0. ]\n",
      " [ 5.   1.   0.   0.  92.6  1.4]\n",
      " [ 0.8  0.   0.   0.  94.5  4.7]]\n",
      "1160/1160 [==============================] - 152s 131ms/step\n",
      "['F', 'M']\n",
      "[[69.4 30.6]\n",
      " [36.1 63.9]]\n",
      "f1:  62.01\n",
      "acc:  65.29\n"
     ]
    }
   ],
   "source": [
    "anon_model.load_weights('gender_anon_3.h5')\n",
    "rep_anon_test = anon_model.predict(rep_x_test, verbose = 1)[0]\n",
    "\n",
    "\n",
    "eval_act = keras.models.load_model('_best_FCN_.hdf5')\n",
    "preds = eval_act.predict(rep_anon_test, verbose=1)\n",
    "preds = preds.argmax(axis=1)\n",
    "cm_analysis(y_test[:,0], preds, act_list, figsize=(16,16))\n",
    "\n",
    "\n",
    "eval_gen = keras.models.load_model('_best_gen_FCN_.hdf5')\n",
    "preds = eval_gen.predict(rep_anon_test,verbose=1)\n",
    "preds = (preds > 0.5).astype(int)[:,0]\n",
    "cm_analysis(y_test[:,1], preds, ['F','M'], figsize=(16,16))\n",
    "print(\"f1: \", np.round(f1_score(y_test[:,1], preds, average='macro')*100,2))\n",
    "print(\"acc: \", np.round(accuracy_score(y_test[:,1], preds)*100,2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Qmrt1Yej69nU"
   },
   "source": [
    "Finally, we see that after using the AAE we suffer a bit more accuracy loss. However, inferring the gender is so close to the random guess on this dataset."
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "pmc_additional_experiment.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}