tutorial/pmc_additional_experiment.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Uw4lRlTQ5e47"
   },
   "source": [
    "### Addtional Resutls for our PMC Journal Paper: [Privacy and Utility Preserving Sensor-Data Transformations](https://arxiv.org/pdf/1911.05996.pdf)\n",
    "\n",
    "#### (1) Brief Description of the Resutls\n",
    "To show that the compound architecture, proposed in the paper, can generalize across datasets, we repeat the same experiment as Table 9 (in the [paper](https://arxiv.org/pdf/1911.05996.pdf)) on another dataset ([MobiAct](https://bmi.teicrete.gr/en/the-mobifall-and-mobiact-datasets-2/)) by keeping the same architecture for RAE and AAE.\n",
    "\n",
    "In this experiment we consider all kind of **falls** as **sensitive** activities, assuming that they can be considered as symptoms of some diseases. We also consider being **steady** as a **neutral** activity, which in this dataset it means either **sitting** or **standing**. \n",
    "\n",
    "We see summary of the results for two different settings for utility-privacy parameters in the below image, that show almost similar results to what we have on the [MotionSense](https://www.kaggle.com/malekzadeh/motionsense-dataset) dataset in Table 9 in the paper.\n",
    "\n",
    "Accuracy of recognizing the **required** activities are almost same before and after transformations, while accuracy in detecting falls is dropped from 99.6\\% to less than 4.5\\%. Moreover, we can reduce the adversary's accuracy in detecting gender from 97.35\\% to 66.8\\%, which is close to the random guess in this dataset that is 74.5\\%.\n",
    "\n",
    "Note that in this dataset we have 41 males and 14 females. So, randomly guessing a subject as male is 74.5\\% accurate.($\\frac{41 \\text{ males}}{55 \\text{ males and females}}=74.5$).}\n",
    "\n",
    "| <img src=\"additional_mobi_act_pmc_paper.png\" class=\"img-responsive\"> |\n",
    "|:---:|\n",
    "| Reproducing results of Table 9 in the paper (MotionSens dataset) on another dataset (MobiActdataset). |\n",
    "\n",
    "* Note that this notebook just use all the already trained models. If you want to train your own models for each stage, please look at other files in this repository.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "B6A72LVl22Nt"
   },
   "source": [
    "#### (2) Import required libraries "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 64
    },
    "colab_type": "code",
    "id": "llnhh0F0KLiJ",
    "outputId": "d08d50da-f0c3-4429-d799-b8660c4e6fb5"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<p style=\"color: red;\">\n",
       "The default version of TensorFlow in Colab will soon switch to TensorFlow 2.x.<br>\n",
       "We recommend you <a href=\"https://www.tensorflow.org/guide/migrate\" target=\"_blank\">upgrade</a> now \n",
       "or ensure your notebook will continue to use TensorFlow 1.x via the <code>%tensorflow_version 1.x</code> magic:\n",
       "<a href=\"https://colab.research.google.com/notebooks/tensorflow_version.ipynb\" target=\"_blank\">more info</a>.</p>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import tensorflow as tf\n",
    "import tensorflow.keras as keras\n",
    "from scipy import stats\n",
    "pd.set_option('display.float_format', lambda x: '%.4f' % x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "DqUBl8qX3BzJ"
   },
   "source": [
    "#### (3) Dataset\n",
    "(Dataset is available here: [The MobiFall and MobiAct datasets](https://bmi.teicrete.gr/en/the-mobifall-and-mobiact-datasets-2/))\n",
    "\n",
    "We choose 6 activities in the dataset.\n",
    "\n",
    "* **STR**: satir-stepping, including both staris down or stairs up\"\n",
    "* **WAL**: walking\n",
    "* **JOG**: jogging\n",
    "* **JUM**: jumpping\n",
    "* **STD**: being steady: either sitting or standing \n",
    "* **FALL**: falling (Suddenly from standing position to the floor)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "OGhE_W4swYkP"
   },
   "source": [
    "#### Name and Code of each Class"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 52
    },
    "colab_type": "code",
    "id": "lXvf_Vp4_4mi",
    "outputId": "b2009f68-2e97-4520-e47f-0edb13b473b3"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 : STR | 1 : WAL | 2 : JOG | 3 : JUM | 4 : STD | 5 : FALL | \n",
      "0 : Female | 1 : Male | "
     ]
    }
   ],
   "source": [
    "act_list = [\"STR\",\"WAL\",\"JOG\",\"JUM\",\"STD\",\"FALL\"]\n",
    "gender_labels = [\"Female\", \"Male\"]\n",
    "for i, a in enumerate(act_list):\n",
    "  print(i,\":\",a, end=\" | \")\n",
    "print()\n",
    "for i, a in enumerate(gender_labels):\n",
    "  print(i,\":\",a, end=\" | \")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "611b6GDtwc3i"
   },
   "source": [
    "#### Loading Train and Test Data\n",
    "(Note: File XXXX shows hos we build this datasets from the original MobiAct dataset)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "2g0dtCE9_fpc"
   },
   "outputs": [],
   "source": [
    "x_train = np.load(\"x_train.npy\", allow_pickle=True)\n",
    "x_test = np.load(\"x_test.npy\", allow_pickle=True)\n",
    "y_train = np.load(\"y_train.npy\",allow_pickle=True)\n",
    "y_test = np.load(\"y_test.npy\",allow_pickle=True)\n",
    "y_train = y_train.astype(int)\n",
    "y_test = y_test.astype(int)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "colab_type": "code",
    "id": "Imn-IZQhKLmg",
    "outputId": "7bfe65d3-1918-47a1-912a-887a620b09eb"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((182220, 128, 9, 1), (182220, 6), (38617, 128, 9, 1), (38617, 6))"
      ]
     },
     "execution_count": 6,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nb_classes = len(np.unique(y_test[:,0]))\n",
    "batch_size = 64\n",
    "\n",
    "Y_train = keras.utils.to_categorical(y_train[:,0], nb_classes)\n",
    "Y_test = keras.utils.to_categorical(y_test[:,0], nb_classes)\n",
    "x_train = x_train.reshape((x_train.shape[0], x_train.shape[1], x_train.shape[2] ,1))\n",
    "x_test = x_test.reshape((x_test.shape[0], x_test.shape[1], x_test.shape[2] ,1))\n",
    "x_train.shape, Y_train.shape, x_test.shape, Y_test.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "oGOJKQRDr_Xj"
   },
   "outputs": [],
   "source": [
    "np.set_printoptions(suppress=True)\n",
    "\n",
    "import seaborn as sns\n",
    "from sklearn.metrics import confusion_matrix\n",
    "\n",
    "## src: https://gist.github.com/hitvoice/36cf44689065ca9b927431546381a3f7\n",
    "def cm_analysis(y_true, y_pred, labels, ymap=None, figsize=(10,10)):\n",
    "    cm = confusion_matrix(y_true, y_pred, labels=range(len(labels)))\n",
    "    cm_sum = np.sum(cm, axis=1, keepdims=True)\n",
    "    cm_perc = cm / cm_sum.astype(float) * 100\n",
    "    print(labels)\n",
    "    print(cm_perc.round(1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "JM4tNfMixZi_"
   },
   "source": [
    "#### Original Data (without any transformation)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "fTbMvrq5xgpp"
   },
   "source": [
    "##### Activity Recognition"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 400
    },
    "colab_type": "code",
    "id": "ptZ92_7PXeh4",
    "outputId": "a9bfbb76-7a67-4c20-f62b-593a970b24a0"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/init_ops.py:97: calling GlorotUniform.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Call initializer instance with the dtype argument instead of passing it to the constructor\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/init_ops.py:97: calling Zeros.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Call initializer instance with the dtype argument instead of passing it to the constructor\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/init_ops.py:97: calling Ones.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Call initializer instance with the dtype argument instead of passing it to the constructor\n",
      "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "If using Keras pass *_constraint arguments to layers.\n",
      "['STR', 'WAL', 'JOG', 'JUM', 'STD', 'FALL']\n",
      "[[98.5  0.5  0.2  0.   0.7  0.1]\n",
      " [ 0.9 97.8  0.   0.   1.2  0.2]\n",
      " [ 0.6  0.1 94.5  4.8  0.   0. ]\n",
      " [ 1.6  0.   5.1 93.2  0.1  0. ]\n",
      " [ 0.1  1.   0.   0.  98.6  0.2]\n",
      " [ 0.   0.   0.   0.   0.4 99.6]]\n",
      "f1:  97.07\n",
      "acc:  97.55\n"
     ]
    }
   ],
   "source": [
    "model = keras.models.load_model('_best_FCN_.hdf5')\n",
    "preds = model.predict(x_test)\n",
    "preds = preds.argmax(axis=1)\n",
    "cm_analysis(y_test[:,0], preds, act_list, figsize=(16,16))\n",
    "\n",
    "from sklearn.metrics import f1_score\n",
    "print(\"f1: \", np.round(f1_score(y_test[:,0], preds, average='macro')*100,2))\n",
    "print(\"acc: \", np.round(model.evaluate(x_test, Y_test, verbose=0)[1]*100,2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "BYi1SkosMdK8"
   },
   "source": [
    "## Replacement"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "dKRM-Ew9x6SP"
   },
   "source": [
    "As we said, we do not want the app to infer when falls happen, while we want them to infer the other four moving activites ('STR', 'WAL', 'JOG', 'JUM') as accurate as possible."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 245
    },
    "colab_type": "code",
    "id": "G5VcEUDPMutF",
    "outputId": "561d6dbe-afe6-4a8c-c3ba-6eb13149a79c"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/keras/backend.py:4277: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use tf.where in 2.0, which has the same broadcast rule as np.where\n",
      "38617/38617 [==============================] - 7s 171us/sample\n",
      "38617/38617 [==============================] - 5s 134us/sample\n",
      "['STR', 'WAL', 'JOG', 'JUM', 'STD', 'FALL']\n",
      "[[98.4  0.5  0.1  0.   1.   0. ]\n",
      " [ 1.5 96.9  0.   0.   1.3  0.3]\n",
      " [ 1.7  0.1 93.4  4.8  0.   0. ]\n",
      " [ 2.3  0.   4.5 93.2  0.   0. ]\n",
      " [ 0.3  0.9  0.   0.  98.5  0.2]\n",
      " [ 0.3  0.   0.   0.  96.1  3.6]]\n"
     ]
    }
   ],
   "source": [
    "rae = keras.models.load_model(\"_RAE_model.hdf5\")\n",
    "rep_x_test = rae.predict(x_test, verbose=1)\n",
    "\n",
    "# model = keras.models.load_model('_best_FCN_.hdf5')\n",
    "preds = model.predict(rep_x_test, verbose=1)\n",
    "preds = preds.argmax(axis=1)\n",
    "cm_analysis(y_test[:,0], preds, act_list, figsize=(16,16))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "VcCUAKGoycAq"
   },
   "source": [
    "### How about gender information?\n",
    "So, we can hid falls and they are inferred as being steady, while we share the moving activites ('STR', 'WAL', 'JOG', 'JUM') with minimum distortion. \n",
    "The question is: what if somebody could infer our gender from them, when they are not supposed to infer it?!\n",
    "\n",
    "* First, let's see how's the accuracy of gender classifier on the output of RAE: So, we choose the required activites and give them to the already trained gender classifier."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 52
    },
    "colab_type": "code",
    "id": "Nq5k9Mm7ZWgI",
    "outputId": "522a343d-3e00-4e51-bb9b-665592420327"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Required--> STR; WAL; JOG; JUM; (111709, 128, 9, 1) (111709,) (111709,)\n",
      "(22568, 128, 9, 1) (22568,) (22568,)\n"
     ]
    }
   ],
   "source": [
    "print(\"Required\", end=\"--> \")\n",
    "wl = [0,1,2,3]\n",
    "for i in wl:\n",
    "    print(act_list[i], end=\"; \")\n",
    "\n",
    "w_train_data = x_train[np.isin(y_train[:,0],wl)]\n",
    "w_act_train_labels = y_train[np.isin(y_train[:,0],wl)][:,0]\n",
    "w_gen_train_labels = y_train[np.isin(y_train[:,0],wl)][:,1]\n",
    "print(w_train_data.shape,w_act_train_labels.shape, w_gen_train_labels.shape)\n",
    "\n",
    "w_test_data = x_test[np.isin(y_test[:,0],wl)]\n",
    "w_act_test_labels = y_test[np.isin(y_test[:,0],wl)][:,0]\n",
    "w_gen_test_labels = y_test[np.isin(y_test[:,0],wl)][:,1]\n",
    "print(w_test_data.shape,w_act_test_labels.shape, w_gen_test_labels.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 104
    },
    "colab_type": "code",
    "id": "Q-YsuCW1gIcL",
    "outputId": "efe4a510-9ed9-40df-bfdc-ff18510d5605"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['F', 'M']\n",
      "[[96.4  3.6]\n",
      " [ 2.3 97.7]]\n",
      "f1:  96.74\n",
      "acc:  97.35\n"
     ]
    }
   ],
   "source": [
    "eval_gen = keras.models.load_model('_best_gen_FCN_.hdf5')\n",
    "preds = eval_gen.predict(w_test_data)\n",
    "preds = (preds > 0.5).astype(int)[:,0]\n",
    "cm_analysis(w_gen_test_labels, preds, ['F','M'], figsize=(16,16))\n",
    "print(\"f1: \", np.round(f1_score(w_gen_test_labels, preds, average='macro')*100,2))\n",
    "print(\"acc: \", np.round(eval_gen.evaluate(w_test_data, w_gen_test_labels, verbose=0)[1]*100,2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 52
    },
    "colab_type": "code",
    "id": "7MZ2PteTgsUd",
    "outputId": "f4c7f78c-423d-4d1f-c543-aef57e90e3aa"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "111709/111709 [==============================] - 20s 176us/sample\n",
      "22568/22568 [==============================] - 4s 171us/sample\n"
     ]
    }
   ],
   "source": [
    "w_data_train_rep = w_train_data.copy()\n",
    "w_data_train_rep = rae.predict(w_data_train_rep, verbose=1)\n",
    "w_data_test_rep = w_test_data.copy()\n",
    "w_data_test_rep = rae.predict(w_data_test_rep, verbose=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 104
    },
    "colab_type": "code",
    "id": "b4g5NBs8y9W-",
    "outputId": "46eb5796-97e2-456e-e88b-2da3d0cb6f83"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['F', 'M']\n",
      "[[96.7  3.3]\n",
      " [ 3.8 96.2]]\n",
      "f1:  95.57\n",
      "acc:  96.35\n"
     ]
    }
   ],
   "source": [
    "preds = eval_gen.predict(w_data_test_rep)\n",
    "preds = (preds > 0.5).astype(int)[:,0]\n",
    "cm_analysis(w_gen_test_labels, preds, ['F','M'], figsize=(16,16))\n",
    "print(\"f1: \", np.round(f1_score(w_gen_test_labels, preds, average='macro')*100,2))\n",
    "print(\"acc: \", np.round(eval_gen.evaluate(w_data_test_rep, w_gen_test_labels, verbose=0)[1]*100,2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "6YSUegGsz08_"
   },
   "source": [
    "Thus, if the adversary look at the non-sensitive activities such as walking, they can accurately infer the gender. And as we see, even if adversaries do not use the raw data and just look at the output of RAE, they still can infer gender with high accuracy."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "ww2Ooaa52rq_"
   },
   "source": [
    "## Anonymization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Go9WeDEq0StN"
   },
   "source": [
    "Now, we build the AAE and give it the output of the RAE with the goal of hiding the gender."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 0,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "WCKf3Msngsph"
   },
   "outputs": [],
   "source": [
    "class Enc_Reg:\n",
    "    l2p = 0.001\n",
    "    @staticmethod\n",
    "    def early_layers(inp, fm, hid_act_func):\n",
    "        # Start\n",
    "        x = Conv2D(32, fm, padding=\"same\", kernel_regularizer=regularizers.l2(Enc_Reg.l2p), activation=hid_act_func)(inp)\n",
    "        x = MaxPooling2D(pool_size=(2, 1))(x)\n",
    "        x = Dropout(0.25)(x)\n",
    "        \n",
    "        # 1\n",
    "        x = Conv2D(32, fm, padding=\"same\", kernel_regularizer=regularizers.l2(Enc_Reg.l2p), activation=hid_act_func)(x)\n",
    "        x = MaxPooling2D(pool_size=(2, 1))(x)\n",
    "        x = Dropout(0.25)(x)\n",
    "\n",
    "        return x\n",
    "    \n",
    "    @staticmethod\n",
    "    def late_layers(inp, num_classes, fm, act_func, hid_act_func):\n",
    "        # 2\n",
    "        x = Conv2D(32, fm, padding=\"same\", kernel_regularizer=regularizers.l2(Enc_Reg.l2p), activation=hid_act_func)(inp)\n",
    "        x = MaxPooling2D(pool_size=(2, 1))(x)\n",
    "        x = Dropout(0.25)(x)\n",
    "        \n",
    "        # End\n",
    "        x = Flatten()(x)\n",
    "        x = Dense(64, kernel_regularizer=regularizers.l2(Enc_Reg.l2p), activation=hid_act_func)(x)\n",
    "        x = Dropout(0.5)(x)\n",
    "        x = Dense(16, kernel_regularizer=regularizers.l2(Enc_Reg.l2p), activation=hid_act_func)(x)\n",
    "        x = Dropout(0.5)(x)\n",
    "        x = Dense(num_classes, activation=act_func)(x)\n",
    "\n",
    "        return x\n",
    "   \n",
    "    @staticmethod\n",
    "    def build(height, width, num_classes, name, fm, act_func,hid_act_func):\n",
    "        inp = Input(shape=(height, width, 1))\n",
    "        early = Enc_Reg.early_layers(inp, fm, hid_act_func=hid_act_func)\n",
    "        late  = Enc_Reg.late_layers(early, num_classes, fm, act_func=act_func, hid_act_func=hid_act_func)\n",
    "        model = Model(inputs=inp, outputs=late ,name=name)\n",
    "        return model\n",
    "\n",
    "\n",
    "class Dec_Reg:\n",
    "    l2p = 0.001\n",
    "    @staticmethod\n",
    "    def early_layers(inp, fm, hid_act_func):\n",
    "        # Start\n",
    "        x = Conv2D(32, fm, padding=\"same\", kernel_regularizer=regularizers.l2(Dec_Reg.l2p), activation=hid_act_func)(inp)\n",
    "        x = MaxPooling2D(pool_size=(2, 1))(x)\n",
    "        x = Dropout(0.25)(x)\n",
    "        \n",
    "        # 1\n",
    "        x = Conv2D(32, fm, padding=\"same\", kernel_regularizer=regularizers.l2(Dec_Reg.l2p), activation=hid_act_func)(x)\n",
    "        x = MaxPooling2D(pool_size=(2, 1))(x)\n",
    "        x = Dropout(0.25)(x)\n",
    "\n",
    "\n",
    "        return x\n",
    "    \n",
    "    @staticmethod\n",
    "    def late_layers(inp, num_classes, fm, act_func, hid_act_func):\n",
    "        # 2\n",
    "        x = Conv2D(32, fm, padding=\"same\", kernel_regularizer=regularizers.l2(Dec_Reg.l2p), activation=hid_act_func)(inp)\n",
    "        x = MaxPooling2D(pool_size=(2, 1))(x)\n",
    "        x = Dropout(0.25)(x)\n",
    "        \n",
    "        # 3\n",
    "        x = Conv2D(32, fm, padding=\"same\", kernel_regularizer=regularizers.l2(Dec_Reg.l2p), activation=hid_act_func)(x)\n",
    "        x = MaxPooling2D(pool_size=(2, 1))(x)\n",
    "        x = Dropout(0.25)(x)\n",
    "        \n",
    "        #4\n",
    "        x = Conv2D(32, fm, padding=\"same\", kernel_regularizer=regularizers.l2(Dec_Reg.l2p), activation=hid_act_func)(x)\n",
    "        x = MaxPooling2D(pool_size=(2, 1))(x)\n",
    "        x = Dropout(0.25)(x)\n",
    "        \n",
    "        \n",
    "        # End\n",
    "        x = Flatten()(x)\n",
    "        x = Dense(128, kernel_regularizer=regularizers.l2(Dec_Reg.l2p), activation=hid_act_func)(x)\n",
    "        x = Dropout(0.5)(x)\n",
    "        x = Dense(32, kernel_regularizer=regularizers.l2(Dec_Reg.l2p), activation=hid_act_func)(x)\n",
    "        x = Dropout(0.5)(x)\n",
    "        x = Dense(num_classes, activation=act_func)(x)\n",
    "\n",
    "        return x\n",
    "   \n",
    "    @staticmethod\n",
    "    def build(height, width, num_classes, name, fm, act_func,hid_act_func):\n",
    "        inp = Input(shape=(height, width, 1))\n",
    "        early = Dec_Reg.early_layers(inp, fm, hid_act_func=hid_act_func)\n",
    "        late  = Dec_Reg.late_layers(early, num_classes, fm, act_func=act_func, hid_act_func=hid_act_func)\n",
    "        model = Model(inputs=inp, outputs=late ,name=name)\n",
    "        return model\n",
    "\n",
    "\n",
    "class Encoder:\n",
    "    l2p = 0.0001\n",
    "    @staticmethod\n",
    "    def layers(x, fm, act_func, hid_act_func):\n",
    "        x = Conv2D(64, fm, activation=hid_act_func, kernel_regularizer=regularizers.l2(Encoder.l2p), padding='same')(x)\n",
    "        x = BatchNormalization()(x)\n",
    "        \n",
    "        x = Conv2D(64, fm, activation=hid_act_func, kernel_regularizer=regularizers.l2(Encoder.l2p), padding='same')(x)\n",
    "        x = BatchNormalization()(x)\n",
    "        x = MaxPooling2D(pool_size=(2,1))(x)\n",
    "\n",
    "        x = Conv2D(64, fm, activation=hid_act_func,kernel_regularizer=regularizers.l2(Encoder.l2p), padding='same')(x)\n",
    "        x = BatchNormalization()(x)\n",
    "        x = MaxPooling2D(pool_size=(2,1))(x)\n",
    "\n",
    "        x = Conv2D(1, fm, activation=act_func, padding='same')(x) \n",
    "        y = BatchNormalization()(x)\n",
    "\n",
    "        return y\n",
    "   \n",
    "    @staticmethod\n",
    "    def build(height, width, fm, act_func, hid_act_func):\n",
    "        inp = Input(shape=(height, width,1))\n",
    "        enc = Encoder.layers(inp, fm, act_func=act_func, hid_act_func=hid_act_func)\n",
    "        model = Model(inputs=inp, outputs=enc ,name=\"Encoder\")\n",
    "        return model\n",
    "\n",
    "class Decoder:\n",
    "    l2p = 0.0001\n",
    "    @staticmethod\n",
    "    def layers(y, height, width, fm, act_func, hid_act_func):\n",
    "        \n",
    "        x = Conv2DTranspose(64, fm, strides = (1, 1), activation=hid_act_func,kernel_regularizer=regularizers.l2(Decoder.l2p), padding='same')(y)\n",
    "        x = BatchNormalization()(x)\n",
    "        x = Conv2DTranspose(64, fm,  strides = (2, 1), activation=hid_act_func,kernel_regularizer=regularizers.l2(Decoder.l2p), padding='same')(x)\n",
    "        x = BatchNormalization()(x)\n",
    "        x = Conv2DTranspose(64, fm, strides = (2, 1), activation=hid_act_func,kernel_regularizer=regularizers.l2(Decoder.l2p), padding='same')(x)\n",
    "        x = BatchNormalization()(x)\n",
    "        \n",
    "        xh = Conv2D(1, fm, activation=act_func, padding='same')(x)\n",
    "        return xh\n",
    "   \n",
    "    @staticmethod\n",
    "    def build(height, width, fm , act_func, hid_act_func):\n",
    "        inp = Input(shape=(height, width,1))\n",
    "        dec  = Decoder.layers(inp,height, width, fm, act_func=act_func, hid_act_func=hid_act_func)\n",
    "        model = Model(inputs=inp, outputs=dec ,name=\"Decoder\")\n",
    "        return model        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 345
    },
    "colab_type": "code",
    "id": "e9LKF6Q6hZkW",
    "outputId": "d1169282-5371-4289-d534-af81e19a002b"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"anon\"\n",
      "__________________________________________________________________________________________________\n",
      "Layer (type)                    Output Shape         Param #     Connected to                     \n",
      "==================================================================================================\n",
      "input_5 (InputLayer)            [(None, 128, 9, 1)]  0                                            \n",
      "__________________________________________________________________________________________________\n",
      "Encoder (Model)                 (None, 32, 9, 1)     375365      input_5[0][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "Decoder (Model)                 (None, 128, 9, 1)    375361      Encoder[1][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "GenReg (Model)                  (None, 1)            337665      Decoder[1][0]                    \n",
      "__________________________________________________________________________________________________\n",
      "ActReg (Model)                  (None, 4)            337764      Decoder[1][0]                    \n",
      "==================================================================================================\n",
      "Total params: 1,426,155\n",
      "Trainable params: 749,956\n",
      "Non-trainable params: 676,199\n",
      "__________________________________________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "import keras.backend as K\n",
    "def gen_equ_loss_func(y_true, y_pred):\n",
    "    loss = K.mean(K.abs(0.5 - y_pred))\n",
    "    return loss\n",
    "\n",
    "def build_AAE(loss_weights):\n",
    "    id_class_numbers = 1\n",
    "    act_class_numbers = 4\n",
    "    #fm = (2,3)\n",
    "    #reps_id = Enc_Reg.build(height, width//4, id_class_numbers, name =\"EncReg\", fm=fm, act_func=\"sigmoid\",hid_act_func=\"relu\")\n",
    "    fm = (5,9)\n",
    "    rcon_id = Dec_Reg.build(height, width, id_class_numbers, name =\"GenReg\", fm=fm, act_func=\"sigmoid\",hid_act_func=\"relu\")\n",
    "    # print(rcon_id.summary())\n",
    "    rcon_task = Dec_Reg.build(height, width, act_class_numbers, name =\"ActReg\", fm=fm, act_func=\"softmax\",hid_act_func=\"relu\")\n",
    "    # print(rcon_task.summary())\n",
    "    #reps_id.compile( loss=\"binary_crossentropy\", optimizer='adam', metrics=['acc'])\n",
    "    rcon_id.compile( loss=\"binary_crossentropy\", optimizer='adam', metrics=['acc'])\n",
    "    rcon_task.compile( loss=\"categorical_crossentropy\", optimizer='adam', metrics=['acc'])\n",
    "\n",
    "    #reps_id.trainable = False\n",
    "    rcon_id.trainable = False\n",
    "    rcon_task.trainable = False\n",
    "\n",
    "    enc_to_reps = Encoder.build(height, width, fm=fm, act_func=\"linear\", hid_act_func=\"relu\")\n",
    "    # print(enc_to_reps.summary())\n",
    "    reps_to_dec = Decoder.build(height//4, width, fm=fm, act_func=\"linear\", hid_act_func=\"relu\")\n",
    "    # print(reps_to_dec.summary())\n",
    "    enc_to_reps.compile( loss=\"mean_squared_error\", optimizer='adam', metrics=['mse'])\n",
    "    reps_to_dec.compile( loss=\"mean_squared_error\", optimizer='adam', metrics=['mse'])\n",
    "\n",
    "    x = Input(shape=(height, width,1))\n",
    "    z = enc_to_reps(x)\n",
    "    #idz = reps_id(z)\n",
    "    xh = reps_to_dec(z)\n",
    "    idxh = rcon_id(xh)\n",
    "    txh = rcon_task(xh)\n",
    "\n",
    "\n",
    "\n",
    "    anon_model = Model(inputs = x,\n",
    "                       outputs = [xh,\n",
    "                                  #idz,\n",
    "                                  idxh,\n",
    "                                  txh\n",
    "                                 ],\n",
    "                       name =\"anon\") \n",
    "    anon_model.compile(loss = [\"mean_squared_error\",\n",
    "                               #\"binary_crossentropy\",\n",
    "                                gen_equ_loss_func,\n",
    "                               \"categorical_crossentropy\"\n",
    "                              ],\n",
    "                       loss_weights = loss_weights,                 \n",
    "                       optimizer = \"adam\",\n",
    "                       metrics = [\"acc\"])\n",
    "    #enc_to_reps.set_weights(enc_dec_tmp.layers[1].get_weights()) \n",
    "    #reps_to_dec.set_weights(enc_dec_tmp.layers[2].get_weights()) \n",
    "\n",
    "\n",
    "    return anon_model, rcon_task, rcon_id\n",
    "\n",
    "from tensorflow.keras.layers import *\n",
    "from tensorflow.keras import Model\n",
    "from tensorflow.keras import regularizers\n",
    "height = w_data_train_rep.shape[1]\n",
    "width = w_data_train_rep.shape[2]\n",
    "fm = (5,9)\n",
    "loss_weights=[2, 1, 4]            \n",
    "anon_model, rcon_task, rcon_id = build_AAE(loss_weights)\n",
    "anon_model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "fkTCMCbm6DEu"
   },
   "source": [
    "Here we can see the results of anonymization by AAE (hiding the gender in this case) for three different settings of trade-off parameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 259
    },
    "colab_type": "code",
    "id": "wer1NoUHv5oX",
    "outputId": "1360c7be-9cb0-4d96-80de-d5cdb007c8b5"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "38617/38617 [==============================] - 11s 295us/sample\n",
      "38617/38617 [==============================] - 6s 146us/sample\n",
      "['STR', 'WAL', 'JOG', 'JUM', 'STD', 'FALL']\n",
      "[[98.2  0.9  0.   0.   0.9  0. ]\n",
      " [ 1.8 96.7  0.   0.   1.2  0.3]\n",
      " [ 3.3  0.1 92.1  4.5  0.   0. ]\n",
      " [ 4.3  0.   4.3 91.4  0.   0. ]\n",
      " [ 0.5  1.   0.   0.  95.8  2.6]\n",
      " [ 0.7  0.   0.   0.  95.9  3.4]]\n",
      "['F', 'M']\n",
      "[[63.5 36.5]\n",
      " [13.8 86.2]]\n",
      "f1:  74.9\n",
      "acc:  79.96\n"
     ]
    }
   ],
   "source": [
    "anon_model.load_weights('gender_anon_1.h5')\n",
    "rep_anon_test = anon_model.predict(rep_x_test, verbose = 1)[0]\n",
    "\n",
    "\n",
    "eval_act = keras.models.load_model('_best_FCN_.hdf5')\n",
    "preds = eval_act.predict(rep_anon_test, verbose=1)\n",
    "preds = preds.argmax(axis=1)\n",
    "cm_analysis(y_test[:,0], preds, act_list, figsize=(16,16))\n",
    "\n",
    "\n",
    "eval_gen = keras.models.load_model('_best_gen_FCN_.hdf5')\n",
    "preds = eval_gen.predict(rep_anon_test)\n",
    "preds = (preds > 0.5).astype(int)[:,0]\n",
    "cm_analysis(y_test[:,1], preds, ['F','M'], figsize=(16,16))\n",
    "print(\"f1: \", np.round(f1_score(y_test[:,1], preds, average='macro')*100,2))\n",
    "print(\"acc: \", np.round(eval_gen.evaluate(rep_anon_test, y_test[:,1], verbose=0)[1]*100,2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 259
    },
    "colab_type": "code",
    "id": "DRiCBsaN5w64",
    "outputId": "84206b43-742c-4c51-958d-d92a269454bb"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "38617/38617 [==============================] - 11s 283us/sample\n",
      "38617/38617 [==============================] - 6s 150us/sample\n",
      "['STR', 'WAL', 'JOG', 'JUM', 'STD', 'FALL']\n",
      "[[99.3  0.4  0.   0.   0.4  0. ]\n",
      " [ 4.5 94.1  0.   0.   1.2  0.3]\n",
      " [ 4.   0.1 92.1  3.8  0.   0. ]\n",
      " [ 5.5  0.   4.8 89.6  0.   0. ]\n",
      " [ 1.3  0.9  0.   0.  96.3  1.5]\n",
      " [ 0.8  0.   0.   0.  95.7  3.4]]\n",
      "['F', 'M']\n",
      "[[71.7 28.3]\n",
      " [29.6 70.4]]\n",
      "f1:  67.59\n",
      "acc:  70.75\n"
     ]
    }
   ],
   "source": [
    "anon_model.load_weights('gender_anon_2.h5')\n",
    "\n",
    "rep_anon_test = anon_model.predict(rep_x_test, verbose = 1)[0]\n",
    "\n",
    "eval_act = keras.models.load_model('_best_FCN_.hdf5')\n",
    "preds = eval_act.predict(rep_anon_test, verbose=1)\n",
    "preds = preds.argmax(axis=1)\n",
    "cm_analysis(y_test[:,0], preds, act_list, figsize=(16,16))\n",
    "\n",
    "\n",
    "eval_gen = keras.models.load_model('_best_gen_FCN_.hdf5')\n",
    "preds = eval_gen.predict(rep_anon_test)\n",
    "preds = (preds > 0.5).astype(int)[:,0]\n",
    "cm_analysis(y_test[:,1], preds, ['F','M'], figsize=(16,16))\n",
    "print(\"f1: \", np.round(f1_score(y_test[:,1], preds, average='macro')*100,2))\n",
    "print(\"acc: \", np.round(eval_gen.evaluate(rep_anon_test, y_test[:,1], verbose=0)[1]*100,2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 259
    },
    "colab_type": "code",
    "id": "r_Qnbhdm1mCP",
    "outputId": "6af69928-79f4-416b-9bf9-71dc82a53a9f"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "38617/38617 [==============================] - 11s 284us/sample\n",
      "38617/38617 [==============================] - 6s 157us/sample\n",
      "['STR', 'WAL', 'JOG', 'JUM', 'STD', 'FALL']\n",
      "[[98.6  1.   0.   0.   0.4  0. ]\n",
      " [ 4.5 94.   0.   0.   1.   0.5]\n",
      " [ 2.5  0.1 93.3  4.   0.   0. ]\n",
      " [ 5.3  0.   5.2 89.6  0.   0. ]\n",
      " [ 4.9  1.   0.   0.  92.7  1.4]\n",
      " [ 0.7  0.   0.   0.  94.9  4.3]]\n",
      "['F', 'M']\n",
      "[[71.9 28.1]\n",
      " [35.2 64.8]]\n",
      "f1:  64.13\n",
      "acc:  66.77\n"
     ]
    }
   ],
   "source": [
    "anon_model.load_weights('gender_anon_3.h5')\n",
    "rep_anon_test = anon_model.predict(rep_x_test, verbose = 1)[0]\n",
    "\n",
    "\n",
    "eval_act = keras.models.load_model('_best_FCN_.hdf5')\n",
    "preds = eval_act.predict(rep_anon_test, verbose=1)\n",
    "preds = preds.argmax(axis=1)\n",
    "cm_analysis(y_test[:,0], preds, act_list, figsize=(16,16))\n",
    "\n",
    "\n",
    "eval_gen = keras.models.load_model('_best_gen_FCN_.hdf5')\n",
    "preds = eval_gen.predict(rep_anon_test)\n",
    "preds = (preds > 0.5).astype(int)[:,0]\n",
    "cm_analysis(y_test[:,1], preds, ['F','M'], figsize=(16,16))\n",
    "print(\"f1: \", np.round(f1_score(y_test[:,1], preds, average='macro')*100,2))\n",
    "print(\"acc: \", np.round(eval_gen.evaluate(rep_anon_test, y_test[:,1], verbose=0)[1]*100,2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Qmrt1Yej69nU"
   },
   "source": [
    "Finally, we see that after using the AAE we suffer a bit more accuracy loss. However, inferring the gender is so close to the random guess on this dataset."
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "pmc_additional_experiment.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}