docs/tutorials/freq_cat_onln_gfmm_general_use.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Batch-Incremental Learning Algorithm for GFMM using Probability-based Measures for Categorical Features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This example shows how to use the general fuzzy min-max neural network trained by the batch-incremental learning algorithm, in which categorical features are encoded using the ordinal encoding method and the similarity values among categorical feature are computed using frequency of categorical values.\n",
    "\n",
    "Note that the numerical features in training and testing datasets must be in the range of [0, 1] because the GFMM classifiers require features in the unit cube. Therefore, continuous features need to be normalised before training. For categorical features, nothing needs to be done as this FreqCatOnlineGFMM classifier will apply the appropriate encoding method for the categorical values."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Execute directly from the python file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "from sklearn.metrics import accuracy_score"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Get the path to the this jupyter notebook file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'C:\\\\hyperbox-brain\\\\examples\\\\mixed_data'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "this_notebook_dir = os.path.dirname(os.path.abspath(\"__file__\"))\n",
    "this_notebook_dir"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Get the home folder of the Hyperbox-Brain project"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "WindowsPath('C:/hyperbox-brain')"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from pathlib import Path\n",
    "project_dir = Path(this_notebook_dir).parent.parent\n",
    "project_dir"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Create the path to the Python file containing the implementation of the GFMM classifier using the online learning algorithm with the cateogical feature similarity measure based on the frequence of occurence of categorical values for mixed attribute features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'C:\\\\hyperbox-brain\\\\hbbrain\\\\mixed_data\\\\freq_cat_onln_gfmm.py'"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "freq_cat_gfmm_file_path = os.path.join(project_dir, Path(\"hbbrain/mixed_data/freq_cat_onln_gfmm.py\"))\n",
    "freq_cat_gfmm_file_path"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Run the found file by showing the execution directions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "usage: freq_cat_onln_gfmm.py [-h] -training_file TRAINING_FILE -testing_file\n",
      "                             TESTING_FILE -categorical_features\n",
      "                             CATEGORICAL_FEATURES [--theta THETA]\n",
      "                             [--theta_min THETA_MIN] [--eta ETA]\n",
      "                             [--gamma GAMMA] [--alpha ALPHA]\n",
      "\n",
      "The description of parameters\n",
      "\n",
      "required arguments:\n",
      "  -training_file TRAINING_FILE\n",
      "                        A required argument for the path to training data file\n",
      "                        (including file name)\n",
      "  -testing_file TESTING_FILE\n",
      "                        A required argument for the path to testing data file\n",
      "                        (including file name)\n",
      "  -categorical_features CATEGORICAL_FEATURES\n",
      "                        Indices of categorical features\n",
      "\n",
      "optional arguments:\n",
      "  --theta THETA         Maximum hyperbox size (in the range of (0, 1])\n",
      "                        (default: 0.5)\n",
      "  --theta_min THETA_MIN\n",
      "                        Mimimum value of the maximum hyperbox size to escape\n",
      "                        the training loop (in the range of (0, 1]) (default:\n",
      "                        0.5)\n",
      "  --eta ETA             Maximum similarity value for each pair of categorical\n",
      "                        values (in the range of (0, 1] (default: 0.5\n",
      "  --gamma GAMMA         A sensitivity parameter describing the speed of\n",
      "                        decreasing of the membership function in each\n",
      "                        continuous dimension (larger than 0) (default: 1)\n",
      "  --alpha ALPHA         Multiplier showing the decrease of theta in each step\n",
      "                        (default: 0.9)\n"
     ]
    }
   ],
   "source": [
    "!python \"{freq_cat_gfmm_file_path}\" -h"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Create the path to mixed-attribute training and testing datasets stored in the dataset folder.\n",
    "This example uses the japanese_credit dataset for illustration purposes. The continuous features in this dataset were normalised into the range of [0, 1], while categorical features were kept unchanged."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'C:\\\\hyperbox-brain\\\\dataset\\\\japanese_credit_train.csv'"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "training_data_file = os.path.join(project_dir, Path(\"dataset/japanese_credit_train.csv\"))\n",
    "training_data_file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'C:\\\\hyperbox-brain\\\\dataset\\\\japanese_credit_test.csv'"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "testing_data_file = os.path.join(project_dir, Path(\"dataset/japanese_credit_test.csv\"))\n",
    "testing_data_file"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Run a demo program"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of hyperboxes = 266\n",
      "Testing accuracy =  80.92%\n"
     ]
    }
   ],
   "source": [
    "!python \"{freq_cat_gfmm_file_path}\" -training_file \"{training_data_file}\" -testing_file \"{testing_data_file}\" -categorical_features \"[0, 3, 4, 5, 6, 8, 9, 11,12]\" --theta 0.1 --theta_min 0.1 --eta 0.6 --gamma 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Using the FreqCatOnlineGFMM algorithm to train a GFMM classifier for mixed-attribute data through its init, fit, and predict functions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "from hbbrain.mixed_data.freq_cat_onln_gfmm import FreqCatOnlineGFMM\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Create mixed attribute training, validation, and testing data sets.\n",
    "This example will use the japanese_credit dataset for illustration purposes. The continuous features in this dataset were normalised into the range of [0, 1], while categorical features were kept unchanged."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_train = pd.read_csv(training_data_file, header=None)\n",
    "df_test = pd.read_csv(testing_data_file, header=None)\n",
    "\n",
    "Xy_train = df_train.to_numpy()\n",
    "Xy_test = df_test.to_numpy()\n",
    "\n",
    "Xtr = Xy_train[:, :-1]\n",
    "ytr = Xy_train[:, -1].astype(int)\n",
    "\n",
    "Xtest = Xy_test[:, :-1]\n",
    "ytest = Xy_test[:, -1].astype(int)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "val_data_file = os.path.join(project_dir, Path(\"dataset/japanese_credit_val.csv\"))\n",
    "df_val = pd.read_csv(val_data_file, header=None)\n",
    "Xy_val = df_val.to_numpy()\n",
    "Xval = Xy_val[:, :-1]\n",
    "yval = Xy_val[:, -1].astype(int)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Initializing parameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "theta = 0.1 # maximum hyperbox size for continuous features\n",
    "theta_min = 0.1 # Only performing one training loop\n",
    "eta = 0.6 # Maximum similarity value for each pair of categorical values\n",
    "gamma = 1 # speed of decreasing degree in the membership values of continuous features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Indicate the indices of categorical features in the training data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "categorical_features = [0, 3, 4, 5, 6, 8, 9, 11, 12]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "FreqCatOnlineGFMM(C=array([0, 0, 1, 1, 1, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 1, 0, 1, 0, 1, 1, 0,\n",
       "       1, 0, 0, 0, 0, 1, 0, 1, 1, 0, 1, 1, 0, 1, 1, 0, 1, 1, 1, 1, 1, 0,\n",
       "       1, 0, 0, 1, 1, 1, 0, 1, 0, 0, 1, 1, 0, 0, 0, 1, 0, 1, 0, 1, 1, 1,\n",
       "       0, 1, 1, 1, 1, 1, 0, 1, 1, 0, 0, 0, 0, 1, 1, 1, 0, 0, 0, 1, 0, 0,\n",
       "       0, 0, 1, 0, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1,\n",
       "       0, 0, 0, 1, 0, 0, 1, 0, 0, 1, 1, 0, 0, 1, 1, 1, 0, 1, 0, 0, 1, 0,\n",
       "       0, 0, 1, 0, 0, 1, 1, 1, 0, 0, 0, 1, 1, 1, 0, 0, 1, 0, 1, 0, 0, 1,\n",
       "       0, 0, 1, 1, 1, 0, 1, 0...\n",
       "       [1.32222222e-01, 3.92857143e-01, 5.26315789e-02, 0.00000000e+00,\n",
       "        6.00000000e-02, 2.20600000e-02],\n",
       "       ...,\n",
       "       [4.35238095e-01, 2.32142857e-01, 1.75438596e-02, 4.47761194e-02,\n",
       "        7.25000000e-02, 0.00000000e+00],\n",
       "       [1.29682540e-01, 1.78571429e-02, 4.38596491e-03, 0.00000000e+00,\n",
       "        1.80000000e-01, 0.00000000e+00],\n",
       "       [5.38412698e-01, 1.03571429e-02, 5.26315789e-01, 2.98507463e-01,\n",
       "        0.00000000e+00, 1.50000000e-01]]),\n",
       "                  eta=0.6, theta=0.1, theta_min=0.1)"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "freq_cat_onln_gfmm_clf = FreqCatOnlineGFMM(theta=theta, theta_min=theta_min, eta=eta, gamma=gamma)\n",
    "freq_cat_onln_gfmm_clf.fit(Xtr, ytr, categorical_features)   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of hyperboxes = 266\n"
     ]
    }
   ],
   "source": [
    " print('Number of hyperboxes = %d'%freq_cat_onln_gfmm_clf.get_n_hyperboxes())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training time: 1.256 (s)\n"
     ]
    }
   ],
   "source": [
    "print(\"Training time: %.3f (s)\"%freq_cat_onln_gfmm_clf.elapsed_training_time)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Prediction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy =  80.92%\n"
     ]
    }
   ],
   "source": [
    "y_pred = freq_cat_onln_gfmm_clf.predict(Xtest)\n",
    "acc = accuracy_score(ytest, y_pred)\n",
    "print(f'Accuracy = {acc * 100: .2f}%')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Explaining the predicted result for the input sample by showing membership values and hyperboxes for each class"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Explain samples:\n",
      "Membership values for classes:  {0: 0.6642512077294687, 1: 0.75}\n",
      "Predicted class =  1\n",
      "Minimum continuous points of the selected hyperbox for each class:  {0: array([0.1852381 , 0.04017857, 0.04526316, 0.02985075, 0.1       ,\n",
      "       0.        ]), 1: array([0.03301587, 0.02089286, 0.00578947, 0.02985075, 0.05      ,\n",
      "       0.        ])}\n",
      "Maximum continuous points of the selected hyperbox for each class:  {0: array([0.1852381 , 0.04017857, 0.04526316, 0.02985075, 0.1       ,\n",
      "       0.        ]), 1: array([0.10984127, 0.10714286, 0.07315789, 0.07462687, 0.11      ,\n",
      "       0.02503   ])}\n",
      "Minimum categorical points of the selected hyperbox for each class:  {0: array([0.0, 1.0, 0.0, 10.0, 7.0, 1.0, 1.0, 0.0, 0.0], dtype=object), 1: array([0.0, 1.0, 0.0, 7.0, 7.0, 1.0, 1.0, 0.0, 0.0], dtype=object)}\n",
      "Maximum categorical points of the selected hyperbox for each class:  {0: array([100000, 100000, 100000, 100000, 100000, 100000, 100000, 100000,\n",
      "       100000]), 1: array([0, 1, 0, 1, 3, 1, 1, 0, 0])}\n"
     ]
    }
   ],
   "source": [
    "sample_need_explain = 1\n",
    "y_pred_input_0, mem_val_classes, min_points_classes, max_points_classes, dict_min_point_cat_classes, dict_max_point_cat_classes = freq_cat_onln_gfmm_clf.get_sample_explanation(Xtest[sample_need_explain])\n",
    "print(\"Explain samples:\")\n",
    "print(\"Membership values for classes: \", mem_val_classes)\n",
    "print(\"Predicted class = \", y_pred_input_0)\n",
    "print(\"Minimum continuous points of the selected hyperbox for each class: \", min_points_classes)\n",
    "print(\"Maximum continuous points of the selected hyperbox for each class: \", max_points_classes)\n",
    "print(\"Minimum categorical points of the selected hyperbox for each class: \", dict_min_point_cat_classes)\n",
    "print(\"Maximum categorical points of the selected hyperbox for each class: \", dict_max_point_cat_classes)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Apply pruning for the trained classifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "FreqCatOnlineGFMM(C=array([0, 1, 1, 1, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 1, 0, 1, 0, 1, 1, 0, 1,\n",
       "       0, 0, 0, 1, 0, 1, 0, 1, 1, 0, 1, 1, 0, 1, 1, 1, 1, 1, 0, 1, 0, 0,\n",
       "       1, 1, 0, 1, 0, 0, 1, 1, 0, 0, 0, 1, 0, 1, 0, 1, 1, 0, 1, 1, 1, 1,\n",
       "       0, 1, 1, 0, 0, 0, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 1, 1, 1, 0,\n",
       "       1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 0, 1, 0,\n",
       "       0, 1, 1, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 1, 1, 1, 0, 0, 0,\n",
       "       1, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 1, 1, 0, 1, 0, 0, 1, 0, 1,\n",
       "       0, 0, 1, 1, 1, 1, 0, 1...\n",
       "       [2.65873016e-01, 2.32142857e-01, 1.40350877e-01, 1.04477612e-01,\n",
       "        4.95000000e-02, 3.06500000e-02],\n",
       "       ...,\n",
       "       [4.35238095e-01, 2.32142857e-01, 1.75438596e-02, 4.47761194e-02,\n",
       "        7.25000000e-02, 0.00000000e+00],\n",
       "       [1.29682540e-01, 1.78571429e-02, 4.38596491e-03, 0.00000000e+00,\n",
       "        1.80000000e-01, 0.00000000e+00],\n",
       "       [5.38412698e-01, 1.03571429e-02, 5.26315789e-01, 2.98507463e-01,\n",
       "        0.00000000e+00, 1.50000000e-01]]),\n",
       "                  eta=0.6, theta=0.1, theta_min=0.1)"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "acc_threshold = 0.5 # minimum accuracy of hyperboxes being retained\n",
    "keep_empty_boxes = False # do not keep the hyperboxes which do not join the prediction process on the validation set\n",
    "freq_cat_onln_gfmm_clf.simple_pruning(Xval, yval, acc_threshold, keep_empty_boxes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of hyperboxes after pruning = 246\n"
     ]
    }
   ],
   "source": [
    "print('Number of hyperboxes after pruning = %d'%freq_cat_onln_gfmm_clf.get_n_hyperboxes())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Make prediction after pruning"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy after pruning =  83.21%\n"
     ]
    }
   ],
   "source": [
    "y_pred = freq_cat_onln_gfmm_clf.predict(Xtest)\n",
    "acc = accuracy_score(ytest, y_pred)\n",
    "print(f'Accuracy after pruning = {acc * 100: .2f}%')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}