week5/5-2 特征列feature_column.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 5-1 特征列feature_column\n",
    "特征列一般用于结构化数据建模使用，图像和文本数据一般不会用到特征列"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 一 特征列用法概述"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "使用特征列可以将类别特征转换为one-hot编码特征，将连续特征构建分桶特征，以及对多个特征生成交叉特征等等。\n",
    "\n",
    "\n",
    "要创建特征列，请调用 tf.feature_column 模块的函数。该模块中常用的九个函数如下图所示，所有九个函数都会返回一个 Categorical-Column 或一个 \n",
    "Dense-Column 对象，但却不会返回 bucketized_column，后者继承自这两个类。\n",
    "\n",
    "注意：所有的Catogorical Column类型最终都要通过indicator_column转换成Dense Column类型才能传入模型！"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* numeric_column 数值列，最常用。\n",
    "\n",
    "\n",
    "* bucketized_column 分桶列，由数值列生成，可以由一个数值列出多个特征，one-hot编码。\n",
    "\n",
    "\n",
    "* categorical_column_with_identity 分类标识列，one-hot编码，相当于分桶列每个桶为1个整数的情况。\n",
    "\n",
    "\n",
    "* categorical_column_with_vocabulary_list 分类词汇列，one-hot编码，由list指定词典。\n",
    "\n",
    "\n",
    "* categorical_column_with_vocabulary_file 分类词汇列，由文件file指定词典。\n",
    "\n",
    "\n",
    "* categorical_column_with_hash_bucket 哈希列，整数或词典较大时采用。\n",
    "\n",
    "\n",
    "* indicator_column 指标列，由Categorical Column生成，one-hot编码\n",
    "\n",
    "\n",
    "* embedding_column 嵌入列，由Categorical Column生成，嵌入矢量分布参数需要学习。嵌入矢量维数建议取类别数量的 4 次方根。\n",
    "\n",
    "\n",
    "* crossed_column 交叉列，可以由除categorical_column_with_hash_bucket的任意分类列构成。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 二 特征列使用规范\n",
    "以下是一个使用特征列解决Titanic生存问题的范例"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "import pandas as pd\n",
    "from tensorflow.keras import layers, models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Name</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Ticket</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Cabin</th>\n",
       "      <th>Embarked</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>493</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>Molson, Mr. Harry Markland</td>\n",
       "      <td>male</td>\n",
       "      <td>55.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>113787</td>\n",
       "      <td>30.5000</td>\n",
       "      <td>C30</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>53</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>Harper, Mrs. Henry Sleeper (Myna Haxtun)</td>\n",
       "      <td>female</td>\n",
       "      <td>49.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>PC 17572</td>\n",
       "      <td>76.7292</td>\n",
       "      <td>D33</td>\n",
       "      <td>C</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>388</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>Buss, Miss. Kate</td>\n",
       "      <td>female</td>\n",
       "      <td>36.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>27849</td>\n",
       "      <td>13.0000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>192</td>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "      <td>Carbines, Mr. William</td>\n",
       "      <td>male</td>\n",
       "      <td>19.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>28424</td>\n",
       "      <td>13.0000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>687</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>Panula, Mr. Jaako Arnold</td>\n",
       "      <td>male</td>\n",
       "      <td>14.0</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>3101295</td>\n",
       "      <td>39.6875</td>\n",
       "      <td>NaN</td>\n",
       "      <td>S</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   PassengerId  Survived  Pclass                                      Name  \\\n",
       "0          493         0       1                Molson, Mr. Harry Markland   \n",
       "1           53         1       1  Harper, Mrs. Henry Sleeper (Myna Haxtun)   \n",
       "2          388         1       2                          Buss, Miss. Kate   \n",
       "3          192         0       2                     Carbines, Mr. William   \n",
       "4          687         0       3                  Panula, Mr. Jaako Arnold   \n",
       "\n",
       "      Sex   Age  SibSp  Parch    Ticket     Fare Cabin Embarked  \n",
       "0    male  55.0      0      0    113787  30.5000   C30        S  \n",
       "1  female  49.0      1      0  PC 17572  76.7292   D33        C  \n",
       "2  female  36.0      0      0     27849  13.0000   NaN        S  \n",
       "3    male  19.0      0      0     28424  13.0000   NaN        S  \n",
       "4    male  14.0      4      1   3101295  39.6875   NaN        S  "
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 构建数据管道\n",
    "dftrain_raw = pd.read_csv('../data/titanic/train.csv')\n",
    "dftest_raw = pd.read_csv('../data/titanic/test.csv')\n",
    "dfraw = pd.concat([dftrain_raw, dftest_raw])\n",
    "dfraw.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "PassengerId      int64\n",
      "Survived         int64\n",
      "Pclass           int64\n",
      "Name            object\n",
      "Sex             object\n",
      "Age            float64\n",
      "SibSp            int64\n",
      "Parch            int64\n",
      "Ticket          object\n",
      "Fare           float64\n",
      "Cabin           object\n",
      "Embarked        object\n",
      "dtype: object\n"
     ]
    }
   ],
   "source": [
    "print(dfraw.dtypes)  # <class 'pandas.core.series.Series'>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [],
   "source": [
    "def prepare_dfdata(dfraw):\n",
    "    dfdata = dfraw.copy()\n",
    "    dfdata.columns = [x.lower() for x in dfraw.columns]\n",
    "    dfdata = dfdata.rename(columns = {'survived':'label'})  # 把survived列变为标签列\n",
    "    dfdata = dfdata.drop(['passengerid', 'name'], axis = 1)  # 这两个与是否生存没有关系\n",
    "    for col, dtype in dict(dfdata.dtypes).items():\n",
    "        if dfdata[col].hasnans:\n",
    "            # 添加标识符包含是否缺失\n",
    "            dfdata[col + '_nan']  = pd.isna(dfdata[col]).astype('int32')\n",
    "            # 填充\n",
    "            if dtype not in [np.object, np.str, np.unicode]:\n",
    "                dfdata[col].fillna(dfdata[col].mean(), inplace = True)\n",
    "            else:\n",
    "                dfdata[col].fillna('', inplace = True)\n",
    "    return dfdata"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [],
   "source": [
    "dfdata = prepare_dfdata(dfraw)\n",
    "dftrain = dfdata.iloc[0:len(dftrain_raw), :]\n",
    "dftest = dfdata.iloc[len(dftrain_raw):, :]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 从dataframe中导入数据\n",
    "def df_to_dataset(df, shuffle = True, batch_size = 32):\n",
    "    dfdata = df.copy()\n",
    "    if 'label' not in dfdata.columns:\n",
    "        ds = tf.data.Dataset.from_tensor_slices(dfdata.to_dict('list'))\n",
    "    else:\n",
    "        labels = dfdata.pop('label')\n",
    "        ds = tf.data.Dataset.from_tensor_slices((dfdata.to_dict('list'), labels))\n",
    "    if shuffle:\n",
    "        ds = ds.shuffle(buffer_size = len(dfdata))\n",
    "    ds = ds.batch(batch_size)\n",
    "    return ds"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [],
   "source": [
    "ds_train = df_to_dataset(dftrain)\n",
    "ds_test = df_to_dataset(dftest)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "=================\n",
      "{'pclass': <tf.Tensor: shape=(), dtype=int32, numpy=1>, 'sex': <tf.Tensor: shape=(), dtype=string, numpy=b'female'>, 'age': <tf.Tensor: shape=(), dtype=float32, numpy=31.0>, 'sibsp': <tf.Tensor: shape=(), dtype=int32, numpy=0>, 'parch': <tf.Tensor: shape=(), dtype=int32, numpy=2>, 'ticket': <tf.Tensor: shape=(), dtype=string, numpy=b'36928'>, 'fare': <tf.Tensor: shape=(), dtype=float32, numpy=164.8667>, 'cabin': <tf.Tensor: shape=(), dtype=string, numpy=b'C7'>, 'embarked': <tf.Tensor: shape=(), dtype=string, numpy=b'S'>, 'age_nan': <tf.Tensor: shape=(), dtype=int32, numpy=0>, 'cabin_nan': <tf.Tensor: shape=(), dtype=int32, numpy=0>, 'embarked_nan': <tf.Tensor: shape=(), dtype=int32, numpy=0>}\n",
      "------------\n",
      "tf.Tensor(1, shape=(), dtype=int64)\n",
      "=================\n",
      "{'pclass': <tf.Tensor: shape=(), dtype=int32, numpy=3>, 'sex': <tf.Tensor: shape=(), dtype=string, numpy=b'male'>, 'age': <tf.Tensor: shape=(), dtype=float32, numpy=29.699118>, 'sibsp': <tf.Tensor: shape=(), dtype=int32, numpy=0>, 'parch': <tf.Tensor: shape=(), dtype=int32, numpy=0>, 'ticket': <tf.Tensor: shape=(), dtype=string, numpy=b'A/5 2817'>, 'fare': <tf.Tensor: shape=(), dtype=float32, numpy=8.05>, 'cabin': <tf.Tensor: shape=(), dtype=string, numpy=b''>, 'embarked': <tf.Tensor: shape=(), dtype=string, numpy=b'S'>, 'age_nan': <tf.Tensor: shape=(), dtype=int32, numpy=1>, 'cabin_nan': <tf.Tensor: shape=(), dtype=int32, numpy=1>, 'embarked_nan': <tf.Tensor: shape=(), dtype=int32, numpy=0>}\n",
      "------------\n",
      "tf.Tensor(0, shape=(), dtype=int64)\n"
     ]
    }
   ],
   "source": [
    "for features, label in ds_train.unbatch().take(2):\n",
    "    print(\"=================\")\n",
    "    print(features)\n",
    "    print('------------')\n",
    "    print(label)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1, 2, 3, 4]"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[1, 2] + [3, 4]  # expand"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NumericColumn(key='embarked_nan', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None)\n",
      "BucketizedColumn(source_column=NumericColumn(key='age', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None), boundaries=(18, 25, 30, 35, 40, 50, 55, 60, 65))\n"
     ]
    }
   ],
   "source": [
    "# 定义特征列\n",
    "feature_columns = []\n",
    "# 数值列\n",
    "for col in ['age', 'fare', 'parch', 'sibsp'] + [c for c in dfdata.columns if c.endswith('_nan')]:\n",
    "    feature_columns.append(tf.feature_column.numeric_column(col))\n",
    "\n",
    "print(tf.feature_column.numeric_column(col))\n",
    "# 分桶列\n",
    "age  = tf.feature_column.numeric_column('age')\n",
    "age_buckets = tf.feature_column.bucketized_column(age, boundaries = [18, 25, 30, 35, 40, 50, 55, 60, 65])\n",
    "feature_columns.append(age_buckets)\n",
    "print(age_buckets)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "IndicatorColumn(categorical_column=VocabularyListCategoricalColumn(key='sex', vocabulary_list=('male', 'female'), dtype=tf.string, default_value=-1, num_oov_buckets=0))\n"
     ]
    }
   ],
   "source": [
    "# 类别列 \n",
    "# 注意: 所有的Categorical Column类型最终都要通过indicator_column转换为Dense Column才能传入模型\n",
    "sex = tf.feature_column.indicator_column(\n",
    "        tf.feature_column.categorical_column_with_vocabulary_list(\n",
    "          key = 'sex', vocabulary_list = ['male', 'female']\n",
    "        ))\n",
    "feature_columns.append(sex)\n",
    "print(sex)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [],
   "source": [
    "pclass = tf.feature_column.indicator_column(\n",
    "            tf.feature_column.categorical_column_with_vocabulary_list(\n",
    "              key = 'pclass', vocabulary_list = [1, 2, 3]\n",
    "            ))\n",
    "\n",
    "ticket = tf.feature_column.indicator_column(\n",
    "            tf.feature_column.categorical_column_with_hash_bucket('ticket', 3)\n",
    "            )\n",
    "\n",
    "embarked = tf.feature_column.indicator_column(\n",
    "            tf.feature_column.categorical_column_with_vocabulary_list(\n",
    "              key = 'embarked', vocabulary_list = ['S', 'C', 'B']\n",
    "        ))\n",
    "feature_columns.append(pclass)\n",
    "feature_columns.append(ticket)\n",
    "feature_columns.append(embarked)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 嵌入列\n",
    "cabin = tf.feature_column.embedding_column(\n",
    "            tf.feature_column.categorical_column_with_hash_bucket('cabin', 32), 2)\n",
    "feature_columns.append(cabin)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 交叉列\n",
    "pclass_cate = tf.feature_column.categorical_column_with_vocabulary_list(\n",
    "              key = 'pclass', vocabulary_list = [1, 2, 3]\n",
    "            )\n",
    "crossed_feature = tf.feature_column.indicator_column(\n",
    "                    tf.feature_column.crossed_column([age_buckets, pclass_cate], hash_bucket_size = 15))\n",
    "\n",
    "feature_columns.append(crossed_feature)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "14"
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(feature_columns)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n",
      "23/23 [==============================] - 1s 34ms/step - loss: 1.2207 - accuracy: 0.6629 - val_loss: 0.5891 - val_accuracy: 0.6648\n",
      "Epoch 2/10\n",
      "23/23 [==============================] - 0s 3ms/step - loss: 0.6489 - accuracy: 0.7093 - val_loss: 0.5709 - val_accuracy: 0.7263\n",
      "Epoch 3/10\n",
      "23/23 [==============================] - 0s 3ms/step - loss: 0.6951 - accuracy: 0.7079 - val_loss: 0.5195 - val_accuracy: 0.7709\n",
      "Epoch 4/10\n",
      "23/23 [==============================] - 0s 3ms/step - loss: 0.6286 - accuracy: 0.7121 - val_loss: 0.4676 - val_accuracy: 0.7933\n",
      "Epoch 5/10\n",
      "23/23 [==============================] - 0s 3ms/step - loss: 0.5875 - accuracy: 0.7697 - val_loss: 0.6494 - val_accuracy: 0.7039\n",
      "Epoch 6/10\n",
      "23/23 [==============================] - 0s 3ms/step - loss: 0.9001 - accuracy: 0.6854 - val_loss: 0.6609 - val_accuracy: 0.7151\n",
      "Epoch 7/10\n",
      "23/23 [==============================] - 0s 3ms/step - loss: 0.5872 - accuracy: 0.7219 - val_loss: 0.4468 - val_accuracy: 0.8156\n",
      "Epoch 8/10\n",
      "23/23 [==============================] - 0s 3ms/step - loss: 0.7170 - accuracy: 0.7303 - val_loss: 0.5530 - val_accuracy: 0.7430\n",
      "Epoch 9/10\n",
      "23/23 [==============================] - 0s 3ms/step - loss: 0.5792 - accuracy: 0.7669 - val_loss: 0.5186 - val_accuracy: 0.7709\n",
      "Epoch 10/10\n",
      "23/23 [==============================] - 0s 3ms/step - loss: 0.6099 - accuracy: 0.7598 - val_loss: 0.4324 - val_accuracy: 0.8045\n"
     ]
    }
   ],
   "source": [
    "# 定义模型\n",
    "tf.keras.backend.clear_session()\n",
    "model = tf.keras.Sequential([\n",
    "    layers.DenseFeatures(feature_columns),\n",
    "    layers.Dense(64),\n",
    "    layers.Dense(64),\n",
    "    layers.Dense(1, activation = 'sigmoid')\n",
    "])\n",
    "\n",
    "\n",
    "# 训练模型\n",
    "model.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy'])\n",
    "\n",
    "history = model.fit(ds_train, validation_data = ds_test, epochs = 10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "dense_features (DenseFeature multiple                  64        \n",
      "_________________________________________________________________\n",
      "dense (Dense)                multiple                  2944      \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              multiple                  4160      \n",
      "_________________________________________________________________\n",
      "dense_2 (Dense)              multiple                  65        \n",
      "=================================================================\n",
      "Total params: 7,233\n",
      "Trainable params: 7,233\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model.summary()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "tensorflow2",
   "language": "python",
   "name": "tensorflow2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}