clue/classification/iflytek/predict.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# IFLYTEK"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "长文本分类\n",
    "\n",
    "该数据集共有1.7万多条关于app应用描述的长文本标注数据，包含和日常生活相关的各类应用主题，共119个类别：\"打车\":0,\"地图导航\":1,\"免费WIFI\":2,\"租车\":3,….,\"女性\":115,\"经营\":116,\"收款\":117,\"其他\":118(分别用0-118表示)。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 预测"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import json"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pyclue.tf1.tasks.classification.multi_class.predict import Predictor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 加载保存好的模型路径（train.ipynb中最后打印出的model_file_dict中的'pb_model_file'地址）\n",
    "pb_model_file = ''\n",
    "\n",
    "# 测试数据路径（应包含test.txt文件和id_2_label.json文件）\n",
    "data_dir = '/workspace/projects/PyCLUE_Corpus/classification/iflytek'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def submit_iflytek(predictor, data_dir, submit_dir):\n",
    "    test_data_file = os.path.join(data_dir, 'test.txt')\n",
    "    submit_results = []\n",
    "    ids = [item[0] for item in predictor.processor.read_file(file_path=test_data_file)]\n",
    "    labels = [item['prediction'] for item in predictor.predict_from_file(input_file=test_data_file)]\n",
    "    for index, label in zip(ids, labels):\n",
    "        submit_results.append('{\"id\": \"%s\", \"label\": \"%s\"}\\n' % (index, label))\n",
    "    save_path = os.path.join(submit_dir, 'iflytek_predict.json')\n",
    "    with open(save_path, 'w') as f:\n",
    "        f.writelines(submit_results)\n",
    "    return save_path"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 初始化预测器\n",
    "predictor = Predictor(\n",
    "    model_file=pb_model_file)\n",
    "# 生成提交文件\n",
    "save_path = submit_iflytek(\n",
    "    predictor=predictor, data_dir=data_dir, submit_dir=pb_model_file)\n",
    "print(save_path)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}