clue/sentence_pair/cmnli/predict.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# CMNLI"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "语言推理任务 Chinese Multi-Genre NLI\n",
    "\n",
    "CMNLI数据由两部分组成：XNLI和MNLI。数据来自于fiction，telephone，travel，government，slate等，对原始MNLI数据和XNLI数据进行了中英文转化，保留原始训练集，合并XNLI中的dev和MNLI中的matched作为CMNLI的dev，合并XNLI中的test和MNLI中的mismatched作为CMNLI的test，并打乱顺序。该数据集可用于判断给定的两个句子之间属于蕴涵、中立、矛盾关系。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 预测"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import json"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pyclue.tf1.tasks.sentence_pair.siamese.predict import Predictor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 加载保存好的模型路径（train.ipynb中最后打印出的model_file_dict中的'pb_model_file'地址）\n",
    "pb_model_file = ''\n",
    "\n",
    "# 测试数据路径（应包含test.txt文件）\n",
    "data_dir = '/workspace/projects/PyCLUE_Corpus/sentence_pair/cmnli'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def submit_cmnli(predictor, submit_dir):\n",
    "    test_data_file = os.path.join(data_dir, 'test.txt')\n",
    "    submit_results = []\n",
    "    ids = [item[0] for item in predictor.processor.read_file(file_path=test_data_file)]\n",
    "    labels = [item['prediction'] for item in predictor.predict_from_file(input_file=test_data_file)]\n",
    "    for index, label in zip(ids, labels):\n",
    "        submit_results.append('{\"id\": \"%s\", \"label\": \"%s\"}\\n' % (index, label))\n",
    "    save_path = os.path.join(submit_dir, 'cmnli_predict.json')\n",
    "    with open(save_path, 'w') as f:\n",
    "        f.writelines(submit_results)\n",
    "    return save_path"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 初始化预测器\n",
    "predictor = Predictor(\n",
    "    model_file=pb_model_file)\n",
    "# 生成提交文件\n",
    "save_path = submit_cmnli(\n",
    "    predictor=predictor, data_dir=data_dir, submit_dir=pb_model_file)\n",
    "print(save_path)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}