clue/sentence_pair/diagnostics/predict.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# DIAGNOSTICS"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "诊断集，用于评估不同模型在9种语言学家总结的中文语言现象上的表现\n",
    "\n",
    "使用在CMNLI上训练过的模型，直接预测在这个诊断集上的结果，提交格式和CMNLI一致，在排行榜详情页可以看到结果"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 预测"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import json"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pyclue.tf1.tasks.sentence_pair.siamese.predict import Predictor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 加载保存好的模型路径（train.ipynb中最后打印出的model_file_dict中的'pb_model_file'地址）\n",
    "pb_model_file = ''\n",
    "\n",
    "# 测试数据路径（应包含test.txt文件）\n",
    "data_dir = '/workspace/projects/PyCLUE_Corpus/sentence_pair/cmnli'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def submit_diagnostics(predictor, submit_dir):\n",
    "    test_data_file = os.path.join(data_dir, 'test.txt')\n",
    "    submit_results = []\n",
    "    ids = [item[0] for item in predictor.processor.read_file(file_path=test_data_file)]\n",
    "    labels = [item['prediction'] for item in predictor.predict_from_file(input_file=test_data_file)]\n",
    "    for index, label in zip(ids, labels):\n",
    "        submit_results.append('{\"id\": \"%s\", \"label\": \"%s\"}\\n' % (index, label))\n",
    "    save_path = os.path.join(submit_dir, 'diagnostics_predict.json')\n",
    "    with open(save_path, 'w') as f:\n",
    "        f.writelines(submit_results)\n",
    "    return save_path"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 初始化预测器\n",
    "predictor = Predictor(\n",
    "    model_file=pb_model_file)\n",
    "# 生成提交文件\n",
    "save_path = submit_diagnostics(\n",
    "    predictor=predictor, data_dir=data_dir, submit_dir=pb_model_file)\n",
    "print(save_path)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}