feature_gen.ipynb

{
 "metadata": {
  "name": "",
  "signature": "sha256:cee808a469f8a79f28dd23da9e2d334132cdc0347341955ff76954f575c9e212"
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "#Feature Generation for sequence ranking\n",
      "We extract features from word-bigram and character sequence to train a model that can assign each word-bigram proper proper score such that \n",
      "\n",
      "We mainly refer to **Zhang and Clark (2011) features**:\n",
      " <img src=\"zhang2011.png\">\n",
      " \n",
      ">1.Their *c0* is equalivant to our *sent[i:i+1]*, where *i* is the current possition to be considerd, and their *w-2w-1* is one of the entry in our *back_ward_bigram[i][j]*, in which w-1=sent[j:i]\n",
      "\n",
      ">2.These features are basically the same as Zhang and Clark (2007) features except that *feature7* is applid to partial word, while all the other features are applied to full word, i.e. there is a distinction of treatment for partial and full word, which is claimed to improve the result.\n",
      "\n",
      ">3.Two other change in their 2011 paper compard with the 2007 paper: (1) use early-update for the training rather than standard perceptron; (2) use 10CV to decdie best iteration number. Both tricks are irrelavant if you use MaxEnt instead of (structrured) perceptron.\n",
      "\n"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Selected feature\n",
      "As follows:\n",
      "\n",
      "- f1 word *w-1*\n",
      "- f2 bigram *w-2, w-1*\n",
      "- f3. word *w-1* & len(*w-1*)==1\n",
      "- f4. start(*w-1*) & len(*w-1*)\n",
      "- f5. end(*w-1*) & len(*w-1*)\n",
      "- f6. char-bigram *w-1[-1]*,*c0* \n",
      "- f8. *w-1[0]* & *w-1[:-1]*\n",
      "- f9. *w-1* & *c0*\n",
      "- f10. *w-2[:-1]* & *w-1*\n",
      "- f11. *w-1[0]* & *c0*\n",
      "- f12. *w-2[-1]* & *w-1[-1]*\n",
      "- f13. *w-2* & len(*w-1*)\n",
      "- f14. len(*w-2*) & *w-1*\n",
      "\n",
      "Note, the following feature are NOT selected:\n",
      "- f7 as it describes \"in word or wordhood\" properties\n",
      "\n",
      "\n",
      "## Extended feature\n",
      "As follows:\n",
      "\n",
      "#### Group A: head char to substitute begin/end char of word\n",
      "- fa4==fa5 head(*w-1*) & len(*w-1*)\n",
      "- fa6==f11 head(*w-1*) & *c0*\n",
      "- fa10 head(*w-2*) & *w-1*\n",
      "- fa12 head-bigram head(*w-2*), head(*w-1*)\n",
      "\n",
      "#### Group B: word tag to substitute begin/end char of word\n",
      "- fb4 tag(*w-1*) & len(*w-1*)\n",
      "- fb6 tag(*w-1*) & *c0*\n",
      "- fb10 hybrid bigram: tag(*w-2*) & *w-1*\n",
      "- fb12 tag-bigram tag(*w-2*), tag(*w-1*)\n",
      "- fb15 hybrid bigram: *w-2* & tag(*w-1*)\n",
      "\n",
      "#### Group C: other word structure feature\n",
      "- fc1: tag(*w-1*) and lenght pattern of left and right node\n",
      "\n",
      "### Possible further extension\n",
      "- use tag_of_character from *word2vec* to substitute *c0*\n",
      "- use tag_of_character(head) from *word2vec* to substitute *head*\n",
      "- use tag_of_word from *word2vec* to substitute *word*\n",
      "   - this require *modified training* procedure of word2vec to train on **customarized context**, rather than word-window. in this way, one can train word vector without doing segmentation\n"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "#\n",
      "# feature_gen has been tested for base_features\n",
      "#"
     ],
     "language": "python",
     "metadata": {},
     "outputs": []
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [],
     "language": "python",
     "metadata": {},
     "outputs": []
    }
   ],
   "metadata": {}
  }
 ]
}