English-word_sense_experiment.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## How the experiment works\n",
    "contributed by **SM Farhad Ali** and **Iftekhar Ahmen**\n",
    "\n",
    "We used python3 as our programming language to implement the paper. We also used ***nltk*** package which is a widely used python package to work with natural language. We used ***WordNet*** which is a lexical database for the English language. We used four different types of word embedding models, which are:\n",
    "\n",
    " 1. [Glove](http://nlp.stanford.edu/data/glove.6B.zip)\n",
    " 2. [Dependency-Based](http://u.cs.biu.ac.il/~yogo/data/syntemb/deps.words.bz2)\n",
    " 3. [Bag of Words (k = 2)](http://u.cs.biu.ac.il/~yogo/data/syntemb/bow2.words.bz2)\n",
    " 4. [Bag of Words (k = 5)](http://u.cs.biu.ac.il/~yogo/data/syntemb/bow5.words.bz2)\n",
    "\n",
    "Now, before we start to describe how our experiment works we need to describe some terms of our implementation. \n",
    "\n",
    "### Synset\n",
    "> \"Synonyms are words that have similar meanings. A synonym set, or **synset**, is a group of synonyms. A synset, therefore, corresponds to an abstract concept.\"\n",
    "\n",
    "#### Synset definition\n",
    "> \"Any word in the wordnet has some definitions associated with the word. A word can have multiple definitions which means every word can have multiple synsets and every synset has its own definition\"\n",
    "\n",
    "#### Word tokenizer\n",
    "> A word tokenizer splits word from the definition sentence of a synset and keeps those words in an array\n",
    "\n",
    "#### Pos tagger\n",
    "> pos-tagger/parts of speech tagger tags every word with respective parts of speech. For example, Noun, Verb, Adjective, etc\n",
    "\n",
    "#### Pos tagger\n",
    "> pos-tagger/parts of speech tagger tags every word with respective parts of speech. For example, Noun, Verb, Adjective, etc\n",
    "\n",
    "#### Cosine similarity\n",
    " >Cosine similarity is a measure of similarity between two non-zero vectors (in our case it is a word) of an inner product space that measures the cosine of the angle between them.\n",
    "#### WSD- word sense disambiguation\n",
    "> **word-sense disambiguation** (**WSD**)  concerned with identifying which sense of a word is used in a sentence.\n",
    "#### Lemma\n",
    "> The term ***Synset*** stands for \"set of synonyms\". A set of synonyms is a set of words with similar meaning, e.g. _ship, skiff, canoe, kayak_ might all be synonyms for _boat_. In the nltk, a ***Synset*** is in fact a set of ***lemmas*** with related meaning.\n",
    "\n",
    "## Description of the algorithm\n",
    "#### The steps can be described as followings:\n",
    " 1. The code first reads the word embedding from the file and stores it as a dictionary. The key of the dictionary is word and value is numpy array\n",
    " 2. From the input dataset, we read every line from text file and we run our code for each of the line. \n",
    " 3. For each line, we get a list of words with their respective parts of speech (**POS**). Now for every allowed **POS**(Noun, Verb, Adjective, Adverb) we find the associated vector/numpy array from our word embedding and store it in a dictionary. We also make a separate list for these chosen words only. \n",
    " 4. Now from this dictionary, we get word as key and values are vectors. We call this word as ***candidate word*** and vector as ***candidate vector***. For every candidate word, we find list of ***Lemma/Sense***.   For every ***Lemma/Sense*** we find it's **key** and ***defintion/glossary*** sentences.\n",
    " 5.  Now the ***defintion/glossary*** of sense are sentences. We repeat the steps to get words and it's respective parts of speech using ***pos-tagger*** again.  (We take only the Noun, Verb, Adjective, Adverb as per the paper). For every word we again get the word vector from the word embedding dictionary we calculate ***cosine similarity*** with ***candidate vector***. If the cosine similarity is bigger than a threshold we store the vector/array in a list. Lastly, we calculate the average of the vector for every Sense of the word and store the value in a dictionary. So, here the key is the Sense and value is the average of the vector/array. \n",
    " 6. Now for every Sense for every word for the original sentence we get a dictionary where the key is the Sense and value is the vector/numpy array. We calculate the number of Sense against the word and sort it by the length of Sense. \n",
    " 7. In the next step, we store this Sense and it's vector/numpy array against its actual word in a dictionary. \n",
    " 8. Now we find the length of the Sense and store in against its actual word in a different dictionary and we sort it by its length. \n",
    " 9. We initialize a context vector by the average of each word's vector. For every word in the context vector, we calculate the cosine similarity with the candidate word. We take the word which has the highest cosine similarity. We call the process as ***WSD- word sense disambiguation***. \n",
    " 10. If we can disambiguate any word we find the sense key and its name.\n",
    " 11. Now from the experiment dataset we read the text files and separate the key. With this key, we find it's corresponding sense key from the gold standard dataset. If we find it we call it as a recall. If we can not find it then it's a failure. \n",
    "> #### According to the paper, four persons had put the sentence key and several sense keys in the gold standard dataset. We counted the most common sense keys and selected one single sense key. \n",
    "  \n",
    "## Example for success\n",
    "- Let's assume we read a line from our source data file as ***\"bank.n.bnc.5425  bank ? 13 A beach of bleached stones gleamed bonewhite against the long stretch of grassy bank which rolled up to the pastures lining the valley floor . \"*** .\n",
    "- Now the code reads the line and splits the line by three parts. Which are:\n",
    "\t - ***bank.n.bnc.5425***\n",
    "\t - ***bank***\n",
    "\t - ***A beach of bleached stones gleamed bonewhite against the long stretch of grassy bank which rolled up to the pastures lining the valley floor .*** \n",
    "- With pos tagger we get below list from the sentence: \n",
    " \n",
    "\t\t    ('A', 'DT'),  ('beach', 'NN'),  ('of', 'IN'),  ('bleached', 'JJ'), \n",
    "    \t     ('stones', 'NNS'),  ('gleamed', 'VBD'),  ('bonewhite', 'RB'), \n",
    "    \t     ('against', 'IN'),  ('the', 'DT'),  ('long', 'JJ'),  ('stretch',\n",
    "    \t     'NN'),  ('of', 'IN'),  ('grassy', 'JJ'),  ('bank', 'NN'), \n",
    "    \t     ('which', 'WDT'),  ('rolled', 'VBD'),  ('up', 'RP'),  ('to', 'TO'),\n",
    "    \t     ('the', 'DT'),  ('pastures', 'NNS'),  ('lining', 'VBG'),  ('the',\n",
    "    \t     'DT'),  ('valley', 'NN'),  ('floor', 'NN'),  ('.', '.')\n",
    "- From this list we only take Noun, Verb, Adjectibe and Adverb. \n",
    "- Here we have to pick the ***bank*** word to disambiguate. We get 18 Senses for this word which are listed below. Note that for every ***Sense/Lemma*** of this list there is a list of array which we call vector. \n",
    "\t - *Lemma( bank.n.01)*\n",
    "\t - *Lemma(depository_financial_institution.n.01)*\n",
    "\t - *Lemma(bank.n.03)*\n",
    "\t - *Lemma(bank.n.04)*\n",
    "\t - *Lemma(bank.n.05)*\n",
    "\t - *Lemma(bank.n.06)*\n",
    "\t - *Lemma(bank.n.07)*\n",
    "\t - *Lemma(savings_bank.n.02)*\n",
    "\t - *Lemma(bank.n.09)*\n",
    "\t - *Lemma(bank.n.10 bank.v.01)*\n",
    "\t - *Lemma(bank.v.02)*\n",
    "\t - *Lemma(bank.v.03)*\n",
    "\t - *Lemma(bank.v.04)*\n",
    "\t - *Lemma(bank.v.05)*\n",
    "\t - *Lemma(deposit.v.02)*\n",
    "\t - *Lemma(bank.v.07)*\n",
    "\t - *Lemma(trust.v.01)*\n",
    "- If we run our code, we will see that for our example sentence and example word's (***bank***)  senses only the ***bank.n.01*** sense has the nearest sense value. \n",
    "- In our example sense key is ***bank%1:17:01::***\n",
    "- Now from the gold standard dataset we can see that, there are three lemma keys (***bank%1:17:01::*** , ***bank%1:17:00::*** , ***bank%1:17:01::*** ) for our sentence key ***bank.n.bnc.5425***. From the sense keys we choose the most frequent key which is ***bank%1:17:01::*** . So it's a match. For the word ***bank*** and sentence key  ***bank.n.bnc.5425*** our algorithm works. \n",
    "## Example for failure \n",
    "- Now for failure example let's assume our sentence from the dataset is: ***\"bank.n.bnc.9  bank ? 12 These are not just fizzy lager joints , but environments where a bank of six beer engines can sit happily on the bar counter offering a wide selection of real ales . \"*** .\n",
    "- Now the code reads the line and splits the line by three parts. Which are:\n",
    "\t - ***bank.n.bnc.9***\n",
    "\t - ***bank***\n",
    "\t - ***These are not just fizzy lager joints , but environments where a bank of six beer engines can sit happily on the bar counter offering a wide selection of real ales .*** \n",
    "- If we repeat the steps of our algorithm, we will find that for our word ***bank*** we will find lemma key ***'bank%1:17:02::'*** from the wordnet. \n",
    "- However we will find sense key ***bank%1:14:01::*** in our gold standard dataset against our sentence key ***bank.n.bnc.9***. So they don't match. So we consider this as a failure. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<table style=\"width:100%\">\n",
    "  <tr>\n",
    "      <td colspan=\"1\" style=\"text-align:left;background-color:#0071BD;color:white\">\n",
    "        <a rel=\"license\" href=\"http://creativecommons.org/licenses/by-nc/4.0/\">\n",
    "            <img alt=\"Creative Commons License\" style=\"border-width:0;float:left;padding-right:10pt\"\n",
    "                 src=\"https://i.creativecommons.org/l/by-nc/4.0/88x31.png\" />\n",
    "        </a>\n",
    "        &copy; T. Dong, C. Bauckhage<br/>\n",
    "        Licensed under a \n",
    "        <a rel=\"license\" href=\"http://creativecommons.org/licenses/by-nc/4.0/\" style=\"color:white\">\n",
    "            CC BY-NC 4.0\n",
    "        </a>.\n",
    "      </td>\n",
    "      <td colspan=\"2\" style=\"text-align:left;background-color:#66A5D1\">\n",
    "          <b>Acknowledgments:</b>\n",
    "          This material was prepared within the project\n",
    "          <a href=\"http://www.b-it-center.de/b-it-programmes/teaching-material/p3ml/\" style=\"color:black\">\n",
    "              P3ML\n",
    "          </a> \n",
    "          which is funded by the Ministry of Education and Research of Germany (BMBF)\n",
    "          under grant number 01/S17064. The authors gratefully acknowledge this support.\n",
    "      </td>\n",
    "  </tr>\n",
    "</table>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}