300151071_Notebook6.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Notebook 6 - Knowledge Representation and NER"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "CSI4106 Artificial Intelligence  \n",
    "Fall 2019  \n",
    "Prepared by Caroline Barrière and Julian Templeton"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***INTRODUCTION***:  \n",
    "\n",
    "In this notebook you will explore one of the techniques and one of the resources that you have seen in class for Name-Entity Recognition (NER). Instead of typical PER, LOC, ORG entity types, we will go into the restaurant world and focus on entities of type *Cuisine*, *Dish* and others.\n",
    "\n",
    "You will begin by loading in our train and test corpora of restaurant related sentences and move to using a Gazetteer for NER of the words in the corpus.   \n",
    "\n",
    "Then you will explore Wordnet, a lexical semantic network in which knowledge is organized by interrelated synsets (groups of synonyms). Using Wordnet you will use the hyponyms of the *dish* to recognize how to tag tokens as *Dish*.   \n",
    "\n",
    "An important note is that this notebook will be working with *Single Words* rather than *Multi-words*. This means that when tagging words you will not be able to get all of the tags for each word. For example, a sentence \"Find me a restaurant where they serve fettucini alfredo\" is a sentence where the tag *Cuisine* should be used for several words (fettucini alfredo). But looking at single words, only fettucini may be tagged correctly. Dealing with multi-words is quite complex, and that is why we limit ourselves to single words in this notebook, but keep in mind that we would typically want to consider several words, not just single words.\n",
    "\n",
    "Also, the evaluation in this notebook is more qualitative in nature.  We will not perform the usual quantitative methods of precision/recall, but rather just look at examples of output and discuss."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***HOMEWORK***:  \n",
    "Go through the notebook by running each cell, one at a time.  \n",
    "Look for **(TO DO)** for the tasks that you need to perform. Do not edit the code outside of the questions which you are asked to answer unless specifically asked. Once you're done, Sign the notebook (at the end of the notebook), and submit it.  \n",
    "\n",
    "*The notebook will be marked on 15.  \n",
    "Each **(TO DO)** has a number of points associated with it.*\n",
    "***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {},
   "outputs": [],
   "source": [
    "import re # For the regular expressions that we will be using\n",
    "from nltk.corpus import wordnet # Import Wordnet"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**1. Setting up the corpus**   \n",
    "Before working on our NER tasks we must setup the corpus that we will be using. For this notebook we will be working with a corpus related to restaurants.   \n",
    "\n",
    "The restaurant corpus is provided on Brightspace but is also available at: https://groups.csail.mit.edu/sls/downloads/ as the *MIT Restaurant Corpus*. This corpus provides lists of tokens from sentences, where each sentence is separated by a blank line, with each token provided a NER tag such as *Cuisine*, *Dish*, *Other (O)*, ... We will explore these soon."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Begin collecting the sentences and tags from restauranttrain.bio.txt\n",
    "corpus_train = []\n",
    "with open(\"restauranttrain.bio.txt\") as f:\n",
    "    corpus_train = f.readlines()\n",
    "    \n",
    "# Begin collecting the sentences and tags from restauranttest.bio.txt\n",
    "corpus_test = []\n",
    "with open(\"restauranttest.bio.txt\") as f:\n",
    "    corpus_test = f.readlines()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We will convert the corpus into pairs of sentence tokens and their respective tags.\n",
    "# We will also limit the minimum length of sentences that we keep along with the total number\n",
    "# of sentences.\n",
    "def setup_sentences(corpus, min_sentence_len, max_sentences):\n",
    "    sentences = [] # All of our tokenized sentences with their respective NER tags\n",
    "    sentence = [[], []] # Storing the NER tags along with the sentences in two separate lists\n",
    "    # Format our corpus as a list of lists for easy use later\n",
    "    for line in corpus:\n",
    "        vals = line.split()\n",
    "        if (vals == []):\n",
    "            sentences.append(sentence)\n",
    "            sentence = [[], []]\n",
    "        else:\n",
    "            sentence[0].append(re.sub(r\"[A-Z]-\", \"\", vals[0])) # NER tag (we remove prefixes for simplicity)\n",
    "            sentence[1].append(vals[1]) # Sentence\n",
    "    # Now we will only keep sentences containing at least min_sentence_len words\n",
    "    sentences = [sentence for sentence in sentences if len(sentence[1]) >= min_sentence_len]\n",
    "    # Finally return up to max_sentences sentences\n",
    "    return sentences[:max_sentences]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_sentences = setup_sentences(corpus_train, 5, 1000)\n",
    "test_sentences = setup_sentences(corpus_test, 5, 500)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1000\n",
      "500\n"
     ]
    }
   ],
   "source": [
    "print(len(train_sentences))\n",
    "print(len(test_sentences))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['5', 'star', 'resturants', 'in', 'my', 'town'] \n",
      " ['Rating', 'Rating', 'O', 'Location', 'Location', 'Location'] \n",
      "\n",
      "['about', 'how', 'much', 'is', 'a', 'midpriced', 'bottle', 'of', 'good', 'wine', 'at', 'davidos', 'italian', 'palace'] \n",
      " ['O', 'O', 'O', 'O', 'O', 'Price', 'Rating', 'O', 'Rating', 'Cuisine', 'O', 'Restaurant_Name', 'Restaurant_Name', 'Restaurant_Name'] \n",
      "\n",
      "['are', 'there', 'any', 'authentic', 'vietnamese', 'restaurants', 'in', 'the', 'area', 'that', 'specialize', 'in', 'regional', 'dishes'] \n",
      " ['O', 'O', 'O', 'Cuisine', 'Cuisine', 'O', 'Location', 'Location', 'Location', 'O', 'O', 'O', 'Cuisine', 'O'] \n",
      "\n",
      "['can', 'i', 'get', 'a', 'list', 'of', 'restaurants', 'that', 'are', 'still', 'open', 'that', 'serve', 'pancakes'] \n",
      " ['O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'O', 'Hours', 'Hours', 'O', 'O', 'Dish'] \n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Let us take a look at a few sample sentences and their NER tags from train_sentences\n",
    "# We will discuss the tags themselves very soon!\n",
    "print(train_sentences[1][1], \"\\n\", train_sentences[1][0], \"\\n\")\n",
    "print(train_sentences[10][1], \"\\n\", train_sentences[10][0], \"\\n\")\n",
    "print(train_sentences[100][1], \"\\n\", train_sentences[100][0], \"\\n\")\n",
    "print(train_sentences[500][1], \"\\n\", train_sentences[500][0], \"\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**2. Understanding the corpus**   \n",
    "Let us explore the corpus to look at the kinds of NER tags being used."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Rating\n",
      "O\n",
      "Amenity\n",
      "Location\n",
      "Restaurant_Name\n",
      "Price\n",
      "Hours\n",
      "Dish\n",
      "Cuisine\n"
     ]
    }
   ],
   "source": [
    "tag_dict = { }\n",
    "# Collect all unique tags via a dictionary\n",
    "for tags, sentence in train_sentences:\n",
    "    for tag in tags:\n",
    "        tag_dict[tag] = True\n",
    "        \n",
    "tags = []\n",
    "# Output all tags used\n",
    "for key in tag_dict:\n",
    "    print(key)\n",
    "    tags.append(key)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, there are 9 different NER tags being used. Note that the tag *O* means Other.   \n",
    "So when we look at the sentence below, we can see that each word of a sentence in our corpus contains a NER tag:  \n",
    "['5', 'star', 'resturants', 'in', 'my', 'town']      \n",
    "['Rating', 'Rating', 'O', 'Location', 'Location', 'Location']   \n",
    "\n",
    "For example '5' is tagged *Rating*, 'in' is tagged *Location*, 'town' is tagged *Location*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Given a tag and list of tag/sentence lists, returns the list of all unique words with that\n",
    "# NER tag.\n",
    "def tag_words(target_tag, sentences):\n",
    "    words = []\n",
    "    for tags, sentence in sentences:\n",
    "        for tag, word in zip(tags, sentence):\n",
    "            if (target_tag == tag) and (not word in words):\n",
    "                words.append(word)\n",
    "    words.sort()\n",
    "    return words"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(TO DO) Q1 - 2 marks**   \n",
    "Using the function *tag_words()* above, explore the words associated with each type of named entity from *train_sentences*.  Write a few lines of code to output a few examples of *each type* of entity, as well as the number of different words for each type of entity."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "************************************************************\n",
      "The name of type of entity: Rating\n",
      "The number of different words: 58\n",
      "Examples: \n",
      "['four', 'high', 'superior', 'crowd', 'excellent', 'favorite', 'star', 'delicious', 'toppings', 'zagat', 'first', 'reviews', 'month', 'five', 'food', 'descent', 'health', 'pleasing', '5', 'nicest', '3', 'great', 'busy', 'notable', 'secret', 'rating', 'service', 'bottle', 'local', 'wonderful', 'start', 'best', 'least', 'on', 'starred', 'reviewed', 'stars', 'highly', 'few', 'views', 'rate', '4', 'famous', 'rated', 'score', 'a', 'highest', 'tripadvisor', 'good', 'quality', 'past', 'well', '2', 'top', 'nice', 'at', 'suggestions', 'ratings']\n",
      "************************************************************\n",
      "The name of type of entity: O\n",
      "The number of different words: 367\n",
      "Examples: \n",
      "['also', 'establishment', 'small', 'valentines', 'lunch', 'what', 'check', 'dining', 'breakfast', 'seen', 'anything', 'operation', 'rating', '00', 'spot', 'style', 'frequented', 'number', 'such', 'anniversary', 'pay', 'tad', 'proportions', 'joints', 'specialties', 'venues', 'and', 'places', '30', 'they', 'more', 'booked', 'bite', 'person', 'mark', 'parlors', 'restaurants', 'taste', 'car', 'accept', 'big', 'prices', 'on', 'few', 'get', 'is', '20', 'full', 'both', 'off', 'well', 'offered', 'make', 'four', 'allow', 'were', 'new', 'anywhere', 'along', 'which', '3', 'me', 'in', 'just', 'thats', 'miles', 'phone', 'see', 'call', 'restarurants', 'place', 'out', 'most', 'take', 'currently', 'upscale', 'day', 'go', 'needed', 'card', 'hit', 'starving', 'wednesday', 'expensive', 'arrange', 'stores', 'the', 'alot', 'kids', 'or', 'opens', 'offer', 'room', 'craving', 'based', 'grab', 'dish', 'my', 'enjoy', 'ablle', 'something', 'restraunts', 'dishes', '4', 'please', 'be', 'good', 'ti', 'restuarants', 'nice', 'open', 'high', 'should', 'adress', 'recommend', 'served', 'area', 'meals', 'steak', 'now', 'to', 'community', 'entrees', 'an', 'of', 'point', 'serves', 'showing', 'tasty', 'find', 'last', 'hotspot', 'want', 'since', 'specialty', 'cook', 'right', 'address', 'bring', 'located', 'have', 'one', '1', 'us', 'while', 'do', 'joint', 'price', 'too', 'this', 'hurry', 'featuring', 'are', 'it', 'smoke', 'store', 'can', 'use', 'help', 'about', 'try', 'with', 'list', 'serving', 'lujan', 'being', 'takes', 'per', 'need', 'for', 'eateries', 'would', 'catch', 'listing', 'coul', 'where', 'finda', 'diners', 'serve', 'gide', 'whats', 'hotel', 'date', 'restraunt', 'called', 'dont', 'fresh', 'reviews', 'contain', 'offers', 'route', 'tonight', 'bitch', 'resturants', 'its', 'service', 'been', 'eight', 'look', 'around', 'entree', 'people', 'those', 'online', 'much', 'permitted', 'allows', 'really', 'names', 'littleton', 'location', 'sell', 'our', 'whether', 'reservations', 'restaurant', 'set', 'menus', 'up', 'recently', 'credit', 'recipes', 'fine', 'book', 'wife', 'eatery', 'daughters', 'foods', 'feeling', 'near', 'show', 'birthday', '100', 'two', 'rated', 'hungry', 'dine', 'locations', 'a', 'pick', 'facilities', 'at', 'nearby', 'we', 'think', 'looking', 'game', 'specializing', 'party', 'local', 'stay', 'any', 'evening', 'menu', 'that', 'able', 'directions', 'someplace', 'has', 'some', 'type', 'percent', 'so', 'all', 'their', 'name', 'meal', 'you', 'anybody', 'how', 'early', 'options', 'does', 'available', 'i', '98', 'reservation', 'having', 'still', 'know', '8', 'option', 'cuisine', 'there', 'anyplace', 'reserve', 'search', 'food', 'watch', 'resteraunt', 'offering', 'great', 'pho', 'sells', 'telephone', 'going', 'chance', 'helping', 'give', 'close', 'as', 'tell', 'direct', 'order', 'theres', 'time', 'somewhere', 'than', 'terms', 'if', 'coupon', 'although', 'shops', 'home', 'very', 'eat', 'eating', 'house', 'but', 'today', 'numbers', '2', 'will', 'table', 'suitable', 'here', 'suggestions', 'before', 'features', 'ratings', 'end', 'who', 'by', 'dessert', 'got', 'begin', 'clubs', 'locate', 'spots', 'restauarants', 'satiisfy', 'shop', '6', 'paid', 'priced', 'within', 'hold', 'dinner', 'restoraunt', 'im', 'hours', 'sounding', 'other', 'baby', 'next', 'provides', 'least', 'like', 'am', 'restoraunts', 'pm', 'from', 'family', 'resturant', 'could', 'specialize', 'suggest']\n",
      "************************************************************\n",
      "The name of type of entity: Amenity\n",
      "The number of different words: 292\n",
      "Examples: \n",
      "['occasion', 'hotel', 'reservation', 'wait', 'date', '50s', '10', 'option', 'playing', 'hour', 'not', 'seven', 'view', 'happy', 'small', 'traditional', 'food', 'healthy', 'lunch', 'entertainment', 'door', 'offers', 'thru', 'watch', 'buffets', 'check', 'dining', 'alcohol', 'breakfast', 'great', 'carry', 'produce', 'my', 'throughs', 'visa', 'hangouts', 'bird', 'stand', 'code', 'service', 'children', 'stting', 'beer', '17', 'causual', 'style', 'isnt', 'people', 'pets', 'pay', 'design', 'brewpub', 'karaoke', 'online', 'red', 'elegant', 'plays', 'down', 'as', 'good', 'perfectly', 'senor', 'sit', 'ambiance', 'mastercard', 'split', 'jazz', 'dress', 'floating', 'nice', 'pet', 'location', 'require', 'stroller', 'child', 'samples', 'outs', 'specials', 'takeout', 'theme', 'no', 'reservations', 'restaurant', 'american', 'tourist', 'menus', 'outdoor', 'deals', 'delivery', 'seeing', 'casual', 'credit', 'huge', 'quiet', 'bands', 'rooms', 'dog', '12', 'jeans', 'owned', 'payment', 'arrangements', 'cabbage', 'carryout', 'fine', 'members', 'car', 'completely', 'if', 'fancy', 'college', 'standing', 'area', 'accept', 'big', 'dinning', 'meals', 'your', 'chinese', 'music', 'prices', 'on', 'setting', 'birthday', 'two', 'friendly', 'to', 'fun', 'entrees', 'pizza', 'eat', 'an', 'fixed', 'of', 'full', 'dine', 'a', 'lot', 'atmosphere', 'section', 'non', 'beverages', 'play', 'drink', 'serves', 'allowed', 'crowded', 'fixe', 'handicapped', 'make', 'at', 'seating', 'table', 'bar', 'fast', 'features', 'games', 'hang', 'hangout', 'by', 'catering', 'romantic', 'cloakroom', 'sports', 'grown', 'allow', 'internet', 'want', 'travelers', 'private', 'own', 'stylish', 'deck', 'groups', 'inside', 'dogs', 'allday', 'night', 'locally', 'celebrity', 'live', 'theater', 'smoking', 'cigars', 'comfortable', 'chjildren', 'excepts', 'families', 'childs', 'paypal', 'party', 'special', 'accepts', 'tables', 'local', 'bring', 'portioned', 'bathrooms', 'sitting', 'prix', 'menu', 'clean', 'delivers', 'eye', 'in', 'famous', 'group', 'quickly', 'kis', 'us', 'halal', 'valet', 'without', 'see', 'while', 'do', 'checks', 'dinner', 'fireplace', 'caters', 'place', 'playrooms', 'patio', 'are', 'some', 'water', 'diabetic', 'smoke', 'charming', 'can', 'out', 'discounts', 'line', 'promote', 'type', 'use', 'take', 'drive', 'variety', 'sightseeing', 'waterfront', 'celebs', 'all', 'with', 'deliver', 'wear', 'day', 'list', 'kid', 'bars', 'card', 'takes', 'outdoors', 'portions', 'carhops', 'meal', 'buffet', 'historic', 'you', 'for', 'ordered', 'early', 'historical', 'hotels', 'brewpubs', 'the', 'access', 'byob', 'eateries', 'would', 'partying', 'free', 'cards', 'debit', 'kids', 'wine', 'reservatuibs', 'business', 'family', 'soda', 'accessible', 'express', 'i', 'impress', 'outside', 'parking', 'refills', 'serve', 'skyline', 'celebrities', 'services', 'large']\n",
      "************************************************************\n",
      "The name of type of entity: Location\n",
      "The number of different words: 244\n",
      "Examples: \n",
      "['hotel', 'beacon', 'middleboro', '10', '8', 'detroit', 'kingston', 'hour', 'not', 'carolina', 'fort', 'three', 'ky', 'atlantic', 'fifteen', 'rd', 'francisco', 'route', 'maple', 'dining', 'closet', 'los', 'walking', '135', 'my', 'tri', 'washington', 'boulevard', '52nd', 'ocean', 'service', 'closed', 'jamaica', 'canton', 'march', 'atlanta', '25', 'post', 'isnt', 'york', 'around', '4', 'billerica', 'west', 'close', 'down', 'port', 'miami', 'tuscon', 'block', 'street', 'twenty', 'sunswept', 'and', 'far', 'location', 'government', 'places', 'battery', 'district', 'our', 'paris', 'chinatown', 'high', 'more', 'angeles', 'bay', 'valley', 'baltimore', '86th', 'time', 'chester', 'glen', 'redding', 'leoshe', 'town', 'than', 'china', 'central', 'berkeley', 'avenue', 'n', 'quincy', 'terminal', 'beach', 'minute', 'supermarket', 'independence', 'radius', 'area', 'near', 'md', 'shawsheen', 'arent', 'main', 'hospital', 'museum', 'state', 'to', 'current', '15', 'home', 'very', 'arlington', 'center', 'airport', 'georgian', 'of', 'memphis', 'locations', 'a', 'westbury', 'house', '2', 'nearest', 'stockton', 'at', 'nearby', 'napa', 'between', 'here', 'side', 'square', 'end', 'by', 'four', 'str', 'centre', 'sacramento', 'prescott', 'new', 'closest', 'distract', 'anywhere', 'island', 'th', 'along', 'hal', 'park', 'hear', 'five', 'theater', 'job', '3', 'blocks', 'thirty', 'stop', 'dc', 'vicinity', 'germantown', 'long', 'myrlte', 'manhattan', 'office', 'downtown', 'southborough', 'hicksville', 'local', 'located', 'me', 'ny', 'withing', 'florence', 'school', 'annapolis', 'in', 'one', 'received', 'volusia', 'california', '1', 'forest', 'miles', 'within', 'less', 'ten', 'inman', 'san', 'crossing', 'winter', 'plain', 'indiana', 'kerry', 'ashland', 'st', 'too', 'place', 'this', 'east', 'city', 'abington', 'montana', 'hill', 'lexington', 'roxbury', 'wichita', 'locust', 'minutes', 'away', 'ca', 'road', 'drive', 'oakland', 'waterfront', 'with', 'hovendon', 'tilburg', '5', 'next', 'jersey', 'millis', 'way', 'work', 'marina', 'westborough', 'montauk', 'little', 'mall', 'south', 'cleveland', 'ave', 'milford', 'mile', 'diego', 'the', 'governement', 'oak', 'neighborhood', 'pier', 'harvard', 'natick', 'elm', 'from', 'beaumont', 'distance', 'head', 'north', 'or', 'outside', 'myers', 'chattanooga', 'gona', 'roslyn', 'theatre']\n",
      "************************************************************\n",
      "The name of type of entity: Restaurant_Name\n",
      "The number of different words: 230\n",
      "Examples: \n",
      "['ichiban', '1790', 'yogurt', 'bertuccis', 'fishhouse', 'daisy', 'gs', 'henpeckss', 'cuisine', 'pie', 'popeyes', 'picos', 'bella', 'barrel', 'gallaghers', 'martins', 'tims', 'world', 'lobster', 'belle', 'cinnabon', 'kookoo', 'gourmet', 'chichis', 'stephen', 'bakery', 'italian', 'savanos', 'fox', 'ohgane', 'mcdonalds', 'shack', 'hound', 'ihop', 'outback', 'taco', 'hidden', 'white', 'apple', 'hampton', 'fortune', 'red', 'corral', 'rogers', 'q', 'paisas', 'russos', 'stables', 'and', 'cookhouse', 'cjs', 'johnny', 'mc', 'mirror', 'restaurant', 'fries', 'kong', 'denneys', 'bamboo', 'burgers', 'obriens', 'shrimp', 'tavern', 'bisuteki', 'ihops', 'restaurants', 'changs', 'china', 'landrys', 'lincolns', 'mercys', 'papa', 'lecooks', 'landing', 'steakhouse', 'n', 'bells', 'country', 'fine', 'hut', 'jacks', 'dunkin', 'aromas', 'spices', 'steak', 'hals', 'kitchen', 'donuts', 'hong', 'robin', 'olive', 'bees', 'home', 'rockets', 'pizza', 'greek', 'davidos', 'yogis', 'of', 'carlitos', 'toy', 'cafe', 'beverly', 'house', 'ho', 'cadete', 'charlies', 'nikos', 'queen', 'le', 'marlenes', 'bell', 'evans', 'sevans', 'bar', 'golden', 'outbacks', 'fridays', 'brighton', 'heidleburg', 'delicious', 'buckram', 'piacis', 'starbucks', 'tgif', 'huts', 'cracker', 'pub', 'kings', 'five', 'lees', 'applebees', 'sebastians', 'chicago', 'vie', 'lucky', 'long', 'oconnors', 'palace', 'subway', 'hotpot', 'sonics', 'blue', 'coffee', 'dennys', 'wendys', 'co', 'cheesecake', 'johns', 'romys', 'coast', 'asai', 'in', 'john', 'napoli', 'guys', 'polo', 'butterfields', 'garden', 'marco', 'murphys', 'mi', 'ashland', 'japanese', 'griffins', 'place', 'bertucci', 'biscuit', 'enterprise', 'la', 'bellinis', 'out', 'line', 'brook', 'carinos', 'burger', 'pacos', 'county', 'michelin', 'list', '5', 'factory', 'grill', 'maxs', 'mcdills', 'bob', 'gardens', 'ol', 'way', 'little', 'rivertown', 'buffet', 'rooftop', 'bs', 'pizzeria', 'monicas', 'uno', 'chilis', 'bj', 'inernational', 'hotels', 'dinosaur', 'donalds', 'madelines', 'passims', 'arbys', 'anthonys', 'castle', 'cicis', 'tomachhios', 'beanery', 'bouchon', 'kfc', 'dominos', 'silvers', 'nu', 'roy', 'dairy', 'donovans', 'king', 'north', 'diner', 'cj', 'bbq', 'needham', 'tudors', 'cristinas', 'abuelos', 'bueno', 'black']\n",
      "************************************************************\n",
      "The name of type of entity: Price\n",
      "The number of different words: 63\n",
      "Examples: \n",
      "['no', 'fantastic', 'high', '10', 'cost', 'more', 'deal', 'reasonable', 'deals', 'sort', 'not', 'person', 'than', 'inexpensive', 'upscale', 'amazing', 'five', 'dollars', 'midpriced', 'fine', 'dining', 'dollar', 'great', 'low', 'pricy', 'real', 'meal', 'mid', 'fairly', 'per', 'expensive', 'somewhat', 'wonderful', 'for', '00', 'average', 'only', 'prices', 'arent', 'two', '15', 'cheapest', 'isnt', 'specials', 'reasonably', 'slightly', 'free', 'cheaply', 'moderately', 'affordable', 'fair', 'decent', 'of', 'a', '1', 'nicely', 'under', 'good', 'really', 'moderate', 'too', 'place', 'cheap']\n",
      "************************************************************\n",
      "The name of type of entity: Hours\n",
      "The number of different words: 71\n",
      "Examples: \n",
      "['open', 'still', '10', '8', 'hour', 'seven', 'until', 'late', '7', '700', 'hours', '12', 'all', 'morning', 'night', 'lunch', 'till', 'day', '5', 'times', 'tonight', 'dining', 'brunch', 'breakfast', 'opened', 'nine', 'right', 'weekdays', 'midnight', '11', 'daily', '00', 'days', 'for', 'm', 'least', 'early', 'on', 'p', 'now', 'the', 'eight', '6', 'frequented', 'thanksgiving', 'this', 'around', '4', 'am', 'in', 'noon', '2am', 'evenings', 'pm', '9', 'week', '11pm', '1', 'a', 'past', 'supper', 'til', '2', 'dinner', 'opens', 'at', 'order', 'after', 'before', '30', '24']\n",
      "************************************************************\n",
      "The name of type of entity: Dish\n",
      "The number of different words: 165\n",
      "Examples: \n",
      "['sashimi', 'brisket', 'cookies', 'cuisine', 'pie', 'subs', 'corn', 'tofu', 'fresh', 'lobster', 'grits', 'chips', 'boca', 'food', 'healthy', 'roll', 'burrito', 'tapas', 'bowl', 'sub', 'cupcakes', 'onion', 'messy', 'sandwichs', 'dish', 'brownies', 'caeser', 'soft', 'swedish', 'soy', 'filet', 'sangrias', 'beans', 'beer', 'cocktails', 'spicy', 'sushi', 'seafood', 'style', 'dishes', 'york', 'entree', 'margaritas', 'wings', 'made', 'cheese', 'boardwalk', 'appetizers', 'curry', 'salmon', 'and', 'sake', 'sandwich', 'crispy', 'chocolate', 'fries', 'rib', 'fruity', 'gyro', 'pickle', 'choices', 'burgers', 'sandwiches', 'shrimp', 'town', 'curd', 'mojito', 'crabs', 'dog', 'fried', 'pita', 'chili', 'meats', 'bagels', 'steak', 'hip', 'buffalo', 'chowder', 'donuts', 'birthday', 'spit', 'to', 'home', 'salad', 'pizza', 'bratwurst', 'is', 'of', 'french', 'submarine', 'pastry', 'baba', 'bagel', 'cakes', 'tacos', 'hot', 'meatloaf', 'thin', 'bar', 'potato', 'tea', 'bottles', 'new', 'guacamole', 'enchiladas', 'dough', 'thick', 'beef', 'cream', 'bread', 'soup', 'gyros', 'chicago', 'rings', 'bourek', 'substitute', 'coffee', 'skillet', 'cheeseburger', 'dumplings', 'fake', 'pancakes', 'eye', 'toast', 'ice', 'meat', 'ribs', 'handcrafted', 'chicken', 'known', 'crusts', 'this', 'sauce', 'turkey', 'caesar', 'clam', 'fruits', 'take', 'toppings', 'burger', 'drinks', 'waffles', 'with', 'vegetarian', 'ganoush', 'shakes', 'southern', 'sweet', 'slices', 'sausage', 'fish', 'you', 'bacon', 'for', 'souvaki', 'spaghetti', 'milkshakes', 'bean', 'noodle', 'polish', 'wine', 'mignon', 'serve', 'meatballs', 'bbq']\n",
      "************************************************************\n",
      "The name of type of entity: Cuisine\n",
      "The number of different words: 130\n",
      "Examples: \n",
      "['hotel', 'african', 'irish', 'fresh', 'cali', 'tapas', 'barbecue', 'food', 'chilean', 'health', 'burrito', 'lunch', 'mussels', 'dining', 'low', 'breakfast', 'bakery', 'hoa', 'italian', 'vegeterian', 'indian', 'beer', 'sushi', 'seafood', 'style', 'taco', 'good', 'mystery', 'cuban', 'donut', 'sandwich', 'american', 'kong', 'korean', 'parlors', 'authentic', 'mongolian', 'deli', 'steakhouse', 'homemade', 'pancake', 'fine', 'brunch', 'ethiopian', 'eatery', 'steak', 'chinese', 'chowder', 'hong', 'theaters', 'home', 'cajun', 'bistro', 'pizza', 'greek', 'cafes', 'of', 'truck', 'cooked', 'french', 'cafe', 'kosher', 'pastry', 'vegan', 'house', 'smoothie', 'desert', 'drink', 'bar', 'fast', 'tea', 'soul', 'pad', 'dessert', 'southwestern', 'european', 'snack', 'brewery', 'cream', 'vietnamese', 'delis', 'filipino', 'eastern', 'coffee', 'market', 'mexican', 'english', 'ice', 'meat', 'garden', 'suishi', 'halal', 'lots', 'raw', 'dinner', 'joint', 'japanese', 'houses', 'spanish', 'steakhouses', 'asian', 'doughnut', 'mediterranean', 'regional', 'burger', 'vegetarian', 'barbeque', 'bars', 'jamaican', 'organic', 'pizzeria', 'for', 'afghan', 'colombian', 'eateries', 'scandanavian', 'thai', 'beanery', 'wine', 'brazilian', 'malaysian', 'family', 'german', 'diners', 'pizzerias', 'diner', 'garlic', 'fat', 'trucks', 'chain']\n"
     ]
    }
   ],
   "source": [
    "# Give some examples (words) for each type of entity along with the number of different words\n",
    "# tagged with each entity in train_sentences\n",
    "for i in range(len(tags)):\n",
    "    print(\"*\"*60)\n",
    "    print(\"The name of type of entity: \"+tags[i])\n",
    "    tag = tag_words(tags[i],train_sentences)\n",
    "    tag = list(set(tag))\n",
    "    print(\"The number of different words: \"+str(len(tag)))\n",
    "    print(\"Examples: \")\n",
    "    print(tag)    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**3. Using Gazetteers for NER**    \n",
    "Now we will explore using Gazetteers for NER. Specifically, we will be using Wikipedia's list of Cuisines to along with Edit Distance to tag any tokens as *Cuisine*."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***3.1. Setting up the Gazetteer***    \n",
    "First thing to do is to setup the Gazetteer from Wikipedia's list of Cuisines (https://en.wikipedia.org/wiki/List_of_cuisines). We could scrape the webpage and extract these, but for the sake of keeping it simple the notebook includes the file *cuisine_gazetteer.txt* that we will use to load the Gazetteer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Setup the Gazetteer\n",
    "gazetteer = []\n",
    "with open(\"cuisine_gazetteer.txt\") as f:\n",
    "    gazetteer = f.read().splitlines()\n",
    "\n",
    "# Since we are working with Single Words, we will only keep the first word \n",
    "# of each index in our Gazatteer.\n",
    "gazetteer = list(set([cuisine.split()[0].lower() for cuisine in gazetteer]))\n",
    "gazetteer.sort()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['ainu',\n",
       " 'albanian',\n",
       " 'andhra',\n",
       " 'anglo-indian',\n",
       " 'arab',\n",
       " 'argentine',\n",
       " 'armenian',\n",
       " 'assyrian',\n",
       " 'awadhi',\n",
       " 'azerbaijani',\n",
       " 'balochi',\n",
       " 'bangladeshi',\n",
       " 'belarusian',\n",
       " 'bengali',\n",
       " 'berber',\n",
       " 'brazilian',\n",
       " 'buddhist',\n",
       " 'bulgarian',\n",
       " 'cajun',\n",
       " 'cantonese',\n",
       " 'caribbean',\n",
       " 'chechen',\n",
       " 'chinese',\n",
       " 'circassian',\n",
       " 'crimean',\n",
       " 'cypriot',\n",
       " 'danish',\n",
       " 'english',\n",
       " 'estonian',\n",
       " 'filipino',\n",
       " 'french',\n",
       " 'georgian',\n",
       " 'german',\n",
       " 'goan',\n",
       " 'greek',\n",
       " 'gujarati',\n",
       " 'hong',\n",
       " 'hyderabad',\n",
       " 'indian',\n",
       " 'indonesian',\n",
       " 'inuit',\n",
       " 'irish',\n",
       " 'italian',\n",
       " 'jamaican',\n",
       " 'japanese',\n",
       " 'jewish',\n",
       " 'karnataka',\n",
       " 'kazakh',\n",
       " 'keralite',\n",
       " 'korean',\n",
       " 'kurdish',\n",
       " 'laotian',\n",
       " 'latvian',\n",
       " 'lebanese',\n",
       " 'lithuanian',\n",
       " 'louisiana',\n",
       " 'maharashtrian',\n",
       " 'malay',\n",
       " 'malaysian',\n",
       " 'mangalorean',\n",
       " 'mediterranean',\n",
       " 'mexican',\n",
       " 'mordovian',\n",
       " 'mughal',\n",
       " 'native',\n",
       " 'nepalese',\n",
       " 'new',\n",
       " 'odia',\n",
       " 'pakistani',\n",
       " 'parsi',\n",
       " 'pashtun',\n",
       " 'pennsylvania',\n",
       " 'peranakan',\n",
       " 'persian',\n",
       " 'peruvian',\n",
       " 'polish',\n",
       " 'portuguese',\n",
       " 'punjabi',\n",
       " 'rajasthani',\n",
       " 'romanian',\n",
       " 'russian',\n",
       " 'sami',\n",
       " 'serbian',\n",
       " 'sindhi',\n",
       " 'slovak',\n",
       " 'slovenian',\n",
       " 'somali',\n",
       " 'south',\n",
       " 'soviet',\n",
       " 'spanish',\n",
       " 'sri',\n",
       " 'taiwanese',\n",
       " 'tamil',\n",
       " 'tatar',\n",
       " 'thai',\n",
       " 'turkish',\n",
       " 'udupi',\n",
       " 'ukrainian',\n",
       " 'vietnamese',\n",
       " 'yamal',\n",
       " 'zambian',\n",
       " 'zanzibari']"
      ]
     },
     "execution_count": 107,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Look at the contents of the Gazetteer\n",
    "gazetteer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***3.2. Tagging Cuisines using Edit Distance***    \n",
    "Now that we have the Gazetteer, we need to define a metric to determine whether a token should be tagged as *Cuisine*. To do this we will use *Edit Distance*.   \n",
    "Recall that for regular edit distance we only look at *Deletion* (+1), *Insertion* (+1), and *Replacement* (+1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Calculates the Edit Distance between two words s1 and s2\n",
    "def edit_distance(s1, s2):\n",
    "    m=len(s1)+1\n",
    "    n=len(s2)+1\n",
    "\n",
    "    tbl = {}\n",
    "    for i in range(m): tbl[i,0]=i\n",
    "    for j in range(n): tbl[0,j]=j\n",
    "    for i in range(1, m):\n",
    "        for j in range(1, n):\n",
    "            cost = 0 if s1[i-1] == s2[j-1] else 1\n",
    "            tbl[i,j] = min(tbl[i, j-1]+1, tbl[i-1, j]+1, tbl[i-1, j-1]+cost)\n",
    "\n",
    "    return tbl[i,j]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Edit Distance between tuesday and thursday: 2\n",
      "Edit Distance between artificial and intelligence: 8\n",
      "Edit Distance between fresh and frozen: 4\n"
     ]
    }
   ],
   "source": [
    "# Let us check the edit-distance with a few examples\n",
    "print(\"Edit Distance between tuesday and thursday:\", edit_distance(\"tuesday\", \"thursday\"))\n",
    "print(\"Edit Distance between artificial and intelligence:\", edit_distance(\"artificial\", \"thursday\"))\n",
    "print(\"Edit Distance between fresh and frozen:\", edit_distance(\"fresh\", \"frozen\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(TO DO) Q2 - 2 marks**      \n",
    "With an understanding of the edit_distance function, we will define the function *ner_ed* that computes the edit distance between all tokens in our corpus with each item in our Gazetteer. If the edit distance between a token and a cuisine from our Gazetteer is under a specified threshold (an integer) we classify it as the same tag as those in our Gazetteer (in this case, *Cuisine*) by appending it to the list of words containing that tag. Specifically, we append the token and the gazetteer term as 'token/term'. The function returns a sorted list of the words that are determined to have the same tag as specified by the Gazetteer.    \n",
    "\n",
    "Below is a partially completed implementation of ner_ed which you must complete. Understanding the description of the function above you must complete the implementation below by adding the appropriate loop structure to allow the edit_distance logic to be correctly computed (already there, no need to edit it just need to setup the loops to be compatible). Recall that the structure of sentences is a list of lists ([[tags], [tokens]]). Also note that we will call a word from a Gazetteer a *term* and a word from a sentence a *token*."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Given a list of tag/sentence lists, a gazetteer, and a threshold, compute the edit distance\n",
    "# between each token from each sentence to each cuisine from the gazatteer. If the edit distance\n",
    "# is <= a provided threshold, we collect the word as a word that would be tagged with the same tag\n",
    "# as the Gazetteer content.\n",
    "# In this section the tag would be cuisine since the gazetteer is based on cuisines.\n",
    "def ner_ed(sentences, gazetteer, threshold):\n",
    "    words = []\n",
    "    # TO DO - Setup the three loops:\n",
    "    # Loop 1\n",
    "    for sentence in sentences:\n",
    "        # Loop 2 (introduces the variable token used below)\n",
    "        for token in sentence[1]:\n",
    "            # Loop 3 (introduces the variable term used below)\n",
    "            for term in gazetteer:\n",
    "                # Computing the edit distance between sentence tokens and \n",
    "                # the Gazetteer terms.\n",
    "                if (edit_distance(token, term) <= threshold):\n",
    "                    if (not token + \"/\" + term in words):\n",
    "                        words.append(token + \"/\" + term)\n",
    "                    break\n",
    "    # Sort the list\n",
    "    words.sort()\n",
    "    return words"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With this function, let's test it using three different thresholds on *test_sentences*."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The below will give sorted lists of words that tagged as Cuisine\n",
    "# This may take 40+ seconds (depending on your CPU), so be patient\n",
    "\n",
    "# Threshold = 0\n",
    "gz_ed_th0_words = ner_ed(test_sentences, gazetteer, threshold=0)\n",
    "# Threshold = 1\n",
    "gz_ed_th1_words = ner_ed(test_sentences, gazetteer, threshold=1)\n",
    "# Threshold = 2\n",
    "gz_ed_th2_words = ner_ed(test_sentences, gazetteer, threshold=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Distance 0\n",
      "['brazilian/brazilian', 'cajun/cajun', 'chinese/chinese', 'english/english', 'french/french', 'german/german', 'greek/greek', 'indian/indian', 'irish/irish', 'italian/italian', 'japanese/japanese', 'korean/korean', 'malaysian/malaysian', 'mexican/mexican', 'new/new', 'spanish/spanish', 'taiwanese/taiwanese', 'thai/thai', 'turkish/turkish', 'vietnamese/vietnamese']\n",
      "Distance 1\n",
      "['brazilian/brazilian', 'cajun/cajun', 'chinese/chinese', 'crab/arab', 'english/english', 'few/new', 'french/french', 'german/german', 'greek/greek', 'green/greek', 'indian/indian', 'irish/irish', 'italian/italian', 'japanese/japanese', 'korean/korean', 'long/hong', 'malaysian/malaysian', 'mexican/mexican', 'new/new', 'now/new', 'portugues/portuguese', 'spanish/spanish', 'taiwanese/taiwanese', 'thai/thai', 'than/thai', 'that/thai', 'tong/hong', 'turkish/turkish', 'vietnamese/vietnamese']\n",
      "Distance 2\n",
      "['along/hong', 'am/sami', 'american/mexican', 'an/ainu', 'and/ainu', 'any/ainu', 'are/arab', 'area/arab', 'atire/native', 'away/arab', 'barat/arab', 'bars/parsi', 'bean/goan', 'bec/new', 'beer/berber', 'both/south', 'brazilian/brazilian', 'brunch/french', 'burger/berber', 'cajun/cajun', 'can/cajun', 'caual/yamal', 'cheese/chinese', 'chicken/chechen', 'chinese/chinese', 'crab/arab', 'dim/odia', 'dine/ainu', 'dish/danish', 'dog/hong', 'down/goan', 'e/new', 'english/english', 'family/tamil', 'few/new', 'fin/ainu', 'find/ainu', 'fine/ainu', 'free/greek', 'french/french', 'fresh/french', 'fruit/inuit', 'fry/sri', 'ga/goan', 'german/german', 'get/new', 'go/goan', 'going/goan', 'good/goan', 'grat/arab', 'great/greek', 'greek/greek', 'green/greek', 'has/thai', 'hey/new', 'hi/sri', 'home/hong', 'hot/hong', 'hour/hong', 'how/hong', 'i/sri', 'idea/odia', 'in/ainu', 'indian/indian', 'irish/irish', 'italian/italian', 'japanese/japanese', 'john/goan', 'king/ainu', 'know/new', 'korean/korean', 'lamb/sami', 'latin/laotian', 'le/new', 'let/new', 'lone/hong', 'long/hong', 'malaysian/malaysian', 'mall/malay', 'may/malay', 'me/new', 'menu/ainu', 'mexican/mexican', 'morgan/korean', 'movie/soviet', 'nail/tamil', 'name/sami', 'near/new', 'neck/new', 'need/new', 'new/new', 'no/new', 'non/goan', 'north/south', 'not/new', 'now/new', 'ny/new', 'oak/goan', 'ohio/odia', 'oil/odia', 'on/goan', 'one/hong', 'or/sri', 'out/south', 'own/goan', 'park/parsi', 'per/new', 'pet/new', 'phone/hong', 'pine/ainu', 'portugues/portuguese', 'prix/sri', 'ranch/french', 'raw/arab', 'rd/sri', 'rib/arab', 'road/goan', 'route/south', 'salad/malay', 'sea/new', 'see/new', 'sin/ainu', 'small/somali', 'some/sami', 'soul/south', 'soup/south', 'spanish/danish', 'st/sri', 'star/tatar', 'sub/sri', 'sum/sami', 'taiwanese/taiwanese', 'tapas/tatar', 'ten/new', 'tgi/sri', 'th/thai', 'thai/thai', 'than/goan', 'that/thai', 'thats/thai', 'the/thai', 'their/thai', 'them/thai', 'they/thai', 'this/thai', 'thru/thai', 'tin/ainu', 'todai/thai', 'tong/hong', 'town/goan', 'try/sri', 'turkish/kurdish', 'vietnamese/vietnamese', 'view/new', 'what/thai', 'wine/ainu', 'wing/ainu', 'wrap/arab', 'yes/new', 'yet/new']\n"
     ]
    }
   ],
   "source": [
    "print(\"Distance 0\")\n",
    "print(gz_ed_th0_words)\n",
    "print(\"Distance 1\")\n",
    "print(gz_ed_th1_words)\n",
    "print(\"Distance 2\")\n",
    "print(gz_ed_th2_words)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(TO DO) Q3 - 4 marks**   \n",
    "1) Describe what starts to happen as the threshold increases. Are any other cuisines found by increasing the threshold (That weren't previously there)?  \n",
    "2) A good threshold should be relative to the length of the term. This means that rather than using an integer threshold, we use a float percentage. Then, when checking the threshold, we multiply the threshold by the length of the Gazetteer term. Copy over your implementation of the *ner_ed* function into the *ner_ed_rel* below, modify the function to use a relative threshold as described above.  \n",
    "3) Test this new approach using a 25% and 50% threshold. Print the returned lists.    \n",
    "4) Does relative thresholding better capture variations than not using a relative thresholding? ***Look at the short words vs longer words.***"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Q3 - ANSWER (1)   \n",
    "Describe what starts to happen as the threshold increases\n",
    ": As a result, the number of words in the list has increased as even slightly different words have been added to the list.\n",
    "Are any other cuisines found by increasing the thershold (that weren't previously there)? \n",
    ": By increasing the thershold from 0 to 1 or 2, we can find 'portugues/portuguese'.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Q3 - ANSWER (2)\n",
    "\n",
    "def ner_ed_rel(sentences, gazetteer, threshold):\n",
    "    words = []\n",
    "    for sentence in sentences:\n",
    "        for token in sentence[1]:\n",
    "            for term in gazetteer:\n",
    "                if (edit_distance(token, term) <= threshold*len(term)):\n",
    "                    if (not token + \"/\" + term in words):\n",
    "                        words.append(token + \"/\" + term)\n",
    "                    break\n",
    "    # Sort the list\n",
    "    words.sort()\n",
    "    return words"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "25% Threshold\n",
      "['brazilian/brazilian', 'cajun/cajun', 'chinese/chinese', 'english/english', 'french/french', 'german/german', 'greek/greek', 'indian/indian', 'irish/irish', 'italian/italian', 'japanese/japanese', 'korean/korean', 'malaysian/malaysian', 'mexican/mexican', 'new/new', 'spanish/spanish', 'taiwanese/taiwanese', 'thai/thai', 'turkish/turkish', 'vietnamese/vietnamese']\n",
      "50% Threshold\n",
      "['brazilian/brazilian', 'cajun/cajun', 'chinese/chinese', 'english/english', 'french/french', 'german/german', 'greek/greek', 'indian/indian', 'irish/irish', 'italian/italian', 'japanese/japanese', 'korean/korean', 'malaysian/malaysian', 'mexican/mexican', 'new/new', 'spanish/spanish', 'taiwanese/taiwanese', 'thai/thai', 'turkish/turkish', 'vietnamese/vietnamese']\n"
     ]
    }
   ],
   "source": [
    "# Q3 - ANSWER (3)\n",
    "# 25% Threshold\n",
    "gz_ed_th25_words = ner_ed(test_sentences, gazetteer, threshold=0.25)\n",
    "print(\"25% Threshold\")\n",
    "print(gz_ed_th25_words)\n",
    "# 50% threshold\n",
    "gz_ed_th50_words = ner_ed(test_sentences, gazetteer, threshold=0.5)\n",
    "print(\"50% Threshold\")\n",
    "print(gz_ed_th50_words)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Q3 - ANSWER (4)    \n",
    "Does relative thresholding better capture variations than not using a relative thresholding?   \n",
    "\n",
    "Yes. When I didn't use relative thresholding, the more threshold was incremented, the more errors were made mainly in the case of short words. Here, errors mean that pair of words that are not relevant at all but just slightly different from each other are added to the list.\n",
    "\n",
    "However, when I use relative thresholding, increasing threshold did not produce the above-mentioned error and gave the same result."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**4. Exploring Wordnet**   \n",
    "Let's first explore a bit the wordnet interface within nltk.  \n",
    "You can also look a the [WordNet interface description](http://www.nltk.org/howto/wordnet.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Synset('evaluation.n.02'), Synset('evaluation.n.01'), Synset('rating.n.03'), Synset('military_rank.n.01'), Synset('rate.v.01'), Synset('rate.v.02'), Synset('rate.v.03'), Synset('rat.v.01'), Synset('rat.v.02'), Synset('fink.v.01'), Synset('rat.v.04'), Synset('rat.v.05'), Synset('denounce.v.04')]\n",
      "13\n"
     ]
    }
   ],
   "source": [
    "from nltk.corpus import wordnet # Import Wordnet\n",
    "# A synset is a concept associated with a set of synonyms\n",
    "ratingSenses = wordnet.synsets('rating')\n",
    "print(ratingSenses)\n",
    "print(len(ratingSenses))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This shows that there are 13 senses of rating, 4 nouns and 9 verbs.  The word displayed is the most representative word for each sense.  \n",
    "\n",
    "You can try other words.  I recommend that you also perform the same search [online](http://wordnetweb.princeton.edu/perl/webwn) to better understand the results."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's look at the basic information in each synset.        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We define a function to print the basic information\n",
    "def printBasicSynsetInfo(d):\n",
    "    print(\"SynLemmas\")\n",
    "    print(d.lemmas())\n",
    "    # Print the synonyms\n",
    "    print(\"Synonyms\")\n",
    "    synonyms = [l.name() for l in d.lemmas()]\n",
    "    print(synonyms)\n",
    "    # Print the definition\n",
    "    print(\"Definition\")\n",
    "    print(d.definition())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Sense 0]\n",
      "SynLemmas\n",
      "[Lemma('evaluation.n.02.evaluation'), Lemma('evaluation.n.02.valuation'), Lemma('evaluation.n.02.rating')]\n",
      "Synonyms\n",
      "['evaluation', 'valuation', 'rating']\n",
      "Definition\n",
      "an appraisal of the value of something\n",
      "\n",
      "[Sense 1]\n",
      "SynLemmas\n",
      "[Lemma('evaluation.n.01.evaluation'), Lemma('evaluation.n.01.rating')]\n",
      "Synonyms\n",
      "['evaluation', 'rating']\n",
      "Definition\n",
      "act of ascertaining or fixing the value or worth of\n",
      "\n",
      "[Sense 2]\n",
      "SynLemmas\n",
      "[Lemma('rating.n.03.rating')]\n",
      "Synonyms\n",
      "['rating']\n",
      "Definition\n",
      "standing or position on a scale\n",
      "\n",
      "[Sense 3]\n",
      "SynLemmas\n",
      "[Lemma('military_rank.n.01.military_rank'), Lemma('military_rank.n.01.military_rating'), Lemma('military_rank.n.01.paygrade'), Lemma('military_rank.n.01.rating')]\n",
      "Synonyms\n",
      "['military_rank', 'military_rating', 'paygrade', 'rating']\n",
      "Definition\n",
      "rank in a military organization\n",
      "\n",
      "[Sense 4]\n",
      "SynLemmas\n",
      "[Lemma('rate.v.01.rate'), Lemma('rate.v.01.rank'), Lemma('rate.v.01.range'), Lemma('rate.v.01.order'), Lemma('rate.v.01.grade'), Lemma('rate.v.01.place')]\n",
      "Synonyms\n",
      "['rate', 'rank', 'range', 'order', 'grade', 'place']\n",
      "Definition\n",
      "assign a rank or rating to\n",
      "\n",
      "[Sense 5]\n",
      "SynLemmas\n",
      "[Lemma('rate.v.02.rate')]\n",
      "Synonyms\n",
      "['rate']\n",
      "Definition\n",
      "be worthy of or have a certain rating\n",
      "\n",
      "[Sense 6]\n",
      "SynLemmas\n",
      "[Lemma('rate.v.03.rate'), Lemma('rate.v.03.value')]\n",
      "Synonyms\n",
      "['rate', 'value']\n",
      "Definition\n",
      "estimate the value of\n",
      "\n",
      "[Sense 7]\n",
      "SynLemmas\n",
      "[Lemma('rat.v.01.rat')]\n",
      "Synonyms\n",
      "['rat']\n",
      "Definition\n",
      "desert one's party or group of friends, for example, for one's personal advantage\n",
      "\n",
      "[Sense 8]\n",
      "SynLemmas\n",
      "[Lemma('rat.v.02.rat')]\n",
      "Synonyms\n",
      "['rat']\n",
      "Definition\n",
      "employ scabs or strike breakers in\n",
      "\n",
      "[Sense 9]\n",
      "SynLemmas\n",
      "[Lemma('fink.v.01.fink'), Lemma('fink.v.01.scab'), Lemma('fink.v.01.rat'), Lemma('fink.v.01.blackleg')]\n",
      "Synonyms\n",
      "['fink', 'scab', 'rat', 'blackleg']\n",
      "Definition\n",
      "take the place of work of someone on strike\n",
      "\n",
      "[Sense 10]\n",
      "SynLemmas\n",
      "[Lemma('rat.v.04.rat')]\n",
      "Synonyms\n",
      "['rat']\n",
      "Definition\n",
      "give (hair) the appearance of being fuller by using a rat\n",
      "\n",
      "[Sense 11]\n",
      "SynLemmas\n",
      "[Lemma('rat.v.05.rat')]\n",
      "Synonyms\n",
      "['rat']\n",
      "Definition\n",
      "catch rats, especially with dogs\n",
      "\n",
      "[Sense 12]\n",
      "SynLemmas\n",
      "[Lemma('denounce.v.04.denounce'), Lemma('denounce.v.04.tell_on'), Lemma('denounce.v.04.betray'), Lemma('denounce.v.04.give_away'), Lemma('denounce.v.04.rat'), Lemma('denounce.v.04.grass'), Lemma('denounce.v.04.shit'), Lemma('denounce.v.04.shop'), Lemma('denounce.v.04.snitch'), Lemma('denounce.v.04.stag')]\n",
      "Synonyms\n",
      "['denounce', 'tell_on', 'betray', 'give_away', 'rat', 'grass', 'shit', 'shop', 'snitch', 'stag']\n",
      "Definition\n",
      "give away information about somebody\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# We can print the information for each sense of \"rating\"\n",
    "for i in range(len(ratingSenses)):\n",
    "    print(\"[Sense \" + str(i) + \"]\")\n",
    "    printBasicSynsetInfo(ratingSenses[i])\n",
    "    print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A rich taxonomy has been manually developed in Wordnet, making it a rich resource.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We define a function to print the basic information, receives a synset\n",
    "def printTaxonomyInfo(d):\n",
    "    # Print the synonmyms\n",
    "    synonyms = [l.name() for l in d.lemmas()]\n",
    "    print(\"Synonyms:\")\n",
    "    print(synonyms)\n",
    "    # Print the hypernyms\n",
    "    print(\"Hypernyms:\")\n",
    "    print(d.hypernyms())\n",
    "    # Print the hyponyms *** We will use these later so note how we get the hyponyms ***\n",
    "    print(\"Hyponyms:\")\n",
    "    print(d.hyponyms())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(TO DO) Q4 - 2 marks**   \n",
    "Choose two words and write code to print the taxonomic information for all senses of those words using printTaxonomyInfo()."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Sense 0]\n",
      "Synonyms:\n",
      "['talk', 'talking']\n",
      "Hypernyms:\n",
      "[Synset('conversation.n.01')]\n",
      "Hyponyms:\n",
      "[Synset('cant.n.04'), Synset('dialogue.n.01'), Synset('heart-to-heart.n.01'), Synset('shmooze.n.01'), Synset('shop_talk.n.01'), Synset('wind.n.04'), Synset('yak.n.01')]\n",
      "\n",
      "[Sense 1]\n",
      "Synonyms:\n",
      "['talk']\n",
      "Hypernyms:\n",
      "[Synset('discussion.n.01')]\n",
      "Hyponyms:\n",
      "[]\n",
      "\n",
      "[Sense 2]\n",
      "Synonyms:\n",
      "['talk']\n",
      "Hypernyms:\n",
      "[Synset('lecture.n.03')]\n",
      "Hyponyms:\n",
      "[Synset('chalk_talk.n.01')]\n",
      "\n",
      "[Sense 3]\n",
      "Synonyms:\n",
      "['lecture', 'public_lecture', 'talk']\n",
      "Hypernyms:\n",
      "[Synset('address.n.03')]\n",
      "Hyponyms:\n",
      "[]\n",
      "\n",
      "[Sense 4]\n",
      "Synonyms:\n",
      "['talk', 'talk_of_the_town']\n",
      "Hypernyms:\n",
      "[Synset('gossip.n.02')]\n",
      "Hyponyms:\n",
      "[]\n",
      "\n",
      "[Sense 5]\n",
      "Synonyms:\n",
      "['talk', 'speak']\n",
      "Hypernyms:\n",
      "[Synset('communicate.v.02')]\n",
      "Hyponyms:\n",
      "[Synset('chat_up.v.01'), Synset('cheek.v.01'), Synset('continue.v.02'), Synset('converse.v.01'), Synset('dish_the_dirt.v.01'), Synset('dogmatize.v.02'), Synset('hold_forth.v.01'), Synset('level.v.05'), Synset('monologuize.v.01'), Synset('orate.v.01'), Synset('pontificate.v.02'), Synset('rap.v.04'), Synset('spiel.v.02'), Synset('talk_down.v.02'), Synset('talk_turkey.v.01')]\n",
      "\n",
      "[Sense 6]\n",
      "Synonyms:\n",
      "['talk', 'speak', 'utter', 'mouth', 'verbalize', 'verbalise']\n",
      "Hypernyms:\n",
      "[Synset('communicate.v.02')]\n",
      "Hyponyms:\n",
      "[Synset('babble.v.01'), Synset('bark.v.01'), Synset('bay.v.01'), Synset('begin.v.04'), Synset('blubber.v.02'), Synset('blurt_out.v.01'), Synset('bumble.v.03'), Synset('cackle.v.01'), Synset('chatter.v.04'), Synset('chatter.v.05'), Synset('deliver.v.01'), Synset('drone.v.02'), Synset('enthuse.v.02'), Synset('generalize.v.02'), Synset('gulp.v.02'), Synset('hiss.v.03'), Synset('lip_off.v.01'), Synset('mumble.v.01'), Synset('murmur.v.01'), Synset('open_up.v.07'), Synset('peep.v.04'), Synset('rant.v.01'), Synset('rasp.v.02'), Synset('read.v.03'), Synset('shout.v.01'), Synset('sing.v.02'), Synset('slur.v.03'), Synset('snap.v.01'), Synset('snivel.v.01'), Synset('speak_in_tongues.v.01'), Synset('speak_up.v.02'), Synset('swallow.v.04'), Synset('talk_of.v.01'), Synset('tone.v.01'), Synset('tone.v.02'), Synset('troll.v.07'), Synset('verbalize.v.01'), Synset('vocalize.v.05'), Synset('whiff.v.05'), Synset('whisper.v.01'), Synset('yack.v.01')]\n",
      "\n",
      "[Sense 7]\n",
      "Synonyms:\n",
      "['speak', 'talk']\n",
      "Hypernyms:\n",
      "[Synset('communicate.v.02')]\n",
      "Hyponyms:\n",
      "[Synset('run_on.v.01'), Synset('slang.v.01'), Synset('smatter.v.03')]\n",
      "\n",
      "[Sense 8]\n",
      "Synonyms:\n",
      "['spill', 'talk']\n",
      "Hypernyms:\n",
      "[Synset('tell.v.02')]\n",
      "Hyponyms:\n",
      "[]\n",
      "\n",
      "[Sense 9]\n",
      "Synonyms:\n",
      "['spill_the_beans', 'let_the_cat_out_of_the_bag', 'talk', 'tattle', 'blab', 'peach', 'babble', 'sing', 'babble_out', 'blab_out']\n",
      "Hypernyms:\n",
      "[Synset('unwrap.v.02')]\n",
      "Hyponyms:\n",
      "[]\n",
      "\n",
      "[Sense 10]\n",
      "Synonyms:\n",
      "['lecture', 'talk']\n",
      "Hypernyms:\n",
      "[Synset('teach.v.01')]\n",
      "Hyponyms:\n",
      "[Synset('preach.v.01')]\n",
      "\n",
      "[Sense 0]\n",
      "Synonyms:\n",
      "['deliver', 'present']\n",
      "Hypernyms:\n",
      "[Synset('talk.v.02')]\n",
      "Hyponyms:\n",
      "[Synset('speechify.v.01')]\n",
      "\n",
      "[Sense 1]\n",
      "Synonyms:\n",
      "['deliver']\n",
      "Hypernyms:\n",
      "[Synset('bring.v.04')]\n",
      "Hyponyms:\n",
      "[Synset('consign.v.03'), Synset('drop.v.08'), Synset('misdeliver.v.01'), Synset('serve.v.11')]\n",
      "\n",
      "[Sense 2]\n",
      "Synonyms:\n",
      "['hand_over', 'fork_over', 'fork_out', 'fork_up', 'turn_in', 'deliver', 'render']\n",
      "Hypernyms:\n",
      "[Synset('pass.v.05')]\n",
      "Hyponyms:\n",
      "[Synset('bail.v.02'), Synset('give_away.v.03')]\n",
      "\n",
      "[Sense 3]\n",
      "Synonyms:\n",
      "['rescue', 'deliver']\n",
      "Hypernyms:\n",
      "[Synset('save.v.03')]\n",
      "Hyponyms:\n",
      "[Synset('reprieve.v.02'), Synset('salvage.v.01')]\n",
      "\n",
      "[Sense 4]\n",
      "Synonyms:\n",
      "['extradite', 'deliver', 'deport']\n",
      "Hypernyms:\n",
      "[Synset('expel.v.01')]\n",
      "Hyponyms:\n",
      "[Synset('repatriate.v.01')]\n",
      "\n",
      "[Sense 5]\n",
      "Synonyms:\n",
      "['render', 'deliver', 'return']\n",
      "Hypernyms:\n",
      "[Synset('communicate.v.01')]\n",
      "Hyponyms:\n",
      "[]\n",
      "\n",
      "[Sense 6]\n",
      "Synonyms:\n",
      "['deliver']\n",
      "Hypernyms:\n",
      "[Synset('utter.v.02')]\n",
      "Hyponyms:\n",
      "[]\n",
      "\n",
      "[Sense 7]\n",
      "Synonyms:\n",
      "['deliver', 'redeem', 'save']\n",
      "Hypernyms:\n",
      "[]\n",
      "Hyponyms:\n",
      "[]\n",
      "\n",
      "[Sense 8]\n",
      "Synonyms:\n",
      "['deliver', 'drive_home']\n",
      "Hypernyms:\n",
      "[Synset('give.v.01')]\n",
      "Hyponyms:\n",
      "[Synset('land.v.05')]\n",
      "\n",
      "[Sense 9]\n",
      "Synonyms:\n",
      "['surrender', 'cede', 'deliver', 'give_up']\n",
      "Hypernyms:\n",
      "[Synset('give.v.08')]\n",
      "Hyponyms:\n",
      "[Synset('sell.v.05'), Synset('sign_away.v.01'), Synset('yield_up.v.01')]\n",
      "\n",
      "[Sense 10]\n",
      "Synonyms:\n",
      "['deliver', 'pitch']\n",
      "Hypernyms:\n",
      "[Synset('throw.v.01')]\n",
      "Hyponyms:\n",
      "[]\n",
      "\n",
      "[Sense 11]\n",
      "Synonyms:\n",
      "['give_birth', 'deliver', 'bear', 'birth', 'have']\n",
      "Hypernyms:\n",
      "[Synset('produce.v.01')]\n",
      "Hyponyms:\n",
      "[Synset('calve.v.02'), Synset('cub.v.01'), Synset('drop.v.23'), Synset('farrow.v.01'), Synset('fawn.v.03'), Synset('foal.v.01'), Synset('have_a_bun_in_the_oven.v.01'), Synset('kitten.v.01'), Synset('lamb.v.01'), Synset('litter.v.03'), Synset('twin.v.04'), Synset('whelp.v.01')]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# TO DO - Choose two words and print the taxonomix information for all \n",
    "# senses of those words using printTaxonomyInfo\n",
    "\n",
    "# Word 1\n",
    "talkSenses = wordnet.synsets('talk')\n",
    "\n",
    "for i in range(len(talkSenses)):\n",
    "    print(\"[Sense \" + str(i) + \"]\")\n",
    "    printTaxonomyInfo(talkSenses[i])\n",
    "    print()\n",
    "    \n",
    "# Word 2\n",
    "deliverSenses = wordnet.synsets('deliver')\n",
    "\n",
    "for i in range(len(deliverSenses)):\n",
    "    print(\"[Sense \" + str(i) + \"]\")\n",
    "    printTaxonomyInfo(deliverSenses[i])\n",
    "    print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now to show how to get the names from each hyponym by using our example *ratingSenses*."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Sense 0]\n",
      "bond_rating\n",
      "mark\n",
      "overvaluation\n",
      "pricing\n",
      "reevaluation\n",
      "undervaluation\n",
      "\n",
      "[Sense 1]\n",
      "marking\n",
      "\n",
      "[Sense 2]\n",
      "\n",
      "[Sense 3]\n",
      "flag_rank\n",
      "\n",
      "[Sense 4]\n",
      "downgrade\n",
      "prioritize\n",
      "reorder\n",
      "seed\n",
      "sequence\n",
      "shortlist\n",
      "subordinate\n",
      "superordinate\n",
      "upgrade\n",
      "\n",
      "[Sense 5]\n",
      "\n",
      "[Sense 6]\n",
      "revalue\n",
      "\n",
      "[Sense 7]\n",
      "\n",
      "[Sense 8]\n",
      "\n",
      "[Sense 9]\n",
      "\n",
      "[Sense 10]\n",
      "\n",
      "[Sense 11]\n",
      "\n",
      "[Sense 12]\n",
      "sell_out\n",
      "\n"
     ]
    }
   ],
   "source": [
    "for i in range(len(ratingSenses)):\n",
    "    print(\"[Sense \" +  str(i) + \"]\")\n",
    "    for hyponym in ratingSenses[i].hyponyms():\n",
    "        # Access the hyponym names with: hyponym.name().split(\".\")[0]\n",
    "        print(hyponym.name().split(\".\")[0])\n",
    "    print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**5. Using Wordnet for NER**   \n",
    "Having explored Wordnet, we will use what we have learned to try another method of NER tagging. Specifically, we will be trying to tag any words labelled *Dish* by using the edit distance and the edit distance thresholding technique that we used in section 3 with the *hyponyms* for the word *dish*.   \n",
    "  \n",
    "Since we have covered everything needed to answer this question in this notebook, you will be doing most of the coding yourself. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**(TO DO) Q5 - 5 marks**   \n",
    "1) Go through the senses for the word '*dish*' (recall how we used the functions printBasicSynsetInfo() and wordnet.synsets() above). Looking through the senses, select the single most appropriate sense of the word *dish* for our problem (Restaurant Corpus, look into the corpus or file if you need more context on which words are labelled *Dish*). Collect a list of all *hyponyms* for the word '*dish*' based on the selected sense of the word. You can think of this as a Gazetteer.    \n",
    "2) Using thresholds 0, 1, and 2 with ner_ed, collect three  lists of words that we would tag as *Dish* based on the *Edit Distance* between our Dish Gazetteer and test_sentences. Print the returned lists.     \n",
    "3) Are more dishes being found with a larger threshold? Give an example to justify your answer.    \n",
    "4) Using thresholds 25% and 50% with ner_ed_rel, collect two lists of words that we would tag as *Dish* based on the *Edit Distance* between our Dish Gazetteer and test_sentences. Print the returned lists.    \n",
    "5) Does relative thresholding better capture variations than not using a relative thresholding for the tag *Dish*? Give an example to justify your answer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[Sense 0]\n",
      "SynLemmas\n",
      "[Lemma('dish.n.01.dish')]\n",
      "Synonyms\n",
      "['dish']\n",
      "Definition\n",
      "a piece of dishware normally used as a container for holding or serving food\n",
      "\n",
      "[Sense 1]\n",
      "SynLemmas\n",
      "[Lemma('dish.n.02.dish')]\n",
      "Synonyms\n",
      "['dish']\n",
      "Definition\n",
      "a particular item of prepared food\n",
      "\n",
      "[Sense 2]\n",
      "SynLemmas\n",
      "[Lemma('dish.n.03.dish'), Lemma('dish.n.03.dishful')]\n",
      "Synonyms\n",
      "['dish', 'dishful']\n",
      "Definition\n",
      "the quantity that a dish will hold\n",
      "\n",
      "[Sense 3]\n",
      "SynLemmas\n",
      "[Lemma('smasher.n.02.smasher'), Lemma('smasher.n.02.stunner'), Lemma('smasher.n.02.knockout'), Lemma('smasher.n.02.beauty'), Lemma('smasher.n.02.ravisher'), Lemma('smasher.n.02.sweetheart'), Lemma('smasher.n.02.peach'), Lemma('smasher.n.02.lulu'), Lemma('smasher.n.02.looker'), Lemma('smasher.n.02.mantrap'), Lemma('smasher.n.02.dish')]\n",
      "Synonyms\n",
      "['smasher', 'stunner', 'knockout', 'beauty', 'ravisher', 'sweetheart', 'peach', 'lulu', 'looker', 'mantrap', 'dish']\n",
      "Definition\n",
      "a very attractive or seductive looking woman\n",
      "\n",
      "[Sense 4]\n",
      "SynLemmas\n",
      "[Lemma('dish.n.05.dish'), Lemma('dish.n.05.dish_aerial'), Lemma('dish.n.05.dish_antenna'), Lemma('dish.n.05.saucer')]\n",
      "Synonyms\n",
      "['dish', 'dish_aerial', 'dish_antenna', 'saucer']\n",
      "Definition\n",
      "directional antenna consisting of a parabolic reflector for microwave or radio frequency radiation\n",
      "\n",
      "[Sense 5]\n",
      "SynLemmas\n",
      "[Lemma('cup_of_tea.n.01.cup_of_tea'), Lemma('cup_of_tea.n.01.bag'), Lemma('cup_of_tea.n.01.dish')]\n",
      "Synonyms\n",
      "['cup_of_tea', 'bag', 'dish']\n",
      "Definition\n",
      "an activity that you like or at which you are superior\n",
      "\n",
      "[Sense 6]\n",
      "SynLemmas\n",
      "[Lemma('serve.v.06.serve'), Lemma('serve.v.06.serve_up'), Lemma('serve.v.06.dish_out'), Lemma('serve.v.06.dish_up'), Lemma('serve.v.06.dish')]\n",
      "Synonyms\n",
      "['serve', 'serve_up', 'dish_out', 'dish_up', 'dish']\n",
      "Definition\n",
      "provide (usually but not necessarily food)\n",
      "\n",
      "[Sense 7]\n",
      "SynLemmas\n",
      "[Lemma('dish.v.02.dish')]\n",
      "Synonyms\n",
      "['dish']\n",
      "Definition\n",
      "make concave; shape like a dish\n",
      "\n",
      "Sense 1\n",
      "\n",
      "['adobo', 'applesauce', 'bacon_and_eggs', 'barbecue', 'barbecued_spareribs', 'barbecued_wing', 'beef_bourguignonne', 'beef_stroganoff', 'beef_wellington', 'biryani', 'bitok', 'boiled_dinner', 'boiled_egg', 'boston_baked_beans', 'bubble_and_squeak', 'buffalo_wing', 'burrito', 'carbonnade_flamande', 'casserole', 'chicken_and_rice', 'chicken_cordon_bleu', 'chicken_kiev', 'chicken_marengo', 'chicken_paprika', 'chicken_provencale', 'chili', 'chop_suey', 'chow_mein', 'coq_au_vin', 'coquille', 'coquilles_saint-jacques', 'cottage_pie', 'couscous', 'croquette', 'curry', 'custard', 'deviled_egg', 'dolmas', 'egg_roll', 'eggs_benedict', 'enchilada', 'escalope_de_veau_orloff', 'falafel', 'fish_and_chips', 'fish_stick', 'fondue', 'fondue', 'french_toast', 'fried_egg', 'fried_rice', 'frittata', 'frog_legs', 'galantine', 'gefilte_fish', 'haggis', 'ham_and_eggs', 'hash', 'jambalaya', 'kabob', 'kedgeree', 'kishke', 'lobster_thermidor', 'lutefisk', 'macedoine', 'maryland_chicken', 'meat_loaf', 'meatball', 'mold', 'moo_goo_gai_pan', 'moussaka', 'mousse', 'omelet', 'osso_buco', 'paella', 'pasta', 'patty', 'pepper_steak', 'pheasant_under_glass', 'piece_de_resistance', 'pilaf', 'pizza', 'poached_egg', 'poi', 'pork_and_beans', 'porridge', 'potpie', 'pudding', 'ramekin', 'refried_beans', 'rijsttaffel', 'risotto', 'rissole', 'roulade', 'salad', 'salisbury_steak', 'sandwich_plate', 'sashimi', 'sauerbraten', 'sauerkraut', 'saute', 'scallopine', 'scampi', 'schnitzel', 'scotch_egg', 'scotch_woodcock', 'scrambled_eggs', 'scrapple', 'seafood_newburg', 'shirred_egg', 'side_dish', 'snack_food', 'souffle', 'soup', 'spaghetti_and_meatballs', 'spanish_rice', 'special', 'steak_au_poivre', 'steak_tartare', 'stew', 'stuffed_cabbage', 'stuffed_peppers', 'stuffed_tomato', 'stuffed_tomato', 'succotash', 'sukiyaki', 'sushi', 'swiss_steak', 'taco', 'tamale', 'tamale_pie', 'tempura', 'teriyaki', 'terrine', 'tetrazzini', 'timbale', 'tostada', 'turnover', 'veal_cordon_bleu', 'veal_parmesan', 'viand', 'welsh_rarebit']\n"
     ]
    }
   ],
   "source": [
    "# TO DO (1)\n",
    "# Look through the senses of the word dish to select the most appropriate sense for our corpus\n",
    "# Recall how we used the printBasicSynsetInfo() function above\n",
    "dishSenses = wordnet.synsets('dish')\n",
    "for i in range(len(dishSenses)):\n",
    "    print(\"[Sense \" + str(i) + \"]\")\n",
    "    printBasicSynsetInfo(dishSenses[i])\n",
    "    print()\n",
    "\n",
    "# Print which sense you have selected (ex: Sense X)\n",
    "print(\"Sense 1\\n\")\n",
    "\n",
    "# Collect a list of all hyponyms for the word 'dish' only from the selected sense\n",
    "\n",
    "dish_sense1 = [hyponym.name().split(\".\")[0] for hyponym in dishSenses[1].hyponyms()]\n",
    "    \n",
    "# Print the Dish Gazetteer\n",
    "print(dish_sense1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Distance 0\n",
      "['barbecue/barbecue', 'burrito/burrito', 'pasta/pasta', 'pizza/pizza', 'salad/salad', 'soup/soup', 'special/special', 'sushi/sushi', 'taco/taco']\n",
      "Distance 1\n",
      "['barbecue/barbecue', 'barbeque/barbecue', 'burrito/burrito', 'carry/curry', 'child/chili', 'has/hash', 'meatballs/meatball', 'pasta/pasta', 'pizza/pizza', 'salad/salad', 'sauce/saute', 'soul/soup', 'soup/soup', 'special/special', 'specials/special', 'sushi/sushi', 'taco/taco', 'tacos/taco']\n",
      "Distance 2\n",
      "['and/viand', 'as/hash', 'back/taco', 'barbecue/barbecue', 'barbeque/barbecue', 'box/poi', 'burrito/burrito', 'carry/curry', 'casa/hash', 'chase/hash', 'chick/chili', 'child/chili', 'china/chili', 'could/mold', 'course/mousse', 'curb/curry', 'date/saute', 'dish/hash', 'do/poi', 'dog/poi', 'dollars/dolmas', 'east/hash', 'fast/hash', 'few/stew', 'find/viand', 'food/mold', 'for/poi', 'four/soup', 'gate/saute', 'go/poi', 'good/mold', 'hard/hash', 'has/hash', 'have/hash', 'hi/poi', 'high/hash', 'hot/poi', 'hour/soup', 'house/mousse', 'how/poi', 'i/poi', 'jack/taco', 'late/saute', 'los/poi', 'maid/mold', 'mall/mold', 'meatballs/meatball', 'mid/mold', 'mile/mold', 'milk/mold', 'more/mold', 'most/mold', 'new/stew', 'no/poi', 'non/poi', 'not/poi', 'now/poi', 'of/poi', 'oil/poi', 'on/poi', 'or/poi', 'our/soup', 'out/soup', 'pasta/pasta', 'per/poi', 'pesto/pasta', 'pet/poi', 'pf/poi', 'pho/poi', 'pie/poi', 'pizza/pizza', 'pm/poi', 'pork/poi', 'port/poi', 'prix/poi', 'pub/poi', 'road/mold', 'route/saute', 'salad/salad', 'salon/salad', 'sauce/saute', 'sea/stew', 'see/stew', 'shop/soup', 'site/saute', 'some/soup', 'soul/soup', 'soup/soup', 'special/special', 'specials/special', 'spot/poi', 'st/stew', 'stand/viand', 'star/stew', 'state/saute', 'steak/stew', 'stop/soup', 'sub/soup', 'sum/soup', 'sushi/sushi', 'table/tamale', 'taco/taco', 'tacos/taco', 'take/taco', 'tameles/tamale', 'tasty/pasta', 'ten/stew', 'tgi/poi', 'to/poi', 'too/poi', 'two/taco', 'up/soup', 'view/stew', 'wild/mold', 'wok/poi', 'wood/mold', 'would/mold', 'you/poi', 'zoo/poi']\n"
     ]
    }
   ],
   "source": [
    "# TO DO (2)\n",
    "# Using thresholds 0, 1, and 2 with ner_ed, collect three lists of words that we\n",
    "# would tag as *Dish* based on the *Edit Distance* between our Dish Gazetteer and test_sentences. \n",
    "dish0 = ner_ed(test_sentences, dish_sense1, threshold=0)\n",
    "dish1 = ner_ed(test_sentences, dish_sense1, threshold=1)\n",
    "dish2 = ner_ed(test_sentences, dish_sense1, threshold=2)\n",
    "print(\"Distance 0\")\n",
    "print(dish0)\n",
    "print(\"Distance 1\")\n",
    "print(dish1)\n",
    "print(\"Distance 2\")\n",
    "print(dish2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Q5 - TO DO (3)   \n",
    "Are more dishes being found with a larger threshold? Give an example to justify your answer.  \n",
    "\n",
    "By increasing thershold from 0 to 1 or 2, there are more dishes being found.\n",
    "ex) barbeque/barbecue, meatballs/meatball, tacos/taco  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Threshold 25%\n",
      "['barbecue/barbecue', 'burrito/burrito', 'pasta/pasta', 'pizza/pizza', 'salad/salad', 'soup/soup', 'special/special', 'sushi/sushi', 'taco/taco']\n",
      "Threshold 50%\n",
      "['barbecue/barbecue', 'burrito/burrito', 'pasta/pasta', 'pizza/pizza', 'salad/salad', 'soup/soup', 'special/special', 'sushi/sushi', 'taco/taco']\n"
     ]
    }
   ],
   "source": [
    "# TO DO (4)\n",
    "# Using thresholds 25% and 50% with ner_ed_rel, collect two lists of words that we would tag \n",
    "# as *Dish* based on the *Edit Distance* between our Dish Gazetteer and test_sentences.     \n",
    "dish25 = ner_ed(test_sentences, dish_sense1, threshold=0.25)\n",
    "dish50 = ner_ed(test_sentences, dish_sense1, threshold=0.5)\n",
    "print(\"Threshold 25%\")\n",
    "print(dish25)\n",
    "print(\"Threshold 50%\")\n",
    "print(dish50)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Q5 - TO DO (5)    \n",
    "Does relative thresholding better capture variations than not using a relative thresholding for the tag *Dish*? Give an example to justify your answer.     \n",
    "\n",
    "No. For the tag 'Dish', there are words that point to the same food but only slightly different spellings like in Q5-3(barbeque/barbecue, meatballs/meatball, tacos/taco). Normal thresholding could catch those words while increasing threshold, but relative thresholding couldn't capture them even though increasing threshold.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**This is the last notebook, hope you enjoyed them and learned some new things :) Best of luck with your projects!**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Signature\n",
    "\n",
    "I, SUZIE OH, declare that the answers provided in this notebook are my own."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}