07. Learning Text Representations/7.01 Understanding Word2vec Model.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Understanding Word2vec model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Word2vec is one of the most popular and widely used models for generating the word\n",
    "embeddings. What are word embeddings though? Word embeddings are the vector\n",
    "representations of words in a vector space. The embedding generated by the word2vec\n",
    "model captures the syntactic and semantic meanings of a word. Having a meaningful\n",
    "vector representation of a word helps the neural network to understand the word better.\n",
    "\n",
    "For instance, let us consider the following text:\n",
    "\n",
    "  _Archie used to live in New York, he then moved to Santa Clara. He loves apples and strawberries._\n",
    "\n",
    "\n",
    "Word2vec model generates the vector representation for each of the words in the above\n",
    "text. If we project and visualize the vectors in embedding space, we can see how all the\n",
    "similar words are plotted close together.\n",
    "As you can see in the below figure, words apples and strawberries are plotted close\n",
    "together, and New York and Santa Clara are plotted close together. They are plotted close\n",
    "together because the word2vec model has learned that apples and strawberries are similar\n",
    "entities i.e fruits, New York and Santa Clara are similar entities i.e cities and so their vectors\n",
    "(embeddings) are similar to each other and which is why the distance between them is less. \n",
    "\n",
    "![image](images/1.png)\n",
    "\n",
    "Thus, with word2vec, we can learn the meaningful vector representation of a word which\n",
    "helps the neural networks to understand what the word is about. Having a good\n",
    "representation of a text would be useful in various tasks. Since our network can understand\n",
    "the contextual and syntactic meaning of words, this will branch out to various use cases\n",
    "such as text summarization, sentiment analysis, text generation and more.\n",
    "\n",
    "\n",
    "Okay. But how do the word2vec model learns the word embeddings? There are two types\n",
    "of word2vec model for learning the embeddings of a word,\n",
    "1. CBOW (Continous Bag of Words)\n",
    "2. Skip-gram\n",
    "We will go into detail and learn how each of these models learns the vector representations\n",
    "of a word. "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [default]",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}