NLP -Spacy Custom Entity Ruler.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# NLP- Spacy Custom Entity ruler  Model to find Entity "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Rule-based matching"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Load All lib"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import spacy\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Spacy Default Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "#pip install en_core_web_sm"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "# download default spacy model using\n",
    "python -m spacy download en_core_web_sm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Load default spacy model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "nlp = spacy.load(\"en_core_web_sm\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "pass any string"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "string=\"India is a countrty and state is RAJASTHAN\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "pass string to obj nlp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "nlp_data=nlp(string)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Extract entity"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "GPE is:- India\n"
     ]
    }
   ],
   "source": [
    "for ent in nlp_data.ents:\n",
    "    word,explain=ent.text,ent.label_\n",
    "    print(explain,\"is:-\", word)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "** unable to get state in india whic is rajasthan \n",
    "*** for this we train our model , and make custom model .."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Spacy Custom Model "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Load Data of States (state name )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "state_data=['MAHARASHTRA', 'MEGHALAYA', 'JHARKHAND', 'MANIPUR', 'UTTARANCHAL', 'MIZORAM', 'JAMMU & KASHMIR', 'ANDAMAN NICOBAR', 'BIHAR', 'SIKKIM', 'HIMACHAL PRADESH', 'ARUNACHAL PRADESH', 'CHANDIGARG', 'ASSAM', 'CHHATISGARH', 'DELHI', 'ANDHRA PRADESH', 'KARNATAKA', 'APS CIRCLE', 'TRIPURA', 'ARMY POSTAL SERVICE', 'NAGALAND', 'KERALA', 'RAJASTHAN', 'WEST BENGAL', 'GOA', 'HARYANA', 'TAMIL NADU', 'GUJARAT', 'CHANDIGARH', 'ORISSA', 'UTTAR PRADESH', 'PUNJAB', 'MADHYA PRADESH']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Load data of Pin Code of india (pincode)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "pincode_data=['917504', '524289', '524290', '524291', '524292', '524293', '524294', '524295', '524296', '524297', '524298', '524299', '917511', '524301', '524302', '524303', '524304', '524305', '524306', '524307', '524308', '524309', '524310', '524311', '524312', '524313', '524314', '524315', '524316', '131101']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Load a blank model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "nlp=spacy.blank(\"en\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "create pipe "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "ruler = nlp.create_pipe(\"entity_ruler\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And add pattern to ruler"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "ruler.add_patterns([{\"label\": \"STATE\", \"pattern\": state} for state in state_data  ])\n",
    "ruler.add_patterns([{\"label\": \"PINCODE\", \"pattern\": pincode} for pincode in pincode_data])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Add Pipe to nlp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "nlp.add_pipe(ruler)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Save model "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "nlp.to_disk(\"entity_ruler\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Load Model "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "nlp=spacy.load(\"entity_ruler\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Check Model working"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "String"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "string=\"Tej .Parkash . JANAKIRAMAN AHRPR5043C NEW NO.  32,  3RD MAIN ROAD, 75 FLAT NO 7,2ND FLOOR, R.A PURAM RAJASTHAN \\nINDIA\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Pass \"String\" to model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_nlp=nlp(string)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Get Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "PINCODE is:- RAJASTHAN\n"
     ]
    }
   ],
   "source": [
    "for ent in data_nlp.ents:\n",
    "    word,explain=ent.text,ent.label_\n",
    "    print(explain,\"is:-\", word)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**  Now we Have rajasthan as state"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*****"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}