example_of_intent_and_entity_classification_with_NLU_engine_class.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from nlu_engine import NLUEngine\n",
    "from nlu_engine import DataUtils\n",
    "from nlu_engine import IntentMatcher, LR\n",
    "from nlu_engine import EntityExtractor\n",
    "import nltk"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Example of intent and entity classification with NLU engine class\n",
    "This is just a small example notebook to help users understand how to use the NLU engine.\n",
    "\n",
    "* Intent example\n",
    "* Entity example"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Load data set. For this example, we will use the cleaned dataset, although you can load any dataset you like."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "nlu_data_df = DataUtils.load_data(\n",
    "    'data/NLU-Data-Home-Domain-Annotated-All-Cleaned.csv'\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Intent classification: example of a single utterance"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Both the intents and the domains (scenarios/skills) can be used to label an utterance. In this example we will use domains to label the utterances' intents. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#TODO: refactor\n",
    "domains = nlu_data_df.scenario.values\n",
    "\n",
    "LR_domain_classifier_model, tfidf_vectorizer = NLUEngine.train_intent_classifier(\n",
    "    data_df_path=nlu_data_df,\n",
    "    labels_to_predict='scenario',\n",
    "    classifier=LR\n",
    ")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "intent = nlu_data_df.intent.values\n",
    "\n",
    "LR_domain_classifier_model, tfidf_vectorizer = NLUEngine.train_intent_classifier(\n",
    "    data_df_path=nlu_data_df,\n",
    "    labels_to_predict='intent',\n",
    "    classifier=LR\n",
    ")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Example: Let's try to predict an utterances intent label using the domains."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "utterance = \"turn off the kitchen lights\"\n",
    "\n",
    "print(IntentMatcher.predict_label(\n",
    "    LR_domain_classifier_model, tfidf_vectorizer, utterance))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Entity extraction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The entity extraction could be greatly improved by improving the features it uses. It would be great if someone would take a look at this. Perhaps the CRF features similar to what Snips uses would be better such as Brown clustering (probably)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is important to have the NLTK tokenizer to be able to extract entities."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "try:\n",
    "    nltk.data.find('tokenizers/punkt')\n",
    "except LookupError:\n",
    "    nltk.download('punkt')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example: Extracting entities from an utterance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "crf_model = NLUEngine.train_entity_classifier(data_df=nlu_data_df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "utterance = 'wake me up at five pm this week'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can get the entity tags of a specific utterance with the EntityExtractor.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "EntityExtractor.get_entity_tags(utterance, crf_model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also get the entity tagged utterance with the NLUEngine.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "entity_tagged_utterance = NLUEngine.create_entity_tagged_utterance(\n",
    "    utterance, crf_model)\n",
    "\n",
    "entity_tagged_utterance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#TODO remove everything from here (perhaps move it into another notebook?), this was just to quickly evaluate entity matching using spaCy for PoS."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import spacy\n",
    "\n",
    "nlp = spacy.load(\"en_core_web_sm\")\n",
    "doc = nlp(\"Get busy living or get busy dying.\")\n",
    "\n",
    "print(f\"{'text':{8}} {'POS':{6}} {'TAG':{6}} {'Dep':{6}} {'POS explained':{20}} {'tag explained'} \")\n",
    "for token in doc:\n",
    "    print(f'{token.text:{8}} {token.pos_:{6}} {token.tag_:{6}} {token.dep_:{6}} {spacy.explain(token.pos_):{20}} {spacy.explain(token.tag_)}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "list_of_words_and_tags = []\n",
    "for token in doc:\n",
    "    list_of_words_and_tags.append((token.text, token.tag_))\n",
    "\n",
    "list_of_words_and_tags\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "EntityExtractor.pos_tag_utterance(\n",
    "    utterance=\"Get busy living or get busy dying.\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "entity_reviewed_report_df = NLUEngine.evaluate_entity_classifier(\n",
    "    data_df=nlu_data_df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "entity_reviewed_report_df.to_csv('data/nltk_pos_entity_report.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "entity_reviewed_report_df\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from nlu_engine import Analytics\n",
    "from nlu_engine.entity_extractor import crf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import spacy\n",
    "\n",
    "nlp = spacy.load(\"en_core_web_sm\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def spacy_pos_tag_utterance(utterance):\n",
    "    doc = nlp(utterance)\n",
    "    list_of_words_and_tags = []\n",
    "    for token in doc:\n",
    "        list_of_words_and_tags.append((token.text, token.tag_))\n",
    "    return list_of_words_and_tags\n",
    "\n",
    "def create_feature_dataset(data_df):\n",
    "    \"\"\"\n",
    "    Creates a feature dataset from the annotated utterances.\n",
    "    \"\"\"\n",
    "    feature_dataset = []\n",
    "    for utterance, utterance_with_tagging in zip(data_df['answer_normalised'], data_df['answer_annotation']):\n",
    "        entities = EntityExtractor.extract_entities(utterance_with_tagging)\n",
    "        utterance_pos = spacy_pos_tag_utterance(utterance)\n",
    "        feature_dataset.append(\n",
    "            EntityExtractor.combine_pos_and_entity_tags(entities, utterance_pos))\n",
    "    return feature_dataset\n",
    "\n",
    "def get_targets_and_labels(data_df):\n",
    "    feature_dataset = create_feature_dataset(data_df)\n",
    "    X = [EntityExtractor.utterance2features(utterance)\n",
    "            for utterance in feature_dataset]\n",
    "    y = [EntityExtractor.utterance2labels(utterance)\n",
    "            for utterance in feature_dataset]\n",
    "\n",
    "    return X, y\n",
    "\n",
    "def evaluate_entity_classifier(data_df):\n",
    "    \"\"\"\n",
    "    Evaluates the entity classifier and generates a report\n",
    "    \"\"\"\n",
    "\n",
    "    print('Evaluating entity classifier')\n",
    "\n",
    "    X, y = get_targets_and_labels(data_df)\n",
    "    predictions = Analytics.cross_validate_classifier(crf, X, y)\n",
    "    report_df = Analytics.generate_entity_classification_report(\n",
    "        predictions, y)\n",
    "    return report_df\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "entity_spacy_report_df = evaluate_entity_classifier(nlu_data_df)\n",
    "entity_spacy_report_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "entity_spacy_report_df.to_csv('data/spacy_entity_report.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "f1ecf1ecc6a840da86e8b827c66035ad900dc97d6a10e234826dd106c37257af"
  },
  "kernelspec": {
   "display_name": "Python 3.7.7 ('.venv': venv)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}