pincelate-tutorial-and-cookbook.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Pincelate tutorial and cookbook\n",
    "\n",
    "By [Allison Parrish](http://www.decontextualize.com/)\n",
    "\n",
    "This notebook shows you how to use [Pincelate](https://pincelate.readthedocs.io/) and how to do some interesting things with it.\n",
    "\n",
    "Pincelate is a Python library that provides a simple interface for a machine learning model that can sound out English words and spell English words based on how they sound. \"Sounding out\" here means converting letters (\"orthography\") to sounds (\"phonemes\"), and \"spelling\" means converting sounds to letters (phonemes to orthography). The model is trained on the [CMU Pronouncing Dictionary](http://www.speech.cs.cmu.edu/cgi-bin/cmudict), which means it generally sounds words out as though speaking \"standard\" North American English, and spells words according to \"standard\" North American English rules (at least as far as the model itself is accurate).\n",
    "\n",
    "## Preliminaries\n",
    "\n",
    "Loading various required modules, plus the language model and Pincelate. To run these experiments, you'll need to install Pincelate. Type the following at a command prompt:\n",
    "\n",
    "    pip install tensorflow  # or tensorflow-gpu\n",
    "    pip install pincelate\n",
    "    \n",
    "(Installing Pincelate will also install Pronouncing, which we'll use at various points in the experiments below.)\n",
    "\n",
    "Other libraries you'll need for this notebook: `numpy` and `scipy`. If you're using Anaconda, you already have these libraries. If not, install them like so:\n",
    "\n",
    "    pip install numpy scipy\n",
    "    \n",
    "Importing numpy and Pronouncing:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pronouncing as pr"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now import Pincelate and instantiate a Pincelate object. (This will load the pre-trained model provided with the package.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "from pincelate import Pincelate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "pin = Pincelate()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Later in the notebook, I'm going to use some of Jupyter Notebook's interactive features, so I'll import the libraries here:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "import ipywidgets as widgets\n",
    "from IPython.display import display\n",
    "from ipywidgets import interact, interactive_output, Layout, HBox, VBox"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Sounding out and spelling\n",
    "\n",
    "The CMU Pronouncing Dictionary provides a database of tens of thousands of English words along with their pronunciations. I made a Python library called [Pronouncing](https://github.com/aparrish/pronouncingpy) to make it easier to look up words in dictionary. Here's how it works. To get the pronunciation of a word:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'AE1 L F AH0 B EH2 T'"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pr.phones_for_word(\"alphabet\")[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The CMU Pronouncing Dictionary provides pronunciations as a list of phonemes in a phonetic transcription scheme called [Arpabet](https://en.wikipedia.org/wiki/ARPABET), in which each unique sound in English is given a different symbol.\n",
    "\n",
    "If you want to find words that have a particular pronunciation, you can look them up in the CMU Pronouncing Dictionary like so:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['flour', 'flower']"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pr.search(\"^F L AW1 ER0$\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That all seems pretty straightforward! The problem arises when you want to spell a word that *isn't* in the CMU Pronouncing Dictionary. You'll get an error:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "ename": "IndexError",
     "evalue": "list index out of range",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mIndexError\u001b[0m                                Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-7-4aa47dd2de94>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mpr\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mphones_for_word\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"mimsy\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mIndexError\u001b[0m: list index out of range"
     ]
    }
   ],
   "source": [
    "pr.phones_for_word(\"mimsy\")[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Likewise, if you've just invented a new word and have a pronunciation in mind, the CMU Pronouncing Dictionary won't be able to help you spell it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pr.search(\"^B L AH1 R F$\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is where Pincelate comes in handy. Pincelate's machine learning model can provide phonemes for words that aren't in the CMU Pronouncing Dictionary, and produce plausible spellings of arbitrary sequences of phonemes. To sound out a word, use the `.soundout()` method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['M', 'IH1', 'M', 'S', 'IY0']"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pin.soundout(\"mimsy\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "... and to produce a plausible spelling for a word whose sounds you just made up, use the `.spell()` method, passing it a list of Arpabet phonemes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'blurf'"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pin.spell(['B', 'L', 'AH1', 'R', 'F'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It's important to note that Pincelate's `.soundout()` method will *only* work with letters that appear the CMU Pronouncing Dictionary's vocabulary. (You need to use lowercase letters only.) So the following will throw an error:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "ename": "KeyError",
     "evalue": "'é'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mKeyError\u001b[0m                                  Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-11-0edf2522d617>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mpin\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mspell\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"étui\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m~/Dropbox/projects/workshops-and-talks/pycon-2020/tutorial/env/lib/python3.7/site-packages/pincelate/__init__.py\u001b[0m in \u001b[0;36mspell\u001b[0;34m(self, phones, temperature)\u001b[0m\n\u001b[1;32m    130\u001b[0m                 \u001b[0mthis_feat\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mphone_feature_map\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mitem\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m-\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    131\u001b[0m             \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 132\u001b[0;31m                 \u001b[0mthis_feat\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mphone_feature_map\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mitem\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    133\u001b[0m             \u001b[0mfeats\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mthis_feat\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    134\u001b[0m         state_val = self.phon2orth.infer(\n",
      "\u001b[0;31mKeyError\u001b[0m: 'é'"
     ]
    }
   ],
   "source": [
    "pin.spell(\"étui\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example: phoneme frequency analysis\n",
    "\n",
    "Using Pincelate's model, we can do phonetic analysis on texts, even texts that contain words that aren't in the CMU Pronouncing Dictionary. For example, let's find out what the most common phonemes are in Lewis Carroll's \"Jabberwocky,\" whose text I've included in this repository as `jabberwocky.txt`. Here's the full text:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "'Twas brillig, and the slithy toves\n",
      "      Did gyre and gimble in the wabe:\n",
      "All mimsy were the borogoves,\n",
      "      And the mome raths outgrabe.\n",
      "\n",
      "\"Beware the Jabberwock, my son!\n",
      "      The jaws that bite, the claws that catch!\n",
      "Beware the Jubjub bird, and shun\n",
      "      The frumious Bandersnatch!\"\n",
      "\n",
      "He took his vorpal sword in hand;\n",
      "      Long time the manxome foe he sought---\n",
      "So rested he by the Tumtum tree\n",
      "      And stood awhile in thought.\n",
      "\n",
      "And, as in uffish thought he stood,\n",
      "      The Jabberwock, with eyes of flame,\n",
      "Came whiffling through the tulgey wood,\n",
      "      And burbled as it came!\n",
      "\n",
      "One, two! One, two! And through and through\n",
      "      The vorpal blade went snicker-snack!\n",
      "He left it dead, and with its head\n",
      "      He went galumphing back.\n",
      "\n",
      "\"And hast thou slain the Jabberwock?\n",
      "      Come to my arms, my beamish boy!\n",
      "O frabjous day! Callooh! Callay!\"\n",
      "      He chortled in his joy.\n",
      "\n",
      "'Twas brillig, and the slithy toves\n",
      "      Did gyre and gimble in the wabe:\n",
      "All mimsy were the borogoves,\n",
      "      And the mome raths outgrabe.\n"
     ]
    }
   ],
   "source": [
    "text = open(\"jabberwocky.txt\").read()\n",
    "print(text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, parse the text into words and convert them to lower case:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "import re\n",
    "words = [item.lower() for item in re.findall(r\"\\b(\\w+)\\b\", text)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here's a random sample of the words just to ensure that we've got what we wanted:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['the', 'claws', 'that', 'the', 'the', 'toves', 'foe', 'the', 'did', 'gyre']"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import random\n",
    "random.sample(words, 10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we'll use `.soundout()` to get a list of phonemes for each item, and feed them to a `Counter()` object:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "from collections import Counter\n",
    "phoneme_count = Counter()\n",
    "for word in words:\n",
    "    phoneme_count.update(pin.soundout(word))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And now print out the most common phonemes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('D', 37),\n",
       " ('AE1', 33),\n",
       " ('T', 32),\n",
       " ('N', 32),\n",
       " ('B', 28),\n",
       " ('IY1', 28),\n",
       " ('M', 26),\n",
       " ('R', 25),\n",
       " ('IH1', 25),\n",
       " ('L', 24),\n",
       " ('S', 22),\n",
       " ('W', 20)]"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "phoneme_count.most_common(12)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So a lot of alveolar sounds (`D`, `T`, `N`) and vowels at the front of the mouth (`AE1`, `IY1`, `IH1`). To know if this distribution is unusual, we could compare it with the most common phonemes in all of the CMU Pronouncing Dictionary:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('AH0', 63136),\n",
       " ('N', 61235),\n",
       " ('S', 50435),\n",
       " ('L', 49964),\n",
       " ('T', 49075),\n",
       " ('R', 46469),\n",
       " ('K', 43079),\n",
       " ('D', 32559),\n",
       " ('IH0', 30200),\n",
       " ('M', 29741),\n",
       " ('Z', 28217),\n",
       " ('ER0', 23954)]"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cmu_phoneme_count = Counter()\n",
    "for word, phones in pr.pronunciations:\n",
    "    cmu_phoneme_count.update(phones.split())\n",
    "cmu_phoneme_count.most_common(12)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We could do a more formal analysis and make claims about how Lewis Carroll's *Jabberwocky* differs significantly from typical English from a phonetic standpoint, but just from a quick look we can see that \"Jabberwocky\" is heavy on the `AE1`s (i.e., the vowel sound in \"hand\") and `B`s."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example: Spelling words from random phonemes\n",
    "\n",
    "Having just counted up all of the phonemes in the CMU Pronouncing Dictionary, we can now invent somewhat plausible neologisms by drawing phonemes at random according to their frequency and gluing them together. (\"Neologism\" is a fancy word for \"made-up word.\") The following code normalizes the phoneme frequencies so we can use them in numpy's `np.random.choice` function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "all_phonemes = list(cmu_phoneme_count.keys())\n",
    "phoneme_frequencies = np.array(list(cmu_phoneme_count.values()), dtype=np.float32)\n",
    "phoneme_frequencies /= phoneme_frequencies.sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And then this function will return a random neologism, created from phonemes drawn at random based on their frequency in English words:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "def neologism_phonemes():\n",
    "    return [np.random.choice(all_phonemes, p=phoneme_frequencies)\n",
    "            for item in range(random.randrange(3,10))]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here's a handful, just to get a taste:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['P', 'R', 'N', 'K', 'N', 'AO1']\n",
      "['P', 'N', 'OW2', 'N', 'N']\n",
      "['IY1', 'R', 'AE1', 'T', 'L', 'K', 'AA1', 'UW2']\n",
      "['Z', 'Z', 'AH0', 'AH0', 'AH0', 'S', 'IH0', 'IY0', 'AH1']\n",
      "['L', 'AA1', 'IH0', 'S', 'IH1', 'R', 'AH0', 'UW1']\n"
     ]
    }
   ],
   "source": [
    "for i in range(5):\n",
    "    print(neologism_phonemes())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That's all well and good! Try sounding out some of these on your own (consult the [Arpabet](https://en.wikipedia.org/wiki/ARPABET) table to find the English sound corresponding to each symbol).\n",
    "\n",
    "But how do you *spell* these neologisms? Why, with Pincelate's `.spell()` method of course:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'hmow'"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pin.spell(neologism_phonemes())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here's a for loop that generates neologisms and prints them along with their spellings:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "hado ['EH1', 'AO1', 'D']\n",
      "embervadera ['IY0', 'M', 'B', 'V', 'ER0', 'AH0', 'D', 'ER0', 'AE1']\n",
      "tareltpent ['T', 'AA1', 'R', 'EH1', 'L', 'T', 'P', 'T', 'N']\n",
      "tlinley ['T', 'N', 'N', 'L', 'IY0']\n",
      "urrair ['ER1', 'EH1', 'R']\n",
      "arpgleck ['AH0', 'R', 'P', 'G', 'AH0', 'L', 'K']\n",
      "dubster ['D', 'HH', 'B', 'AH0', 'S', 'R', 'T', 'IH0']\n",
      "klunglawough ['K', 'T', 'N', 'K', 'L', 'W', 'AH0', 'UW2', 'AO1']\n",
      "tterdeeker ['T', 'S', 'D', 'ER0', 'Y', 'K', 'R']\n",
      "snon ['N', 'S', 'AA2', 'N']\n",
      "mfur ['M', 'F', 'ER0']\n",
      "pae ['P', 'AH0', 'EH2']\n"
     ]
    }
   ],
   "source": [
    "for i in range(12):\n",
    "    phonemes = neologism_phonemes()\n",
    "    print(pin.spell(phonemes), phonemes)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Phoneme features\n",
    "\n",
    "The examples above use the phoneme as the basic unit of English phonetics. But each phoneme itself has characteristics, and many phonemes have characteristics in common. For example, the phoneme `/B/` has the following characteristics:\n",
    "\n",
    "* *bilabial*: you put your lips together when you say it\n",
    "* *stop*: airflow from the lungs is completely obstructed\n",
    "* *voiced*: your vocal cords are vibrating while you say it\n",
    "\n",
    "The phoneme `/P/` shares two out of three of these characteristics (it's *bilabial* and a *stop*, but is not voiced). The phoneme `/AE/`, on the other hand, shares *none* of these characteristics. Instead, it has these characteristics:\n",
    "\n",
    "* *vowel*: your mouth doesn't stop or occlude airflow when making this sound\n",
    "* *low*: your tongue is low in the mouth\n",
    "* *front*: your tongue is advanced forward in the mouth\n",
    "* *unrounded*: your lips are not rounded\n",
    "\n",
    "These characteristics of phonemes are traditionally called \"features.\" You can look up the features for particular phonemes using the `phone_feature_map` variable in Pincelate's `featurephone` module:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pincelate.featurephone import phone_feature_map"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For example, to get the features for the vowel `/UW/` (vowel sound in \"toot\"):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('hgh', 'bck', 'rnd', 'vwl')"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "phone_feature_map['UW']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The features are referred to here with short three-letter abbreviations. Here's a full list:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* `alv`: alveolar\n",
    "* `apr`: approximant\n",
    "* `bck`: back\n",
    "* `blb`: bilabial\n",
    "* `cnt`: central\n",
    "* `dnt`: dental\n",
    "* `fnt`: front\n",
    "* `frc`: fricative\n",
    "* `glt`: glottal\n",
    "* `hgh`: high\n",
    "* `lat`: lateral\n",
    "* `lbd`: labiodental\n",
    "* `lbv`: labiovelar\n",
    "* `lmd`: low-mid\n",
    "* `low`: low\n",
    "* `mid`: mid\n",
    "* `nas`: nasal\n",
    "* `pal`: palatal\n",
    "* `pla`: palato-alveolar\n",
    "* `rnd`: rounded\n",
    "* `rzd`: rhoticized\n",
    "* `smh`: semi-high\n",
    "* `stp`: stop\n",
    "* `umd`: upper-mid\n",
    "* `unr`: unrounded\n",
    "* `vcd`: voiced\n",
    "* `vel`: velar\n",
    "* `vls`: voiceless\n",
    "* `vwl`: vowel\n",
    "\n",
    "Additionally, there are two special phoneme features:\n",
    "\n",
    "* `beg`: beginning of word\n",
    "* `end`: end of word\n",
    "\n",
    "... which are found and the beginnings and endings of words."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Internally, Pincelate's model operates on these *phoneme features*, instead of directly on whole phonemes. This allows the model to capture and predict underlying similarities between phonemes.\n",
    "\n",
    "Pincelate's `.phonemefeatures()` method works a lot like `.spell()`, except instead of returning a list of phonemes, it returns a [numpy](https://numpy.org/) array of *phoneme feature probabilities*. This array has one row for each predicted phoneme, and one column for the probability (between 0 and 1) of a phoneme feature being a component of each phoneme. To illustrate, here I get the feature array for the word `cat`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "cat_feats = pin.phonemefeatures(\"cat\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This array has the following shape:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(5, 32)"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cat_feats.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "... which tells us that there are five predicted phonemes. (The `32` is the total number of possible features.) The word `cat`, of course, has only three phonemes (`/K AE T/`)—the extra two are the special \"beginning of the word\" and \"end of the word\" phonemes at the beginning and end, respectively.\n",
    "\n",
    "### Examining predicted phoneme features\n",
    "\n",
    "Let's look at the feature probabilities for the first phoneme (after the special \"beginning of the word\" token at index 0):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1.09146349e-03, 2.64442605e-07, 1.21863934e-07, 4.83202034e-10,\n",
       "       9.05446296e-09, 1.22701867e-05, 3.87583965e-09, 2.57467754e-08,\n",
       "       3.10348078e-05, 1.12015674e-04, 1.80950082e-07, 1.23362767e-07,\n",
       "       2.53906479e-10, 7.39023420e-09, 7.97239996e-10, 2.34866529e-05,\n",
       "       1.97903224e-04, 2.42027836e-05, 1.02661081e-10, 3.20427802e-07,\n",
       "       2.47776796e-07, 2.94954656e-08, 8.40330561e-09, 3.23472051e-07,\n",
       "       9.99906063e-01, 6.39497084e-05, 1.69826153e-09, 2.14709959e-04,\n",
       "       1.71634947e-05, 9.99885082e-01, 9.99977589e-01, 1.45659375e-04])"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cat_feats[1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can look up the index in this array associated with a particular phoneme feature using Pincelate's `.featureidx()` method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.999885082244873"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cat_feats[1][pin.featureidx('vel')]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This tells us that the `vel` (velar) feature for this phoneme is predicted with almost 100% probability—which makes sense, since the phoneme we'd anticipate—`/K/` is a voiceless velar stop."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example: Spelling from phoneme features\n",
    "\n",
    "The Pincelate class has another method, `.spellfeatures()`, which works like `.spell()` except it takes an array of phoneme features (such as that returned from `.phonemefeatures()`) instead of a list of Arpabet phonemes. You can use this to re-spell phoneme feature arrays that you have manipulated. In the following cell, I get the phoneme feature probability array for the word `pug`, then overwrite the probability of the \"voiced\" feature for its first phoneme, then respell:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'bug'"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pug = pin.phonemefeatures(\"pug\")\n",
    "pug[1][pin.featureidx('vcd')] = 1\n",
    "pin.spellfeatures(pug)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or, you can spell from completely random feature probabilities:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "fhuolesiiw\n",
      "whoilliewsh\n",
      "hwhultingiw\n",
      "hhholtioshow\n",
      "kkhollishowigh\n",
      "kholthiechoog\n",
      "khwallishiowe\n",
      "klwolsioskoo\n",
      "klolsinshoog\n",
      "khullsiowivey\n",
      "kwhalekishooe\n",
      "khwolkiewschoo\n"
     ]
    }
   ],
   "source": [
    "for i in range(12):\n",
    "    print(pin.spellfeatures(np.random.uniform(0, 1, size=(12,32))))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "... which (weirdly) seems like someone trying to imitate the sound of white noise.\n",
    "\n",
    "Or, you might want to build up neologism from scratch, specifying their phoneme features by hand. To do this, use Pincelate's `.vectorizefeatures()` method, passing it an array of tuples of phoneme features. It returns a phoneme probability array that you can then send to `.spellfeatures()`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'bie'"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bee = pin.vectorizefeatures([\n",
    "    ['beg'], ['blb', 'stp', 'vcd'], ['hgh', 'fnt', 'vwl'], ['end']\n",
    "])\n",
    "pin.spellfeatures(bee)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example: Resizing feature probability arrays\n",
    "\n",
    "Once you have the phonetic feature probability arrays, you can treat them the same way you'd treat any other numpy array. One thing I like to do is use scipy's image manipulation functions and use them resample the phonetic feature arrays. This lets us use the same phonetic information to spell a shorter or longer word. In particular, `scipy.ndimage.interpolation` has a handy [zoom](https://docs.scipy.org/doc/scipy-0.15.1/reference/generated/scipy.ndimage.interpolation.zoom.html) function that resamples an array and interpolates it. Normally you'd use this to resize an image, but nothing's stopping us from using it to resize our phonetic feature array.\n",
    "\n",
    "First, import the function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "from scipy.ndimage.interpolation import zoom"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then get some phoneme feature probabilities:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "feats = pin.phonemefeatures(\"alphabet\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then resize with `zoom()`. The second parameter to `zoom()` is a tuple with the factor by which to scale the dimensions of the incoming array. We only want to scale along the first axis (i.e., the phonemes), keeping the second axis (i.e., the features) constant.\n",
    "\n",
    "A shorter version of the word:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'albe'"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "shorter = zoom(feats, (0.67, 1))\n",
    "pin.spellfeatures(shorter)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A longer version:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'all-phabeate'"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "longer = zoom(feats, (2.0, 1))\n",
    "pin.spellfeatures(longer)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you've downloaded this notebook and you're following along running the code, the following cell will create an interactive widget that lets you \"stretch\" and \"shrink\" the words that you type into the text box by dragging the slider."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "5b696cfcf7cf482e90aa33698b29d049",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "interactive(children=(Text(value='how to spell expressively', description='words'), FloatSlider(value=1.0, des…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "@interact(words=\"how to spell expressively\", factor=(0.1, 4.0, 0.1))\n",
    "def stretchy(words, factor=1.0):\n",
    "    out = []\n",
    "    for word in words.split():\n",
    "        word = word.lower()\n",
    "        vec = pin.phonemefeatures(word)\n",
    "        if factor < 1.0:\n",
    "            order = 3\n",
    "        else:\n",
    "            order = 0\n",
    "        zoomed = zoom(vec, (factor, 1), order=order)\n",
    "        out.append(pin.spellfeatures(zoomed))\n",
    "    print(\" \".join(out))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Round-trip spelling manipulation\n",
    "\n",
    "Pincelate actually consists of *two* models: one that knows how to sound out words based on how they're spelled, , and another that knows how to spell words from sounds. Pincelate's `.manipulate()` function does a \"round trip\" re-spelling of a word, passing it through both models to return back to the original word. Try it out:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'spelling'"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pin.manipulate(\"spelling\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "On the surface, this isn't very interesting! You don't need Pincelate to tell you how to spell a word that you already know how to spell. But the `.manipulate()` has a handful of parameters that allow you to mess around with the model's internal workings in fun and interesting ways. The first is the `temperature` parameter, which artificially increases or decreases the amount of randomness in the model's output probabilities.\n",
    "\n",
    "### Spelling temperature\n",
    "\n",
    "When the temperature is close to zero, the model will always pick the most likely spelling of the word at each step."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'spelling'"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pin.manipulate(\"spelling\", temperature=0.01)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you increase the temperature to 1.0, the model starts picking values at random according to the underlying probabilities."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'spelling'"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pin.manipulate(\"spelling\", temperature=1.0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "At temperatures above 1.0, the model has a higher chance of picking from letters with lower probabilities, producing a more unlikely spelling:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'spelliz'"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pin.manipulate(\"spelling\", temperature=1.5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "At a high enough temperature, the model's spelling feels essentially random:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'la-zwaslf'"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pin.manipulate(\"spelling\", temperature=3.0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following interactive widget lets you play with the `temperature` parameter:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 163,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "11f8af6298f94fd3802cff4df78be91c",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "interactive(children=(Text(value='getting hot in here', description='s'), FloatSlider(value=1.2500000000000002…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "@interact(s=\"getting hot in here\", temp=(0.05, 2.5, 0.05))\n",
    "def tempadjust(s, temp):\n",
    "    return ' '.join([pin.manipulate(w.lower(), temperature=temp) for w in s.split()])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example: Manipulating letter frequencies\n",
    "\n",
    "The `.manipulate()` method takes keyword arguments `letters` and `features`. The provided values are used to\n",
    "attenuate or emphasize the probability of the given letters and phonetic features at each step of the spelling and sounding out process. Specifically, the decoded probability of the given item (letter or feature) is raised to the power of ``np.exp(n)``, where ``n`` is the provided value. A value of 0 will affect no change; negative values will increase the probability, positive values will decrease the probability. (A good range to try out is -10 to +10.)\n",
    "\n",
    "For example, passing the following parameter for `letters` attenuates the probability that the model will decode the letter `e`. Instead, it finds another alternative:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'spilling'"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pin.manipulate(\"spelling\", letters={'e': 10})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can include multiple key/value pairs. For example, to remove all vowels:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'sph-lyng'"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pin.manipulate(\"spelling\", letters={'a': 10, 'e': 10, 'i': 10, 'o': 10, 'u': 10})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example: Manipulating sounds\n",
    "\n",
    "The `features` parameter does the same thing as `letters`, except it affects the probability of particular phonetic features being predicted at each step of the decoding. This lets you expressively add or remove phonetic features from a word. For example, adding in more nasalness:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'smnenging'"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pin.manipulate(\"spelling\", features={'nas': -10})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Adding some voice and frication and removing voicelessness:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'sbezgizgs'"
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pin.manipulate(\"spelling\", features={'vcd': -5, 'frc': -5, 'vls': 10})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Interactive manipulation tool\n",
    "\n",
    "The code below builds an ipython widgets interface that lets you play around with the `manipulate()` function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [],
   "source": [
    "import ipywidgets as widgets\n",
    "from IPython.display import display\n",
    "from ipywidgets import interact, interactive_output, Layout, HBox, VBox"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [],
   "source": [
    "def manipulate(instr=\"allison\", temp=0.25, **kwargs):\n",
    "    return ' '.join([\n",
    "        pin.manipulate(\n",
    "            w,\n",
    "            letters={k: v*-1 for k, v in kwargs.items()\n",
    "                  if k in pin.orth2phon.src_vocab_idx_map.keys()},\n",
    "            features={k: v*-1 for k, v in kwargs.items()\n",
    "                      if k in pin.orth2phon.target_vocab_idx_map.keys()},\n",
    "            temperature=temp\n",
    "        ) for w in instr.split()]\n",
    "    )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "3286a85a54e54dd58ac419b4ca9f1089",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "VBox(children=(HBox(children=(VBox(children=(FloatSlider(value=0.0, continuous_update=False, description='$', …"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "23c7b41f9c064a0cac80f91d9ee16010",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Output(layout=Layout(height='100px'))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "orth_sliders = {}\n",
    "phon_sliders = {}\n",
    "for ch in pin.orth2phon.src_vocab_idx_map.keys():\n",
    "    if ch in \"'-.\": continue\n",
    "    orth_sliders[ch] = widgets.FloatSlider(description=ch,\n",
    "                               continuous_update=False,\n",
    "                               value=0,\n",
    "                               min=-20,\n",
    "                               max=20,\n",
    "                               step=0.5,\n",
    "                               layout=Layout(height=\"10px\"))\n",
    "for feat in pin.orth2phon.target_vocab_idx_map.keys():\n",
    "    if feat in (\"beg\", \"end\", \"cnt\", \"dnt\"): continue\n",
    "    phon_sliders[feat] = widgets.FloatSlider(description=feat,\n",
    "                               continuous_update=False,\n",
    "                               value=0,\n",
    "                               min=-20,\n",
    "                               max=20,\n",
    "                               step=0.5,\n",
    "                               layout=Layout(height=\"10px\"))\n",
    "instr = widgets.Text(description='input', value=\"spelling words with machine learning\")\n",
    "tempslider = widgets.FloatSlider(description='temp', continuous_update=False, value=0.3, min=0.01, max=5, step=0.05)\n",
    "left_box = VBox(tuple(orth_sliders.values()) + (tempslider,))\n",
    "right_box = VBox(tuple(phon_sliders.values()))\n",
    "all_sliders = HBox([left_box, right_box])\n",
    "\n",
    "out = interactive_output(lambda *args, **kwargs: print(manipulate(*args, **kwargs)),\n",
    "                         dict(instr=instr, temp=tempslider, **orth_sliders, **phon_sliders))\n",
    "out.layout.height = \"100px\"\n",
    "display(VBox([all_sliders, instr]), out)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Phoneme states\n",
    "\n",
    "Pincelate gives you access to a particular intermediary value in the neural network: the hidden state of phonetic encoder's RNN. This is a vector that has what amounts to a \"compressed\" representation of the phonetic content of a given string of characters. You can get this vector for a given string using the `.phonemestate()` method, passing the string whose phoneme state you want to capture. This value can be used for any number of downstream tasks, such as calculating phonetic similarity between two words, or as an initial embedding value for training classifiers on phonetics.\n",
    "\n",
    "We're going to use it for something that's a little bit more fun: generating nonsense words. Pincelate also supplies a method `.spellstate()` that takes the phoneme state and passes it to the orthographic decoder. So you can take an arbitrary value and spell a word from it!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example: Interpolation (blending words)\n",
    "\n",
    "The first example of this I'll show is *interpolating* between two words. To do this, calculate the phoneme state for both words in a pair, then take the average of the two (essentially the midpoint of the line that connects the two vectors), then decode from that vector using `.spellstate()`. The code below demonstrates:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [],
   "source": [
    "pairs = [('paper', 'plastic'),\n",
    "         ('kitten', 'puppy'),\n",
    "         ('birthday', 'anniversary'),\n",
    "         ('artificial', 'intelligence'),\n",
    "         ('allison', 'parrish'),\n",
    "         ('moses', 'middletown'),\n",
    "         ('day', 'night'),\n",
    "         ('january', 'december')]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "paper → palper → plastic\n",
      "kitten → cuptie → puppy\n",
      "birthday → artherday → anniversary\n",
      "artificial → interifical → intelligence\n",
      "allison → aarishen → parrish\n",
      "moses → midelsown's → middletown\n",
      "day → night → night\n",
      "january → denceure → december\n"
     ]
    }
   ],
   "source": [
    "for start_s, end_s in pairs:\n",
    "    start = pin.phonemestate(start_s)\n",
    "    end = pin.phonemestate(end_s)\n",
    "    mid = (start + end) / 2\n",
    "    mid_s = pin.spellstate(mid)\n",
    "    print(\" → \".join([start_s, mid_s, end_s]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example: Phonetic resizing of texts\n",
    "\n",
    "If you calculate the phoneme state for each word in a text, you essentially have a time series, sort of like samples in an audio file, or even like pixels in an image. Many of `scipy`'s image and audio manipulation functions work on 256-dim data as well as they work on 1d audio samples or 3d pixels, like the `zoom` function that we used above. In the cell below, I've made a small interactive widget for \"zooming\" in on a text—i.e., calculating the phoneme states for each word in a text, then resampling that array so it's longer, then decoding from the resampled array. In the process, you introduce interpolated data points, whose corresponding spellings seem to \"smear\" sound:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 162,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "a6931c18be024a1eba0a54d320c1d32a",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "interactive(children=(Text(value='phonetic resizing is fun', description='s'), FloatSlider(value=1.0, continuo…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "@interact(s=\"phonetic resizing is fun\",\n",
    "          factor=widgets.FloatSlider(\n",
    "              min=0.0, max=4, step=0.1, value=1.0,\n",
    "              continuous_update=False))\n",
    "def resizer(s, factor=1.0):\n",
    "    orig = np.array(\n",
    "        [pin.phonemestate(tok.lower()) for tok in s.split()]\n",
    "    )\n",
    "    resized = zoom(orig, (factor, 1), order=4)\n",
    "    return \" \".join([pin.spellstate(vec) for vec in resized])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Example: Phonetic variation\n",
    "\n",
    "You can create random words by decoding from a random phoneme state:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 144,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "am-zows\n",
      "soche\n",
      "tron\n",
      "utsties\n",
      "ixi\n",
      "rair's\n",
      "liufs\n",
      "gille\n",
      "x.\n",
      "yoove\n",
      "froos\n",
      "beas's\n"
     ]
    }
   ],
   "source": [
    "for i in range(12):\n",
    "    print(pin.spellstate(np.random.randn(256)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "However, you don't get a lot of variation in words generated this way, since the distribution of the RNN hidden state doesn't match any particular random distribution. So there are large parts of the \"space\" of possible words that will never be produced by picking numbers from (e.g.) a normal distribution, as above.\n",
    "\n",
    "A better strategy is to pick a word, and then generate variations on it by adding random noise to its phoneme state:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 139,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "t-thine\n",
      "pyt\n",
      "pytin\n",
      "peyhehthon\n",
      "pitheume\n",
      "phohthen\n",
      "pathane\n",
      "ptthengens\n",
      "phyhthen\n",
      "peiturenen\n",
      "piththen\n",
      "petheram\n"
     ]
    }
   ],
   "source": [
    "s = \"python\"\n",
    "state = pin.phonemestate(s)\n",
    "for i in range(12):\n",
    "    # add normal noise with the same number of dims as the state\n",
    "    noisy = state + np.random.randn(state.shape[0])\n",
    "    print(pin.spellstate(noisy))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the following cell, I've written a little interactive widget that applies the same noise to each word in the input. You can adjust the multiplier on the noise to make the words more or less close to the original state, and pick the random seed to use when generating the noise."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 160,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "b33d79190cf54f30a62a2d93040d4a84",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "interactive(children=(Text(value='how doth the little crocodile improve his shining tail', description='s'), F…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "@interact(s=\"how doth the little crocodile improve his shining tail\",\n",
    "          multiplier=widgets.FloatSlider(\n",
    "              min=0, max=2.0, step=0.1, value=1.0, continuous_update=False),\n",
    "          seed=widgets.IntSlider(min=0, max=10000, continuous_update=False))\n",
    "def noisytext(s, multiplier, seed):\n",
    "    # keep random number consistent for same string\n",
    "    np.random.seed(seed)\n",
    "    noise = np.random.randn(256)*multiplier\n",
    "    out = []\n",
    "    for w in s.split():\n",
    "        out.append(pin.spellstate(pin.phonemestate(w)+noise))\n",
    "    print(\" \".join(out))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}