notebooks/elasticsearch/tmdb/hello-ltr (ES).ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "# Hello LTR!"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "Fire up an elastic server with the LTR plugin installed and run thru the cells below to get started with Learning-to-Rank. These notebooks we'll use in this training have something of an ltr client library, and a starting point for demonstrating several important learning to rank capabilities.\n",
    "\n",
    "This notebook will document many of the important pieces so you can reuse them in future training sessions"
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### The library: ltr \n",
    "\n",
    "This is a Python library, located at the top level of the repository in `hello-ltr/ltr/`. It contains helper functions used through out the notebooks.\n",
    "\n",
    "If you want to edit the source code, make sure you are running the Jupyter Notebook server locally and not from a Docker container."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "import ltr\n",
    "import ltr.client as client\n",
    "import ltr.index as index\n",
    "import ltr.helpers.movies as helpers"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Download some requirements\n",
    "\n",
    "Several requirements/datasets are stored in online, these include various training data sets, the data sets, and tools. You'll only need to do this once. But if you lose the data, you can repeat this command if needed."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "corpus = 'http://es-learn-to-rank.labs.o19s.com/tmdb.json'\n",
    "\n",
    "ltr.download([corpus], dest='data/')"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Use the Elastic client\n",
    "\n",
    "Two LTR clients exist in this code, an ElasticClient and a SolrClient. The workflow for doing Learning to Rank is the same in both search engines"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "client = client.ElasticClient()"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Index Movies\n",
    "\n",
    "In these demos, we'll use [TheMovieDB](http://themoviedb.org) alongside some supporting assets from places like movielens.\n",
    "\n",
    "When we reindex, we'll use `ltr.index.rebuild` which deletes and recreates the index, with a few hooks to help us enrich the underlying data or modify the search engine configuration for feature engineering."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "movies = helpers.indexable_movies(movies='data/tmdb.json')\n",
    "\n",
    "index.rebuild(client, index='tmdb', doc_src=movies)"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Configure Learning to Rank\n",
    "\n",
    "We'll discuss the feature sets a bit more. You can think of them as a series of queries that will be stored and executed before we need to train a model. \n",
    "\n",
    "`setup` is our function for preparing learning to rank to optimize search using a set of features. In this stock demo, we just have one feature, the year of the movie's release."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "# wipes out any existing LTR models/feature sets in the tmdb index\n",
    "client.reset_ltr(index='tmdb')"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "# A feature set as a tuple, which looks a lot like JSON\n",
    "feature_set = {\n",
    "    \"featureset\": {\n",
    "        \"features\": [\n",
    "            {\n",
    "                \"name\": \"release_year\",\n",
    "                \"params\": [],\n",
    "                \"template\": {\n",
    "                    \"function_score\": {\n",
    "                        \"field_value_factor\": {\n",
    "                            \"field\": \"release_year\",\n",
    "                            \"missing\": 2000\n",
    "                        },\n",
    "                        \"query\": { \"match_all\": {} }\n",
    "                    }\n",
    "                }\n",
    "            }\n",
    "        ]\n",
    "    }\n",
    "}\n",
    "\n",
    "feature_set"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "# pushes the feature set to the tmdb index's LTR store (a hidden index)\n",
    "client.create_featureset(index='tmdb', name='release', ftr_config=feature_set)"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "## Is this thing on?\n",
    "\n",
    "Before we dive into all the pieces, with a real training set, we'll try out two examples of models. One that always prefers newer movies. And another that always prefers older movies. If you're curious you can opet `classic-training.txt` and `latest-training.txt` after running this to see what the training set looks like. "
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Generate some judgement data\n",
    "\n",
    "This will write out judgment data to a file path.\n",
    "\n",
    "Look at the source code in `ltr/years_as_ratings.py` to see what assumptions are being made in this synthetic judgment. What assumptions do you make in your judgment process?"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "from ltr.years_as_ratings import synthesize\n",
    "\n",
    "synthesize(\n",
    "    client, \n",
    "    featureSet='release', # must match the name set in client.create_featureset(...)\n",
    "    classicTrainingSetOut='data/classic-training.txt',\n",
    "    latestTrainingSetOut='data/latest-training.txt'\n",
    ")"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Format the training data as two arrays of Judgement objects\n",
    "\n",
    "This step is in preparation for passing the traning data into Ranklib."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "import ltr.judgments as judge\n",
    "\n",
    "classic_training_set = [j for j in judge.judgments_from_file(open('data/classic-training.txt'))]\n",
    "latest_training_set = [j for j in judge.judgments_from_file(open('data/latest-training.txt'))]"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Train and Submit\n",
    "\n",
    "We'll train a lot of models in this class! Our ltr library has a `train` method that wraps a tool called `Ranklib` (more on Ranklib later), allows you to pass the most common commands to Ranklib, stores a model in the search engine, and then returns diagnostic output that's worth inspecting. \n",
    "\n",
    "For now we'll just train using the generated training set, and store two models `latest` and `classic`.\n"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "from ltr.ranklib import train\n",
    "\n",
    "train(client, training_set=latest_training_set, \n",
    "      index='tmdb', featureSet='release', modelName='latest')"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "Now train another model based on the 'classsic' movie judgments."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "train(client, training_set=classic_training_set, \n",
    "      index='tmdb', featureSet='release', modelName='classic')"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### Ben Affleck vs Adam West\n",
    "If we search for `batman`, how do the results compare?  Since the `classic` model prefered old movies it has old movies in the top position, and the opposite is true for the `latest` model.  To continue learning LTR, brainstorm more features and generate some real judgments for real queries."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "import ltr.release_date_plot as rdp\n",
    "rdp.plot(client, 'batman')"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### See top 12 results for both models\n",
    "\n",
    "Looking at the `classic` model first."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "import pandas as pd\n",
    "classic_results = rdp.search(client, 'batman', 'classic')\n",
    "pd.json_normalize(classic_results)[['id', 'title', 'release_year', 'score']].head(12)"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "And then the `latest` model."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [
    "latest_results = rdp.search(client, 'batman', 'latest')\n",
    "pd.json_normalize(latest_results)[['id', 'title', 'release_year', 'score']].head(12)"
   ],
   "outputs": [],
   "metadata": {}
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}