Skip to content
An ipython notebook demonstrating working with learning to rank in Solr or Elasticsearch
Jupyter Notebook Python Other
Branch: master
Clone or download
Latest commit 1ca7e78 Aug 13, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
data Initial commit Mar 11, 2019
docker Add WDF and keyword title fields Aug 13, 2019
ltr Add comment Aug 13, 2019
rre RRE config for elastic Apr 21, 2019
.gitignore Ignore data Mar 18, 2019
Dockerfile Initial commit Mar 11, 2019
LICENSE License Apr 23, 2019
README.md consolidate docker; add Kibana docker to ES Jul 19, 2019
click models.ipynb Print output for rambo Aug 13, 2019
conversion-augmented-click-models.ipynb Clean up notebook output Jun 30, 2019
evaluation (Solr).ipynb Clean up notebook output Jun 30, 2019
evaluation.ipynb Clean up notebook output Jun 30, 2019
gonna need a bigger bot (ES).ipynb Remove tmdb in clients to make more generic Jul 25, 2019
gonna need a bigger bot (Solr).ipynb Remove tmdb in clients to make more generic Jul 25, 2019
hello-ltr (ES).ipynb Remove tmdb in clients to make more generic Jul 25, 2019
hello-ltr (Solr).ipynb Remove tmdb in clients to make more generic Jul 25, 2019
loadMart.py Add whoopsie stuff to Netfix Mar 30, 2019
netfix movies(Solr).ipynb Remove tmdb in clients to make more generic Jul 25, 2019
netfix movies.ipynb clean up netfix from training Aug 13, 2019
osc-blog.ipynb Add notes to OSC Blog Aug 13, 2019
rate.py Add copy operator to rate script Jul 19, 2019
rateFuzzySearch.json.jinja Move rate.py up to root Mar 20, 2019
rateSearch.json.jinja Save work Mar 23, 2019
raw-es-commands.ipynb Add search to raw ES Aug 13, 2019
raw-solr-commands.ipynb Remove tmdb in clients to make more generic Jul 25, 2019
requirements.txt Restore jupyter requirements Jun 26, 2019
sandbox.ipynb Clean up notebook output Jun 30, 2019
tale-of-two-queries (ES).ipynb Missing index param Aug 7, 2019
tale-of-two-queries (Solr).ipynb Remove tmdb in clients to make more generic Jul 25, 2019
train_to_csv.py Add a judgments to CSV converter Apr 16, 2019

README.md

hello-ltr

The overall goal of this project is to demonstrate all of the steps required to work with LTR in Elasticsearch or Solr. Follow the setup instructions and check out the LTR notebooks in Solr or Elasticsearch

Setup your search engine

LTR examples here for Solr or Elasticsearch which require the right search engine to be installed

Setup Solr w/ LTR

With Docker installed, a script will launch Solr & the config under the solr/ dir in the console:

cd docker/solr
./launch_solr.sh

Or manually

  • Go into the "Solr" docker directory: cd docker/solr
  • Run docker build . -t ltr-solr to create a image running Solr with LTR
  • Start the instance by running: docker run --name ltr-solr -p 8983:8983 -d ltr-solr
  • Subsequently run with docker start ltr-solr and docker stop ltr-solr

Setup Elasticsearch w/ LTR

With Docker installed, a script will launch Elasticsearch w/ Kibana tooling in the console:

cd docker/elasticsearch
./launch_es.sh

Manually build & run the containers

# Create Elasticsearch
cd es-docker
docker build -t ltr-elasticsearch .

# Create Kibana
cd kb-docker
docker build -t ltr-kibana .

# Launch
cd ..
docker-compose up

Setup & Run Jupyter Notebook Examples

Setup Python requirements

  • Ensure Python 3 is installed on your system
  • Create a virtual environment: python3 -m venv venv
  • Start the virtual environment: source venv/bin/activate
  • Install the requirements pip install -r requirements.txt

Note: The above commands should be run from the root folder of the project.

Start Jupyter notebook and confirm operation

  • Run jupyter notebook
  • Open either the "hello-ltr (Solr)" or "hello-ltr (ES)" as approriatte and ensure you get a graph at the last cell

Getting Started

  • Run jupyter notebook and load the hello-ltr notebook
  • Run thru each cell to get more familiar with the LTR pipeline

Docker Compose

If you hit any snags with the JDK or python dependencies, the docker folder has a docker-compose configuration that prepares an environment to run all of the notebooks.

You can’t perform that action at this time.