Skip to content
Full text search over deliveroo restaurants
JavaScript Python CSS HTML Dockerfile
Branch: master
Clone or download
Latest commit 13f6bfa Dec 5, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
docker dockerfile works! Nov 20, 2019
frontend Update frontend Nov 19, 2019
old
scrapyroo
LICENSE Create LICENSE Feb 14, 2019
README.md
index some readme prettification Nov 19, 2019
requirements.txt handle all parsing by scrapy May 6, 2019
scrape
scrapy.cfg
serve prettify serve script Nov 18, 2019

README.md

Cause I got sick of clicking through Deliveroo restaurants. Full text search is a basic human right!

Slides + speaker notes from November Rust London meetup.

Setup and usage

Scraper

Install dependencies: pip3 install --user -r requirements.txt

Add something like:

./scrape --area 'london/canning-town' --postcode 'E164SA' --json /path/to/menus.jsonl

to your crontab to run once a day, e.g. in the morning or something. Presumably, menus don't change often so that's enough.

Indexer

Install tantivy-py, custom branch needed for extra functionality that's not merged yet:

cargo install  \
--git https://github.com/karlicoss/tantivy-py \
--branch delete-all-documents
./index --index /path/to/tantivy-index /path/to/menus.jsonl

Backend

Install tantivy-cli, custom branch needed at the moment to expose highlights via the API:

cargo install  \
--git https://github.com/karlicoss/tantivy-cli \
--branch serve-snippets

Tantivy runs on port 3000 by default

./serve --index /path/to/tantivy-index

Frontend

NOTE: if you're running page locally, you're gonna need to pass by CORS.

You can do it by e.g. using Chromium with no web security:

chromium-browser --disable-web-security --user-data-dir=/tmp/whatever frontend/index.html
You can’t perform that action at this time.