-
create a virtualenv and activate
-
pip install requirements.txt
-
python crawl.py scrapy ## will download podcast names from itunes pages
-
python crawl.py lookup ## will download more info from podcast using the lookup itunes api https://www.apple.com/itunes/affiliates/resources/documentation/itunes-store-web-service-search-api.html
-
python crawl.py merge ## merge the two datafiles
-
python crawl.py addfeeddata ## download the rss xml and extracts the information
-
python crawl.py elasticsearch ## add data to elasticsearch service
-
cd http; python query.py ## to see the search on a rest api