the scraper for the pirate bay. written in node js. with data saved in a json format. and everything kept track of in mysql.
- install nodejs, npm, php and mysql.
- git clone.
- npm install in the working dir.
- import the sql from /sql
- edit the settings in "setting.json"
- ready to run index.js which does the scraping.
- set getNew.js to run every 5 min with cron for example
- spell checking.
- code review.
- profiling of the code (it is very slow now).
- bringing up issues.
- write some test for the data.
- commenting the code.
if you are interested message me and I will help you start
- finding edge casses.
- writing test.
- publishing data.