Movie Crawler

This is simple movie crawler that scrapes popular movies website from MOVIE DATABASE

Installation

In order to run this project make sure you have python3 installed if not you can always check using this command python3 --version

• Once you have python3 installed now install pipenv by refering to pipenv docs
• After installing pipenv on your local system now clone this repo git clone https://github.com/mraza007/movie-scraper.git.
• Once you clone the repo now cd movie-scraper and run this command to activate virtual environment pipenv shell.
• After activating virtual env now install dependencies using this command pipenv install.

Usage

• In order to use this crawler simply run this command scrapy crawl movies_data.
• After running that command this is going to scrape the results and save into movies.csv files.

Contributions

Any contribution is welcomed in order to improve this software if you have any questions about this crawler feel free to contact me

Note: This only scrapes 200 pages