Business Directory Scraper

Crawls company profile information from a comprehensive business directory of companies in Philippines (PhilippinesBusinessList) using Scrapy and store them into MongoDB.

Environment

Works on Ubuntu 16.04 LTS

Language: Python 3.5.2

Database: MongoDB

Depedencies:

• Scrapy
• Pymongo

Instructions for use

Pre-boot configuration

• Install MongoDB and start without configuration (create database and collection)

• Install Python dependent modules： Scrapy, pymongo, requests or

    $ pip install -r requirements.txt
  

Start up

$ cd PhilippinesBusinessList
$ python3 quickstart.py

Run screenshot