SOURCESCRUB Scraper

Scraper to fetch details of companies from SourceScrub.com
Project Contract   ·   Client - Dustin Johnson   ·   Status - Completed

Table of contents

• Prerequisite
• Unpacking
• Installation Dependencies
• Understanding Project File
• Running The Script
• Show Your Support
• Contact Me

Prerequisite

- Windows
- Python 3.6 or above
- Chrome Webdriver

Install Python on Windows 10

Get Chrome WebDriver according to your Google Chrome Version

Unpacking

• Unzip the SourceScrubScrapper.zip in desired location. Let's assume that we unzip the file in location _E:\SourceScrub
• After this the path of the Project lookes like E:\SourceScrub
• The project folder should look something like this:

Installation Dependencies

• Start terminal and type below command in terminal to point to the project folder:

~$ E:\
~$ cd SourceScrub

• Now we need to download all the dependencies required to run the script. For this we will type below command in terminal:

~$ pip3 install -r requirements.txt

Understanding Project File

requirement.txt ⛔

This file contains all the dependencies that we need to install on our system. You can delete this file but its Ok to keep it there and forget that it exists.

script.py 🚫

This file is the main script that we need to run to get the desired output. Please never touch this file.

config.json ✍️

This file needs to be edited everytime you run the script and so it needs some explation...

• path_to_chromedriver => Path where the chrome webdriver executeable file is located. Take care of the Double Black Slash.

• starting_pont => URL which script should open when initiated.

• sleep_time => List of two integer, represents time for which the script should sleep between each hit to the server.

Running The Script

So before running the script please make sure of two things:

• You have edited the config file properly.
• The pointer in terminal is pointing to the project forlder. If not then use below code:

~$ E:\
~$ cd SourceScrub

Now we need to enter below code to execute the script.

~$ python3 script.py

Now just grab yourself a pint of 🍺 and let the script do its task.

Show Your Support

If you are happy with my work then please give me ⭐⭐⭐⭐⭐ rating and also leave really nice recommendation/feedback on upwork. This will help me a lot in getting more project. A small and happy bonus is always appreciated 🤩. Also kindly rememeber me if you have any such project or any scraping projects.

Thank You for giving me opportunity to work on this project.

Contact Me

Siddhant Shah - Please feel free to connect to me in case there is any issue in the script or any changes are required. You can contact on below mentioned connects

Website >    > Mail Me >    > UpWork >    > Instagram >    > WhatsApp