Skip to content
This repository has been archived by the owner on Jan 30, 2023. It is now read-only.

AndrijaS37N/custom-image-scraper

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1 Commit
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Custom Image Scraper

A simple Python console app.

  • Making for the CS324 Scripting Languages course.
  • Redone for Python 3.6.
  • Made with PyCharm.

External requirements

For base scraping: google_images_download python package.

Note: I only found this package with pip.

pip install google_images_download

Web driver #1: selenium web driver python package.

pip install selenium

Web driver #2: chromedriver web driver for Google Chrome. You'll need to provide a path to the chromedriver executable on you computer in the processing.py file.

Web drivers are needed for downloading more than 200 image files per argument.

Internal requirements

For renaming and resizing of images: renaming-resizing-script which is my own project. Be advised that this project has its own requirements. This bash script I've made it already present for this project, hence the term internal.

Alternatives

Inspect the alts folder for a bing alternative scraping script which isn't made originally by me.

Option #1: For Google scraping check out this link. I haven't tried out any of them yet. The example that I've placed in the alts folder requires the beautifulsoup4 library. This option was deleted.

Option #1: A Bing searching API provided for students; a Microsoft Azure account is needed for a key to use in the scraping script. You'll also need these two for this script:

pip install requests
pip install opencv-python

Usage

Change directory to the code folder and then type in this command in the terminal:

python main.py

Voilà. Make sure to primarily activate your virtual environment of course. Just a reminder.

# for Anaconda
source activate 'the-name-of-your-env'
# for venv
source 'venv/bin/activate'

Please read the comments in the code in order to better understand the execution of the program.

Project results:

Boot ↘

View 1

Search Args ↘

View 2

Work I ↘

View 3

Work II ↘

View 4

Work III ↘

View 5

Work IV ↘

View 6

Script Finished ↘

View 7