Skip to content
Using python-selenium to grab websites in google.
Branch: master
Clone or download
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
images
LICENSE Initial commit Apr 5, 2016
README.md README file. Apr 5, 2016
University.txt
browser1.GIF
browser2.GIF
main.py
makeGif.py
test.json

README.md

1. Preparing

Install packages:

pip install selenium
pip install bs4
pip install json

ChromeDriver were used because I used to use chrome to search google(fxxk G F W). It's OK for Firefox or other browsers if you can use it for searching google. Remember to put chromedriver into $PATH before starting selenium.

option(Removing results):

Remove json file can result in searching all queries, or only queries with "NA" or "None" address would be scanned.

rm test.json

2. Run script:

python main.py

This script may have to be re-run for many times in order to continue a interupted scanning.

In main.py, two browsers would be openned. For the testing data:

browser1:

browser2:

And these gif were generated using makeGif.py. This scripts usedimages2gif, however, a bug have to be fixed: http://stackoverflow.com/questions/19149643/error-in-images2gif-py-with-globalpalette

You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.