Skip to content
Using python-selenium to grab websites in google.
Branch: master
Clone or download
Type Name Latest commit message Commit time
Failed to load latest commit information.
LICENSE Initial commit Apr 5, 2016 README file. Apr 5, 2016

1. Preparing

Install packages:

pip install selenium
pip install bs4
pip install json

ChromeDriver were used because I used to use chrome to search google(fxxk G F W). It's OK for Firefox or other browsers if you can use it for searching google. Remember to put chromedriver into $PATH before starting selenium.

option(Removing results):

Remove json file can result in searching all queries, or only queries with "NA" or "None" address would be scanned.

rm test.json

2. Run script:


This script may have to be re-run for many times in order to continue a interupted scanning.

In, two browsers would be openned. For the testing data:



And these gif were generated using This scripts usedimages2gif, however, a bug have to be fixed:

You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.