E-mail, subdomain and people names harvester
Python
Latest commit eb08d32 Oct 11, 2016 @laramies Added missing search engines.
Added missing search engines, fixed search engine handling.
Permalink
Failed to load latest commit information.
discovery Fixed PGP keyserver search Oct 11, 2016
lib 2.6 May 10, 2015
tests Email parser Apr 18, 2016
.gitignore Email parser Apr 18, 2016
COPYING Initial commit for version 2.0 May 4, 2011
LICENSES 2.5 Dec 16, 2014
README Added missing search engines. Oct 11, 2016
changelog.txt 2.6 May 10, 2015
myparser.py 2.7 Apr 18, 2016
theHarvester.py Added missing search engines. Oct 11, 2016

README

*******************************************************************
*                                                                 *
* | |_| |__   ___    /\  /\__ _ _ ____   _____  ___| |_ ___ _ __  *
* | __| '_ \ / _ \  / /_/ / _` | '__\ \ / / _ \/ __| __/ _ \ '__| *
* | |_| | | |  __/ / __  / (_| | |   \ V /  __/\__ \ ||  __/ |    *
*  \__|_| |_|\___| \/ /_/ \__,_|_|    \_/ \___||___/\__\___|_|    *
*                                                                 *
* TheHarvester Ver. 2.7                                          *
* Coded by Christian Martorella                                   *
* Edge-Security Research                                          *
* cmartorella@edge-security.com                                   *
*******************************************************************

What is this?
-------------

theHarvester is a tool for gathering e-mail accounts, subdomain names, virtual
hosts, open ports/ banners, and employee names from different public sources
(search engines, pgp key servers).

Is a really simple tool, but very effective for the early stages of a penetration
test or just to know the visibility of your company in the Internet.

The sources are:

Passive:
--------
-google: google search engine  - www.google.com

-googleCSE: google custom search engine

-google-profiles: google search engine, specific search for Google profiles

-bing: microsoft search engine  - www.bing.com

-bingapi: microsoft search engine, through the API (you need to add your Key in
          the discovery/bingsearch.py file)

-dogpile: Dogpile search engine - www.dogpile.com

-pgp: pgp key server - mit.edu

-linkedin: google search engine, specific search for Linkedin users


-vhost: Bing virtual hosts search

-twitter: twitter accounts related to an specific domain (uses google search)

-googleplus: users that works in target company (uses google search)

-yahoo: Yahoo search engine

-baidu: Baidu search engine

-shodan: Shodan Computer search engine, will search for ports and banner of the
         discovered hosts  (http://www.shodanhq.com/)


Active:
-------
-DNS brute force: this plugin will run a dictionary brute force enumeration
-DNS reverse lookup: reverse lookup of ip´s discovered in order to find hostnames
-DNS TDL expansion: TLD dictionary brute force enumeration


Modules that need API keys to work:
----------------------------------
-googleCSE: You need to create a Google Custom Search engine(CSE), and add your
 Google API key and CSE ID in the plugin (discovery/googleCSE.py)
-shodan: You need to provide your API key in discovery/shodansearch.py


Dependencies:
------------
-Requests library (http://docs.python-requests.org/en/latest/)
`pip install requests`

Changelog in 2.7:
------------------
-Fixed email parser for cleaner results. Thanks Th4nat0s
-Improved XML exporting file format
-Removed 123people
-Fixed Bing cookies order

Changelog in 2.6:
------------------
-Added Yahoo and Baidu search engines. Thanks to Tatanus
-Added check for the existence of Requests library.
-Fixed email regex to provide cleaner results. Thanks to Peter McAlpine

Changelog in 2.5:
-----------------
-Replaced httplib by Requests http library (for Google related)
-Fixed Google searches


Comments? Bugs? Requests?
------------------------
cmartorella@edge-security.com

Updates:
--------
https://github.com/laramies/theHarvester

Thanks:
-------
John Matherly -  SHODAN project
Lee Baird for suggestions and bugs reporting