Skip to content
Please note that GitHub no longer supports Internet Explorer.

We recommend upgrading to the latest Microsoft Edge, Google Chrome, or Firefox.

Learn more
E-mails, subdomains and names Harvester - OSINT
Python Dockerfile
Branch: master
Clone or download
L1ghtn1ng Fix otx module and make the build pass again (#377)
* Fix otx module

* Fix flake8 issue
Latest commit 57cd908 Dec 30, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.github Bug fixes for file locations and more static linting, pep8 fixes and … Oct 21, 2019
tests Fix unit test due to the pep8 fix for stash lib Oct 21, 2019
theHarvester Fix otx module and make the build pass again (#377) Dec 30, 2019
wordlists Syncing and updated crtsh to work properly. Aug 8, 2019
.gitattributes Removed google-profiles and clean up. Feb 14, 2019
.gitignore Bug fixes for file locations and more static linting, pep8 fixes and … Oct 21, 2019
.lgtm.yml Make sure we set lgtm.yml to use python3 Sep 22, 2019
.travis.yml API key fix plus a few other fixes Oct 19, 2019
CONTRIBUTING.md Move contributing to the correct place and file name convention Sep 7, 2019
COPYING Fix line endings. Feb 4, 2019
Dockerfile Revert "Update ci and dockerfile" Oct 12, 2019
LICENSES Fix line endings. Feb 4, 2019
Pipfile more ci updates Oct 19, 2019
Pipfile.lock more ci updates Oct 19, 2019
README.md Shortened bing under Modules that require an API key. Nov 26, 2019
api-keys.yaml Add Spyse to api-keys.yaml Sep 29, 2019
mypy.ini Update mypy settings to use the new features of the 0.730 release Sep 26, 2019
requirements.txt Add gevent to requirements.txt Nov 13, 2019
setup.cfg remove commented out flake8 param Sep 26, 2019
setup.py Bug fixes for file locations and more static linting, pep8 fixes and … Oct 21, 2019
theHarvester-logo.png Update theHarvester-logo.png Sep 10, 2019
theHarvester.py API key fix plus a few other fixes Oct 19, 2019

README.md

theHarvester

Build Status Language grade: Python Rawsec's CyberSecurity Inventory

What is this?

theHarvester is a very simple, yet effective tool designed to be used in the early
stages of a penetration test. Use it for open source intelligence gathering and
helping to determine a company's external threat landscape on the internet. The
tool gathers emails, names, subdomains, IPs, and URLs using multiple public data
sources that include:

Passive:

  • baidu: Baidu search engine - www.baidu.com

  • bing: Microsoft search engine - www.bing.com

  • bingapi: Microsoft search engine, through the API (Requires an API key, see below.)

  • CertSpotter: Cert Spotter monitors Certificate Transparency logs - https://sslmate.com/certspotter/

  • crtsh: Comodo Certificate search - www.crt.sh

  • dnsdumpster: DNSdumpster search engine - dnsdumpster.com

  • dogpile: Dogpile search engine - www.dogpile.com

  • duckduckgo: DuckDuckGo search engine - www.duckduckgo.com

  • Exalead: a Meta search engine - https://www.exalead.com/search

  • github-code: Github code search engine (Requires a Github Personal Access Token, see below.) - www.github.com

  • google: Google search engine (Optional Google dorking.) - www.google.com

  • hunter: Hunter search engine (Requires an API key, see below.) - www.hunter.io

  • intelx: Intelx search engine (Requires an API key, see below.) - www.intelx.io

  • linkedin: Google search engine, specific search for LinkedIn users - www.linkedin.com

  • netcraft: Internet Security and Data Mining - www.netcraft.com

  • otx: AlienVault Open Threat Exchange - https://otx.alienvault.com

  • securityTrails: Security Trails search engine, the world's largest repository
    of historical DNS data (Requires an API key, see below.) - www.securitytrails.com

  • shodan: Shodan search engine, will search for ports and banners from discovered
    hosts - www.shodanhq.com

  • Spyse: Web research tools for professionals (Requires an API key.) - https://spyse.com/

  • Suip: Web research tools that can take over 10 minutes to run, but worth the wait. - https://suip.biz/

  • threatcrowd: Open source threat intelligence - www.threatcrowd.org

  • trello: Search trello boards (Uses Google search.)

  • twitter: Twitter accounts related to a specific domain (Uses Google search.)

  • vhost: Bing virtual hosts search

  • virustotal: virustotal.com domain search

  • yahoo: Yahoo search engine

Active:

  • DNS brute force: dictionary brute force enumeration

Modules that require an API key:

Add your keys to api-keys.yaml

  • bing
  • github
  • hunter
  • intelx
  • securityTrails
  • shodan
  • spyse

Dependencies:

  • Python 3.7+
  • python3 -m pip install pipenv
  • pipenv install

Comments, bugs, or requests?

Main contributors:

  • Twitter Follow Matthew Brown @NotoriousRebel1
  • Twitter Follow Jay "L1ghtn1ng" Townsend @jay_townsend1
  • Twitter Follow Lee Baird @discoverscripts
  • LinkedIn Janos Zold

Thanks:

  • John Matherly - Shodan project
  • Ahmed Aboul Ela - subdomain names dictionaries (big and small)
You can’t perform that action at this time.