Skip to content
E-mails, subdomains and names Harvester - OSINT
Python Dockerfile
Branch: master
Clone or download

Latest commit

Files

Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.github update GH actions to use v2 Feb 15, 2020
README misc fixes and working on new module Feb 11, 2020
requirements Fixes #407 Mar 23, 2020
tests Updated certspotter search and changed github code to start with test. Feb 13, 2020
theHarvester fix #397 Mar 13, 2020
wordlists Syncing and updated crtsh to work properly. Aug 8, 2019
.gitattributes Removed google-profiles and clean up. Feb 14, 2019
.gitignore Updated .gitignore. Jan 3, 2020
.lgtm.yml Make sure we set lgtm.yml to use python3 Sep 22, 2019
.travis.yml update travis Feb 15, 2020
Dockerfile Update readme and deps and fix docker build Mar 22, 2020
Pipfile Update readme and deps and fix docker build Mar 22, 2020
Pipfile.lock Update readme and deps and fix docker build Mar 22, 2020
README.md Update readme and deps and fix docker build Mar 22, 2020
api-keys.yaml readded file. Feb 7, 2020
mypy.ini Update mypy settings to use the new features of the 0.730 release Sep 26, 2019
proxies.yaml Removed https proxies as aiohttp only officially sports http proxies. Feb 7, 2020
requirements.txt added requirements.txt Dec 31, 2019
setup.cfg remove commented out flake8 param Sep 26, 2019
setup.py Fix proxy in setup that was missed and disable github test Feb 11, 2020
theHarvester-logo.png Update theHarvester-logo.png Sep 10, 2019
theHarvester.py Remove 1 test and tidy up a few things and work on sypse module also … Jan 14, 2020

README.md

theHarvester

Build Status Language grade: Python Rawsec's CyberSecurity Inventory

What is this?

theHarvester is a very simple to use, yet powerful and effective tool designed to be used in the early statges of a
penetration test or red team engagement. Use it for open source intelligence (OSINT) gathering to help determine a
company's external threat landscape on the internet. The tool gathers emails, names, subdomains, IPs and URLs using
multiple public data sources that include:

Passive:

  • baidu: Baidu search engine - www.baidu.com

  • bing: Microsoft search engine - www.bing.com

  • bingapi: Microsoft search engine, through the API (Requires an API key, see below.)

  • Bufferoverun: Uses data from Rapid7's Project Sonar - www.rapid7.com/research/project-sonar/

  • CertSpotter: Cert Spotter monitors Certificate Transparency logs - https://sslmate.com/certspotter/

  • crtsh: Comodo Certificate search - https://crt.sh/

  • dnsdumpster: DNSdumpster search engine - https://dnsdumpster.com/

  • dogpile: Dogpile search engine - www.dogpile.com

  • duckduckgo: DuckDuckGo search engine - www.duckduckgo.com

  • Exalead: a Meta search engine - www.exalead.com/search

  • github-code: GitHub code search engine (Requires a GitHub Personal Access Token, see below.) - www.github.com

  • google: Google search engine (Optional Google dorking.) - www.google.com

  • hunter: Hunter search engine (Requires an API key, see below.) - www.hunter.io

  • intelx: Intelx search engine (Requires an API key, see below.) - www.intelx.io

  • linkedin: Google search engine, specific search for LinkedIn users - www.linkedin.com

  • netcraft: Internet Security and Data Mining - www.netcraft.com

  • otx: AlienVault Open Threat Exchange - otx.alienvault.com

  • securityTrails: Security Trails search engine, the world's largest repository of historical DNS data
    (Requires an API key, see below.) - www.securitytrails.com

  • shodan: Shodan search engine, will search for ports and banners from discovered hosts - www.shodanhq.com

  • Spyse: Web research tools for professionals (Requires an API key.) - spyse.com

  • Suip: Web research tools that can take over 10 minutes to run, but worth the wait - suip.biz

  • threatcrowd: Open source threat intelligence - www.threatcrowd.org

  • trello: Search trello boards (Uses Google search.)

  • twitter: Twitter accounts related to a specific domain (Uses Google search.)

  • vhost: Bing virtual hosts search

  • virustotal: virustotal.com domain search

  • yahoo: Yahoo search engine

Active:

  • DNS brute force: dictionary brute force enumeration

Modules that require an API key:

Add your keys to api-keys.yaml

  • bing
  • github
  • hunter
  • intelx
  • securityTrails
  • shodan
  • spyse

Install and dependencies:

Comments, bugs and requests:

Main contributors:

  • Twitter Follow Matthew Brown @NotoriousRebel1
  • Twitter Follow Jay "L1ghtn1ng" Townsend @jay_townsend1
  • Twitter Follow Lee Baird @discoverscripts
  • LinkedIn Janos Zold

Thanks:

  • John Matherly - Shodan project
  • Ahmed Aboul Ela - subdomain names dictionaries (big and small)
You can’t perform that action at this time.