Small python2 module to reliably detect web spiders using reverse DNS lookups for data harvesting protection.
Python
Switch branches/tags
Nothing to show
Latest commit 312b014 Jun 27, 2012 @badzong badzong Added yasni.de
Permalink
Failed to load latest commit information.
.gitignore Added .gitignore Aug 15, 2011
LICENSE Initial import Aug 12, 2011
README.md README update Aug 12, 2011
botdetector.py Added yasni.de Jun 27, 2012
setup.py Changed python package into module Aug 12, 2011

README.md

botdetector

Small python2 module to reliably detect web spiders using reverse DNS lookups for data harvesting protection.

Spiders supported:

  • Googlebot (googlebot.com)
  • bingbot (search.msn.com)

Adding a new bot is very easy, just append the domain to the bots list of the BotDetector instance.

Install

python setup.py install

Example

>>> from botdetector import BotDetector
>>> bd = BotDetector()
>>> bd.bot('66.249.66.43')
'googlebot.com'
>>> bd.bot('207.46.204.229')
'search.msn.com'
>>> bd.bot('1.2.3.4')
>>>