Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Fetching contributors…

Cannot retrieve contributors at this time

206 lines (202 sloc) 3.066 kb
RobotUA <<EOR
adressendeutschland,
agent,
AltaVista,
appie,
AppleSyndication,
Arachnoidea,
Aranha,
Architext,
archive,
Argus,
Ask,
asterias,
ATN_Worldwide,
Atomz,
AurNet,
Awasu,
BackRub,
bender,
Bookdog,
BookmarkSync,
bot,
Builder,
ccubee,
cfetch,
CFNetwork,
CMC,
collector,
complex_network_group,
Contact,
crawl,
Creep,
Digital*Integrity,
Directory,
dogpile,
Excite,
EZResult,
FavOrg,
FeedDemon,
Feedreader,
FeedValidator,
Ferret,
fido,
find,
Fireball,
gazz,
GetRight,
gonzo,
Google-Sitemaps,
GoogleBot,
grab,
griffon,
Gromit,
Gulliver,
H?m?h?kki,
heritrix,
HTTrack,
Harvest,
holmes,
HTMLDOC,
Hubater,
IncyWincy,
index,
INGRID,
Jack,
JPluck,
KIT*Fireball,
Kototoi,
larbin,
Leech,
legs,
locator,
LWP,
Lycos,
marvin,
Mediapartners,
MegaSheep,
MEGAUPLOAD,
Mercator,
MFC_Tear_Sample,
Microsoft Data Access,
Microsoft Office,
Microsoft URL Control,
Microsoft-WebDAV,
MimeLive,
mirago,
Miva,
moget,
MSFrontPage,
Nazilla,
NetMechanic,
NetScoop,
newscan,
Nutch,
Ocelli,
ozelot,
ozzie,
pagebull,
ParaSite,
pavuk,
POE-Component,
Pokey,
Pompos,
Refiner,
retrieve,
RoboDude,
Rover,
Rssbandit,
RSSOwl,
Rutgers,
Scooter,
search,
seek,
shelob,
silk,
Slurp,
sna,
Snappy,
Snoopy,
speedy,
spider,
Spyder,
suke,
Susie,
swish,
T-H-U-N-D-E-R-S-T-O-N-E,
tarantula,
topiclink,
Toutatis,
TurnitinBot,
Tv*Merc,
Twiceler,
urllib,
VB Project,
Valkyrie,
Voyager,
W3C_Validator,
Walker,
wget,
WhizBang,
whowhere,
Wiki,
WinInet,
winona,
Wire,
Wombat,
WordPress,
worm,
wwwster,
WWW-Mechanize,
xtreme,
Yahoo,
Yandex,
Zeus,
ZyBorg,
EOR
NotRobotUA <<EOR
AskTB,
Seekmo,
EOR
RobotIP <<EOR
202.9.155.123,
204.152.191.41,
208.146.26.19,
208.146.26.233,
209.185.141.209,
209.185.141.211,
209.202.148.36,
209.202.148.41,
216.200.130.207,
216.35.103.6?,
216.35.103.70,
82.99.30.?, # Munax
82.99.30.1?, # Munax
82.99.30.2?, # Munax
82.99.30.3?, # Munax
82.99.30.4?, # Munax
82.99.30.5?, # Munax
82.99.30.6?, # Munax
82.99.30.70, # Munax
82.99.30.71, # Munax
82.99.30.72, # Munax
82.99.30.73, # Munax
EOR
RobotHost <<EOR
*.analys.google.com,
*.ask.com,
*.crawler*.com,
*.csccorporatedomains.com,
*.excite.com,
*.googlebot.com,
*.infoseek.com,
*.inktomi.com,
*.inktomisearch.com,
*.lycos.com,
*.pa-x.dec.com,
*.phx.gbl,
*.search.live.com,
add-url.altavista.com,
msnbot.msn.com,
westinghouse-rsl-com-usa.NorthRoyalton.cw.net,
EOR
Jump to Line
Something went wrong with that request. Please try again.