Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Fetching contributors…

Cannot retrieve contributors at this time

201 lines (198 sloc) 3.021 kB
RobotUA <<EOR
adressendeutschland,
agent,
AltaVista,
appie,
AppleSyndication,
Arachnoidea,
Aranha,
Architext,
archive,
Argus,
Ask,
asterias,
ATN_Worldwide,
Atomz,
AurNet,
Awasu,
BackRub,
bender,
Bookdog,
BookmarkSync,
bot,
Builder,
ccubee,
cfetch,
CFNetwork,
CMC,
collector,
complex_network_group,
Contact,
crawl,
Creep,
Digital*Integrity,
Directory,
dogpile,
Excite,
EZResult,
FavOrg,
FeedDemon,
Feedreader,
FeedValidator,
Ferret,
fido,
find,
Fireball,
gazz,
GetRight,
gonzo,
Google-Sitemaps,
GoogleBot,
grab,
griffon,
Gromit,
Gulliver,
H?m?h?kki,
heritrix,
HTTrack,
Harvest,
holmes,
HTMLDOC,
Hubater,
IncyWincy,
index,
INGRID,
Jack,
JPluck,
KIT*Fireball,
Kototoi,
larbin,
Leech,
legs,
locator,
LWP,
Lycos,
marvin,
Mediapartners,
MegaSheep,
MEGAUPLOAD,
Mercator,
MFC_Tear_Sample,
Microsoft Data Access,
Microsoft Office,
Microsoft URL Control,
Microsoft-WebDAV,
MimeLive,
mirago,
Miva,
moget,
MSFrontPage,
Nazilla,
NetMechanic,
NetScoop,
newscan,
Nutch,
Ocelli,
ozelot,
ozzie,
pagebull,
ParaSite,
pavuk,
POE-Component,
Pokey,
Pompos,
Refiner,
retrieve,
RoboDude,
Rover,
Rssbandit,
RSSOwl,
Rutgers,
Scooter,
search,
seek,
shelob,
silk,
Slurp,
sna,
Snappy,
Snoopy,
speedy,
spider,
Spyder,
suke,
Susie,
swish,
T-H-U-N-D-E-R-S-T-O-N-E,
tarantula,
topiclink,
Toutatis,
TurnitinBot,
Tv*Merc,
Twiceler,
urllib,
VB Project,
Valkyrie,
Voyager,
W3C_Validator,
Walker,
wget,
WhizBang,
whowhere,
Wiki,
WinInet,
winona,
Wire,
Wombat,
WordPress,
worm,
wwwster,
WWW-Mechanize,
xtreme,
Yahoo,
Yandex,
Zeus,
ZyBorg,
EOR
RobotIP <<EOR
202.9.155.123,
204.152.191.41,
208.146.26.19,
208.146.26.233,
209.185.141.209,
209.185.141.211,
209.202.148.36,
209.202.148.41,
216.200.130.207,
216.35.103.6?,
216.35.103.70,
82.99.30.?, # Munax
82.99.30.1?, # Munax
82.99.30.2?, # Munax
82.99.30.3?, # Munax
82.99.30.4?, # Munax
82.99.30.5?, # Munax
82.99.30.6?, # Munax
82.99.30.70, # Munax
82.99.30.71, # Munax
82.99.30.72, # Munax
82.99.30.73, # Munax
EOR
RobotHost <<EOR
*.analys.google.com,
*.ask.com,
*.crawler*.com,
*.csccorporatedomains.com,
*.excite.com,
*.googlebot.com,
*.infoseek.com,
*.inktomi.com,
*.inktomisearch.com,
*.lycos.com,
*.pa-x.dec.com,
*.phx.gbl,
*.search.live.com,
add-url.altavista.com,
msnbot.msn.com,
westinghouse-rsl-com-usa.NorthRoyalton.cw.net,
EOR
Jump to Line
Something went wrong with that request. Please try again.