Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Browse files

added robots.txt

  • Loading branch information...
commit 3aae5aa8c3c74029dfe60a0f02007cf7f886591d 1 parent 8a9fa63
@barbie authored
Showing with 15 additions and 1 deletion.
  1. +0 −1  MANIFEST
  2. +15 −0 vhost/html/robots.txt
View
1  MANIFEST
@@ -108,7 +108,6 @@ vhost/html/css/layout2_text.css
vhost/html/favicon.ico
vhost/html/images/blank.png
vhost/html/images/icons/feed-icon-large.png
-vhost/html/images/icons/feed-icon-small.png
vhost/html/images/icons/feed-icon-small.gif
vhost/html/images/icons/technorati-fav-button.png
vhost/html/images/layout/banner-blog.png
View
15 vhost/html/robots.txt
@@ -0,0 +1,15 @@
+User-agent: Googlebot
+Crawl-delay: 600
+Disallow: /private/
+
+User-agent: msnbot
+Crawl-delay: 600
+Disallow: /private/
+
+User-agent: DotBot
+Crawl-delay: 600
+Disallow: /private/
+
+User-agent: *
+Crawl-delay: 600
+Disallow: /private/

0 comments on commit 3aae5aa

Please sign in to comment.
Something went wrong with that request. Please try again.