Permalink
Browse files

#111 adding robots.txt

  • Loading branch information...
1 parent b7ec5c6 commit 49837c30d66543c3a2817e03c87c10c2d16316c9 Paul Kinlan committed Nov 17, 2011
Binary file not shown.
@@ -0,0 +1,5 @@
+# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file
+#
+# To ban all spiders from the entire site uncomment the next two lines:
+# User-Agent: *
+# Disallow: /
@@ -0,0 +1,5 @@
+# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file
+#
+# To ban all spiders from the entire site uncomment the next two lines:
+# User-Agent: *
+# Disallow: /

0 comments on commit 49837c3

Please sign in to comment.