Permalink
Browse files

fix robots txt

  • Loading branch information...
1 parent 7229be7 commit 534e1cd891889af2113eb4155f472e2a24d443a8 progressbarsk committed Sep 8, 2012
Showing with 16 additions and 2 deletions.
  1. +16 −2 public/robots.txt
View
@@ -1,5 +1,19 @@
# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file
#
# To ban all spiders from the entire site uncomment the next two lines:
-# User-Agent: *
-# Disallow: /
+
+User-Agent: *
+
+Disallow: /refinery/
+Disallow: /admin/
+Disallow: /member/
+Disallow: /search
+Disallow: /users/
+Disallow: /en/admin/
+Disallow: /en/refinery/
+Disallow: /en/member/
+Disallow: /en/search
+Disallow: /en/users/
+
+Disallow: /events/category/
+Disallow: /en/events/category/

0 comments on commit 534e1cd

Please sign in to comment.