Permalink
Browse files

#299582 by hass: Remove outdated items from robots.txt and fix orderi…

…ng of items to make stuff easier to find.
  • Loading branch information...
1 parent f96a1b6 commit c5043af4cf2730ddcf6ea0005ebae1e81ca02681 goba committed Dec 10, 2008
Showing with 9 additions and 11 deletions.
  1. +9 −11 robots.txt
View
@@ -1,4 +1,4 @@
-# $Id: robots.txt,v 1.9 2007-06-27 22:37:44 goba Exp $
+# $Id: robots.txt,v 1.9.2.1 2008-12-10 20:12:19 goba Exp $
#
# robots.txt
#
@@ -20,27 +20,25 @@
User-agent: *
Crawl-delay: 10
# Directories
-Disallow: /database/
Disallow: /includes/
Disallow: /misc/
Disallow: /modules/
+Disallow: /profiles/
+Disallow: /scripts/
Disallow: /sites/
Disallow: /themes/
-Disallow: /scripts/
-Disallow: /updates/
-Disallow: /profiles/
# Files
-Disallow: /xmlrpc.php
+Disallow: /CHANGELOG.txt
Disallow: /cron.php
-Disallow: /update.php
-Disallow: /install.php
-Disallow: /INSTALL.txt
Disallow: /INSTALL.mysql.txt
Disallow: /INSTALL.pgsql.txt
-Disallow: /CHANGELOG.txt
-Disallow: /MAINTAINERS.txt
+Disallow: /install.php
+Disallow: /INSTALL.txt
Disallow: /LICENSE.txt
+Disallow: /MAINTAINERS.txt
+Disallow: /update.php
Disallow: /UPGRADE.txt
+Disallow: /xmlrpc.php
# Paths (clean URLs)
Disallow: /admin/
Disallow: /comment/reply/

0 comments on commit c5043af

Please sign in to comment.