Rollup merge of #59930 - kornelski:first-robots, r=GuillaumeGomez

Exclude some copies of old book editions from search engines These are only stubs that confuse search engine users. There's no useful content in these locations.
rust-lang · Apr 13, 2019 · b613fd2 · b613fd2
2 parents 33561e2 + 0a9b214
commit b613fd2
Showing 1 changed file with 8 additions and 0 deletions.
diff --git a/src/doc/robots.txt b/src/doc/robots.txt
@@ -17,3 +17,11 @@ Disallow: /1.0.0-beta.2/
 Disallow: /1.0.0-beta.3/
 Disallow: /1.0.0-beta.4/
 Disallow: /1.0.0-beta.5/
+Disallow: /book/first-edition/
+Disallow: /book/second-edition/
+Disallow: /stable/book/first-edition/
+Disallow: /stable/book/second-edition/
+Disallow: /beta/book/first-edition/
+Disallow: /beta/book/second-edition/
+Disallow: /nightly/book/first-edition/
+Disallow: /nightly/book/second-edition/