Add robots.txt file to prevent indexing outdated docs

Cherry-picked from 2e37935e73b629fd9ecaabf958943cabc3ac825 and follow-ups.
godotengine · Mar 16, 2020 · eb3fda0 · eb3fda0
1 parent d2d8659
commit eb3fda0
Showing 1 changed file with 7 additions and 0 deletions.
diff --git a/robots.txt b/robots.txt
@@ -0,0 +1,7 @@
+user-agent: *
+disallow: /*/3.2
+disallow: /*/3.1
+disallow: /*/3.0
+disallow: /*/2.1
+
+sitemap: https://docs.godotengine.org/sitemap.xml