docs: added robots.txt

Unleash · Jan 9, 2024 · b8231e6 · b8231e6
1 parent fbeb542
commit b8231e6
Showing 1 changed file with 9 additions and 0 deletions.
diff --git a/website/static/robots.txt b/website/static/robots.txt
@@ -0,0 +1,9 @@
+# *
+User-agent: *
+Allow: /
+
+# Host
+Host: https://docs.getunleash.io
+
+# Sitemaps
+Sitemap: https://docs.getunleash.io/sitemap.xml