From 3cdd06edf7745925d7139a94d3f1c123c49a5231 Mon Sep 17 00:00:00 2001
From: Andy Piper <andypiper@users.noreply.github.com>
Date: Sun, 21 Jan 2024 22:37:25 +0000
Subject: [PATCH] Create robots.txt

Disallow well-behaved robots from poking around in most places.
---
 public/robots.txt | 17 +++++++++++++++++
 1 file changed, 17 insertions(+)
 create mode 100644 public/robots.txt

diff --git a/public/robots.txt b/public/robots.txt
new file mode 100644
index 0000000..8534007
--- /dev/null
+++ b/public/robots.txt
@@ -0,0 +1,17 @@
+# See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file
+
+User-agent: GPTBot
+Disallow: /
+
+User-agent: *
+Disallow: /api/inbox
+Disallow: /admin
+Disallow: /bookmark/
+Disallow: /comment
+Disallow: /login
+Disallow: /logout
+Disallow: /m/
+Disallow: /network
+Disallow: /search
+Disallow: /tagged/
+Disallow: /u/