Beyond moderation: labelers for security/safe browsing, existing services to use

[snarfed]

So far, much of the conversation around third party labelers has focused on content moderation. That's an important first use case, but obviously not the only one.

We talked a bit about the downsides and "exploits" of users constructing their own link preview cards in #general on Discord today, and considered where to put logic for detecting and flagging misleading link previews. The options seem to be:

• PDSes
• clients
• AppViews
• labelers

I've been thinking a bit about the labeler approach, and I like it, for all the obvious reasons. Opens it up to independent developers, doesn't require all PDSes and clients to implement it themselves, lets users opt in and out independent of their PDS/client, etc.

That led me to think about building labelers that are just thin wrappers around mature existing services for detecting spam, malware, phishing, disinformation, etc. Lots of those to choose from! Here are just a few examples:

• https://akismet.com/
• https://www.spamhaus.org/
• https://www.dnsbl.info/
• https://support.mozilla.org/en-US/kb/how-does-phishing-and-malware-protection-work
• https://learn.microsoft.com/en-us/deployedge/microsoft-edge-security-smartscreen
• https://developers.google.com/safe-browsing/
• https://jigsaw.google.com/issues/
• https://www.greynoise.io/
  (and tons more security/threat intel companies...)
• https://www.snopes.com/
• https://toolbox.google.com/factcheck
• https://www.politico.com/interactives/2018/is-this-true/
• https://www.rand.org/research/projects/truth-decay/fighting-disinformation/search.html

[bnewbold]

Yup! These are all good thoughts.

We definitely envision rules-based and graph-based services to flag spam and misleading accounts. Some of those might be atproto-specific, others could build on these many existing services.

A couple related protocol-level pieces that haven't been specified yet:

• ability to label infrastructure, like an entire PDS instance. Labels allow a URI, and we could put DNS names, URLs, or IP addresses in there. Alternatively, we could lean in to a model where every PDS (or other infra service) has a DID associated with it, and we would label that DID instead
• ability to flag content and accounts without generating a specific label. We are thinking through an iteration of the admin/moderation endpoints which would allow creating arbitrary private "events" on subjects, with metadata. This would enable things like automated flagging of an account for human review, without broadcasting a label with the flag value first.
• labels as currently envisioned are probably not a good fit for something like Twitter's Community Notes feature, or something like snopes, which provide additional context. It would be good to support systems like that. My intuition is that it will probably be application-specific (aka, in the app.bsky Lexicons, not com.atproto). Labels might be part of the design, eg, a label indicating "a disclaimer is available elsewhere according to this party".