Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Labeling & moderation updates to SDK #1366

Merged
merged 33 commits into from
Aug 3, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
1f3b455
First pass on label definitions and reference-doc generation for them
pfrazee Jul 19, 2023
39f1365
Tune labels
pfrazee Jul 20, 2023
6bddab2
Drop the 'csam' label in favor of using !hide
pfrazee Jul 20, 2023
136373a
Add label definition code generation
pfrazee Jul 20, 2023
ff06052
Implement all type signatures for moderation
pfrazee Jul 20, 2023
9f33c03
Complete a first pass on the labeling sdk
pfrazee Jul 21, 2023
91c2e05
Add post-moderation test suite
pfrazee Jul 26, 2023
7e62919
Add post moderation behavior documentation
pfrazee Jul 26, 2023
af83028
Add self-post behaviors
pfrazee Jul 26, 2023
7ac61d5
Improve post moderation doc
pfrazee Jul 26, 2023
730701d
Tune up the post moderation doc
pfrazee Jul 26, 2023
da65b7a
Tune up the post moderation doc
pfrazee Jul 26, 2023
37abbb8
Tune up the post moderation doc
pfrazee Jul 26, 2023
413e27d
Simplify the post moderation behavior descriptions
pfrazee Jul 26, 2023
5ca6f1f
More behavior description tuneup
pfrazee Jul 26, 2023
bc7e542
Add profile moderation final behaviors and tests
pfrazee Jul 27, 2023
1cbf0d6
Improve generated post-moderation-behaviors doc
pfrazee Jul 27, 2023
8f06c4d
Add profile moderation behaviors doc
pfrazee Jul 27, 2023
e7a00b7
Test muted-by-list specifically
pfrazee Jul 27, 2023
fe3c173
Fixes to label descriptions
pfrazee Jul 27, 2023
859b990
Fix to muted-by-list behavior
pfrazee Jul 27, 2023
5ab209c
Dont blur account in moderateProfile() when muting
pfrazee Jul 28, 2023
10c4f05
Tune label copy
pfrazee Jul 28, 2023
2e2d626
Apply post embed blurring when account is marked with blurmedia label
pfrazee Jul 28, 2023
46c7cca
Fix output signature
pfrazee Jul 28, 2023
1bb3661
Fixes to blocking behavior
pfrazee Jul 28, 2023
182d20c
Rename LabelDefinitionPreference to LabelPreference
pfrazee Jul 28, 2023
a8c30f9
Update docs
pfrazee Jul 28, 2023
edc49ca
Fix test
pfrazee Jul 28, 2023
dbe4101
Fix: self-harm should blur media only
pfrazee Jul 28, 2023
85c224d
Fixes to tests
pfrazee Aug 2, 2023
54c8646
Improve label copy
pfrazee Aug 2, 2023
2863240
Remove all labels that do not have a specific policy
pfrazee Aug 2, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
81 changes: 81 additions & 0 deletions packages/api/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -160,6 +160,87 @@ console.log(rt3.length) // => 25
console.log(rt3.graphemeLength) // => 1
```

### Moderation
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there an argument for putting this in a separate package?

we use @atproto/api in a the pds/appview for instance and may not want to pull in the whole labeling sdk

don't think this is block but could imagine an sdk package that is comprised of api & labels or similar

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe but I'm not inclined to do it yet


Applying the moderation system is a challenging task, but we've done our best to simplify it for you. The Moderation API helps handle a wide range of tasks, including:

- User muting (including mutelists)
- User blocking
- Moderator labeling

For more information, see the [Moderation Documentation](./docs/moderation.md) or the associated [Labels Reference](./docs/labels.md).

```typescript
import {moderatePost, moderateProfile} from '@atproto/api'

// We call the appropriate moderation function for the content
// =

const postMod = moderatePost(postView, getOpts())
const profileMod = moderateProfile(profileView, getOpts())

// We then use the output to decide how to affect rendering
// =

if (postMod.content.filter) {
// dont render in feeds or similar
// in contexts where this is disruptive (eg threads) you should ignore this and instead check blur
}
if (postMod.content.blur) {
// render the whole object behind a cover (use postMod.content.cause to explain)
if (postMod.content.noOverride) {
// do not allow the cover the be removed
}
}
if (postMod.content.alert) {
// render a warning on the content (use postMod.content.cause to explain)
}
if (postMod.embed.blur) {
// render the embedded media behind a cover (use postMod.embed.cause to explain)
if (postMod.embed.noOverride) {
// do not allow the cover the be removed
}
}
if (postMod.embed.alert) {
// render a warning on the embedded media (use postMod.embed.cause to explain)
}
if (postMod.avatar.blur) {
// render the avatar behind a cover
}
if (postMod.avatar.alert) {
// render an alert on the avatar
}

// The options passed into `apply()` supply the user's preferences
// =

function getOpts() {
return {
// the logged-in user's DID
userDid: 'did:plc:1234...',

// is adult content allowed?
adultContentEnabled: true,

// the user's labeler settings
labelerSettings: [
{
labeler: {
did: '...',
displayName: 'My mod service'
},
settings: {
porn: 'hide',
sexual: 'warn',
nudity: 'ignore',
// ...
}
}
]
}
}
```

## Advanced

### Advanced API calls
Expand Down
212 changes: 212 additions & 0 deletions packages/api/definitions/labels.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,212 @@
[
{
"id": "system",
"configurable": false,
"labels": [
{
"id": "!hide",
"preferences": ["hide"],
"flags": ["no-override"],
"onwarn": "blur"
},
{
"id": "!no-promote",
"preferences": ["hide"],
"flags": [],
"onwarn": null
},
{
"id": "!warn",
"preferences": ["warn"],
"flags": [],
"onwarn": "blur"
}
]
},
{
"id": "legal",
"configurable": false,
"labels": [
{
"id": "dmca-violation",
"preferences": ["hide"],
"flags": ["no-override"],
"onwarn": "blur"
},
{
"id": "doxxing",
"preferences": ["hide"],
"flags": ["no-override"],
"onwarn": "blur"
}
]
},
{
"id": "sexual",
"configurable": true,
"labels": [
{
"id": "porn",
"preferences": ["ignore", "warn", "hide"],
"flags": ["adult"],
"onwarn": "blur-media"
},
{
"id": "sexual",
"preferences": ["ignore", "warn", "hide"],
"flags": ["adult"],
"onwarn": "blur-media"
},
{
"id": "nudity",
"preferences": ["ignore", "warn", "hide"],
"flags": ["adult"],
"onwarn": "blur-media"
}
]
},
{
"id": "violence",
"configurable": true,
"labels": [
{
"id": "nsfl",
"preferences": ["ignore", "warn", "hide"],
"flags": ["adult"],
"onwarn": "blur-media"
},
{
"id": "corpse",
"preferences": ["ignore", "warn", "hide"],
"flags": ["adult"],
"onwarn": "blur-media"
},
{
"id": "gore",
"preferences": ["ignore", "warn", "hide"],
"flags": ["adult"],
"onwarn": "blur-media"
},
{
"id": "torture",
"preferences": ["ignore", "warn", "hide"],
"flags": ["adult"],
"onwarn": "blur"
},
{
"id": "self-harm",
"preferences": ["ignore", "warn", "hide"],
"flags": ["adult"],
"onwarn": "blur-media"
}
]
},
{
"id": "intolerance",
"configurable": true,
"labels": [
{
"id": "intolerant-race",
"preferences": ["ignore", "warn", "hide"],
"flags": [],
"onwarn": "blur"
},
{
"id": "intolerant-gender",
"preferences": ["ignore", "warn", "hide"],
"flags": [],
"onwarn": "blur"
},
{
"id": "intolerant-sexual-orientation",
"preferences": ["ignore", "warn", "hide"],
"flags": [],
"onwarn": "blur"
},
{
"id": "intolerant-religion",
"preferences": ["ignore", "warn", "hide"],
"flags": [],
"onwarn": "blur"
},
{
"id": "intolerant",
"preferences": ["ignore", "warn", "hide"],
"flags": [],
"onwarn": "blur"
},
{
"id": "icon-intolerant",
"preferences": ["ignore", "warn", "hide"],
"flags": [],
"onwarn": "blur-media"
}
]
},
{
"id": "rude",
"configurable": true,
"labels": [
{
"id": "threat",
"preferences": ["ignore", "warn", "hide"],
"flags": [],
"onwarn": "blur"
}
]
},
{
"id": "curation",
"configurable": true,
"labels": [
{
"id": "spoiler",
"preferences": ["ignore", "warn", "hide"],
"flags": [],
"onwarn": "blur"
}
]
},
{
"id": "spam",
"configurable": true,
"labels": [
{
"id": "spam",
"preferences": ["ignore", "warn", "hide"],
"flags": [],
"onwarn": "blur"
}
]
},
{
"id": "misinfo",
"configurable": true,
"labels": [
{
"id": "account-security",
"preferences": ["ignore", "warn", "hide"],
"flags": [],
"onwarn": "blur"
},
{
"id": "net-abuse",
"preferences": ["ignore", "warn", "hide"],
"flags": [],
"onwarn": "blur"
},
{
"id": "impersonation",
"preferences": ["ignore", "warn", "hide"],
"flags": [],
"onwarn": "alert"
},
{
"id": "scam",
"preferences": ["ignore", "warn", "hide"],
"flags": [],
"onwarn": "alert"
}
]
}
]
38 changes: 38 additions & 0 deletions packages/api/definitions/locale/en/label-groups.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
{
"system": {
"name": "System",
"description": "Moderator overrides for special cases."
},
"legal": {
"name": "Legal",
"description": "Content removed for legal reasons."
},
"sexual": {
"name": "Adult Content",
"description": "Content which is sexual in nature."
},
"violence": {
"name": "Violence",
"description": "Content which is violent or deeply disturbing."
},
"intolerance": {
"name": "Intolerance",
"description": "Content or behavior which is hateful or intolerant toward a group of people."
},
"rude": {
"name": "Rude",
"description": "Behavior which is rude toward other users."
},
"curation": {
"name": "Curational",
"description": "Subjective moderation geared towards curating a more positive environment."
},
"spam": {
"name": "Spam",
"description": "Content which doesn't add to the conversation."
},
"misinfo": {
"name": "Misinformation",
"description": "Content which misleads or defrauds users."
}
}
Loading