Add administrative webhooks #18510

Gargron · 2022-05-25T19:30:48Z

Webhooks would benefit automation through the moderation API by notifying applications about system events in real-time. They would also enable integrations with chat apps like Discord, IRC and Slack, helping moderator coordination.

Events currently supported:

report.created
account.created

The X-Hub-Signature header adopted from the WebSub spec can be optionally used to verify that the payloads are authentic.

The payload delivered to the webhook has the form of:

{
  "event": "report.created",
  "created_at": "2022-06-08T12:28:31.000Z",
  "object": {
    // The report entity would be here
  }
}

Mind that created_at at the top-level refers to the event itself, and not the object.

This project was funded through the NGI0 Discovery Fund, a fund established by NLnet with financial support from the European Commission's Next Generation Internet programme, under the aegis of DG Communications Networks, Content and Technology under grant agreement No 825322.

ClearlyClaire

Looks good to me overall, though I made some minor inline comments.

The list of implemented events is pretty limited, but there is no issue with that being extended in subsequent PRs.

ClearlyClaire · 2022-06-08T12:39:51Z

app/services/base_service.rb

+  def call(*)
+    raise NotImplementedError
+  end


Just code quality, as I see it, defining the role of BaseService more clearly...

ClearlyClaire · 2022-06-08T13:28:25Z

app/workers/webhooks/delivery_worker.rb

+  sidekiq_options queue: 'push', retry: 16, dead: false
+
+  def perform(webhook_id, body)
+    @webhook   = Webhook.find(webhook_id)


Maybe use find_by or catch ActiveRecord::RecordNotFound?

ashleyhull-versent · 2022-06-09T13:15:19Z

Brainstormed additional events:

report.resolved
report.sensitive
report.deleted
account.warn
account.freeze
account.forcesensitive
account.limit
account.suspend
account.memoriam
instance.silenced
instance.suspend
announcements.enabled

Other points of interest

sidekiq.queue.latency (over X value)

* Add administrative webhooks * Fix error when webhook is deleted before delivery worker runs

wioxjk · 2022-11-20T17:55:21Z

I tried to get it to work with the Webhook-functionality provided by t2boit.io - in order to get notifications into a matrix-room.
However, it did not work. The suspicion is that this webhook is not "slack compatible" - is that right?

Saiv46 · 2023-01-04T20:41:04Z

If you wanted to make webhooks and stumbled upon this PR, since this is undocumented:

Webhooks are HTTP URLs for sending data of selected events, such as user registration, so it can be processed by other service.

When triggered by some event, instance will make HTTP POST request to configured URL with body with application/json encoding.
{
  "event": "account.created",
  "created_at": "2023-01-04T17:37:53.374Z",
  "object": // Admin::Account
}
JSON body includes event name (event), event creation time (created_at) and object that is related to event (in case of account.created - an instance of Admin::Account).

Recieved response body will be ignored and only status code used:

2XX-3XX - Request successful.

401, 408, 429 - Request will be retried (max 16).

4XX-5XX - Request failed, no further attempts.

To be able to validate request payloads, X-Hub-Signature header can be used, which will be in a form of method=signature where method is one of the recognized algorithm names (SHA1, SHA256, SHA384, SHA512) and signature is the hexadecimal representation of the signature. Signature is computed using request payload as data and signing secret as key.

* Add administrative webhooks * Fix error when webhook is deleted before delivery worker runs

kevquirk · 2023-10-25T20:24:06Z

Is there a facility to reference objects like report number from a webhook?

carrotcypher · 2023-10-25T22:07:50Z

This lacking documentation was an expected head scratcher for a moment. For the benefit of anyone else coming to this page and scratching their heads, here's how we go it working for Mastodon reports > Mattermost mod channel notifications for us at Fosstodon.org.

1) Create the new webhook endpoint on Mattermost

2) Create the new webhook event on Mastodon

For our needs, we only wanted it triggered for the report.created event.

3) Change the "Payload Template" of the Mastodon webhook event to:

{ "text": "Here is the text to send, and you can reference values with {{object.value}} depending on its location in the JSON tree (see below)" }

While you can refer to the Reports API JSON schema for some of the individual variables you are looking for (https://docs.joinmastodon.org/methods/reports/), we set up a listener to view the raw data ourselves and here is the (heavily redacted / fluffed) data received:

{
   "event":"report.created",
   "created_at":"2023-10-26T13:34:00.351Z",
   "object":{
      "id":"8437",
      "action_taken":false,
      "action_taken_at":null,
      "category":"violation",
      "comment":"",
      "forwarded":true,
      "created_at":"2023-10-26T13:34:00.348Z",
      "updated_at":"2023-10-26T13:34:00.348Z",
      "account":{
         "id":"123456789",
         "username":"bobisaburger",
         "domain":null,
         "created_at":"2023-07-13T04:39:22.493Z",
         "email":"bobisaburger@emailservice.com",
         "ip":"12.34.56.78",
         "confirmed":true,
         "suspended":false,
         "silenced":false,
         "sensitized":false,
         "disabled":false,
         "approved":true,
         "locale":"en",
         "invite_request":"I would love to be a member of your instance!",
         "ips":[
            {
               "ip":"12.34.56.78",
               "used_at":"2023-07-13T04:45:31.835Z"
            },
            {
               "ip":"98.76.54.32",
               "used_at":"2023-07-13T04:39:22.722Z"
            }
         ],
         "account":{
            "id":"123456789",
            "username":"bobisaburger",
            "acct":"bobisaburger",
            "display_name":"bobisaburger",
            "locked":false,
            "bot":false,
            "discoverable":null,
            "group":false,
            "created_at":"2023-07-13T00:00:00.000Z",
            "note":"",
            "url":"https://mastodonwebsite/@bobisaburger",
            "uri":"https://mastodonwebsite/users/bobisaburger",
            "avatar":"https://locationofavatar.com/image.jpg",
            "avatar_static":"https://locationofavatar.com/image.jpg",
            "header":"locationofheader.com/image.jpg",
            "header_static":"locationofheader.com/image.jpg",
            "followers_count":100,
            "following_count":200,
            "statuses_count":9,
            "last_status_at":"2023-08-05",
            "noindex":true,
            "emojis":[
               
            ],
            "roles":[
               
            ],
            "fields":[
               
            ]
         },
         "role":{
            "id":"-99",
            "name":"",
            "permissions":"65536",
            "color":"",
            "highlighted":false
         }
      },
      "target_account":{
         "id":"123454321",
         "username":"cheeseperson",
         "domain":"someothermastodonsite.com",
         "created_at":"2023-08-20T00:00:00.000Z",
         "email":null,
         "ip":null,
         "confirmed":null,
         "suspended":false,
         "silenced":false,
         "sensitized":false,
         "disabled":null,
         "approved":null,
         "locale":null,
         "invite_request":null,
         "ips":null,
         "account":{
            "id":"123454321",
            "username":"cheeseperson",
            "acct":"cheeseperson@someothermastodonsite.com",
            "display_name":"cheeseperson",
            "locked":false,
            "bot":false,
            "discoverable":false,
            "group":false,
            "created_at":"2023-08-20T00:00:00.000Z",
            "note":"",
            "url":"https://someothermastodonsite.com/@cheeseperson",
            "uri":"https://someothermastodonsite.com/users/cheeseperson",
            "avatar":"https://someothermastadonsite.com/avatars/original/missing.png",
            "avatar_static":"https://someothermastadonsite.com/avatars/original/missing.png",
            "header":"locationofheader.com/image.jpg",
            "header_static":"locationofheader.com/image.jpg",
            "followers_count":2,
            "following_count":2,
            "statuses_count":95,
            "last_status_at":"2023-10-26",
            "emojis":[
               
            ],
            "fields":[
               
            ]
         },
         "role":null
      },
      "assigned_account":null,
      "action_taken_by_account":null,
      "statuses":[
         {
            "id":"12345678987654321",
            "created_at":"2023-10-26T11:29:13.000Z",
            "in_reply_to_id":"1918282746465",
            "in_reply_to_account_id":"101010101010",
            "sensitive":false,
            "spoiler_text":"",
            "visibility":"public",
            "language":"de",
            "uri":"https://someothermastodonsite.com/users/cheeseperson/statuses/111301083360371621",
            "url":"https://someothermastodonsite.com/@cheeseperson/111301083360371621",
            "replies_count":0,
            "reblogs_count":0,
            "favourites_count":0,
            "edited_at":"2023-10-26T11:30:31.000Z",
            "content":"<p>Here is some content</p>",
            "reblog":null,
            "account":{
               "id":"123454321",
               "username":"cheeseperson",
               "acct":"cheeseperson@someothermastodonsite.com",
               "display_name":"cheeseperson",
               "locked":false,
               "bot":false,
               "discoverable":false,
               "group":false,
               "created_at":"2023-08-20T00:00:00.000Z",
               "note":"",
               "url":"https://someothermastodonsite.com/@cheeseperson",
               "uri":"https://someothermastodonsite.com/users/cheeseperson",
               "avatar":"https://someothermastadonsite.com/avatars/original/missing.png",
               "avatar_static":"https://someothermastadonsite.com/avatars/original/missing.png",
               "header":"locationofheader.com/image.jpg",
               "header_static":"locationofheader.com/image.jpg",
               "followers_count":2,
               "following_count":2,
               "statuses_count":95,
               "last_status_at":"2023-10-26",
               "emojis":[
                  
               ],
               "fields":[
                  
               ]
            },
            "media_attachments":[
               
            ],
            "mentions":[
               {
                  "id":"101010101010",
                  "username":"thirdperson",
                  "url":"https://thirdpersonsinstance.com/@thirdperson",
                  "acct":"thirdperson@emailwebsite.com"
               }
            ],
            "tags":[
               
            ],
            "emojis":[
               
            ],
            "card":null,
            "poll":null
         }
      ],
      "rules":[
         {
            "id":"2",
            "text":"Don't be a meanie!"
         }
      ]
   }
}

For our case, we wanted the following message in the Mattermost channel on each new report:

@channel report 10415 was just filed for the user BobIsACat

To achieve that, we use the fields object.id and object.target_account.account.url as follows:

{ "text": "@channel report [{{object.id}}](https://ourmastadoninstance.com/reports/{{object.id}}) was just filed for the user [{{object.target_account.account.url}}]({{object.target_account.account.url}})" }

Hope this was helpful for someone to get started with at least!

carrotcypher · 2023-10-26T03:35:22Z

@Gargron is there also an event for when a mod leaves a note on the report? Perhaps report.updated or report.note?

Alternatively, are there plans for any incoming webhooks so that moderation could be done from a third party site/service (for example, in our case we would want to add a moderator note to the report from inside a Mattermost channel)?

ClearlyClaire · 2023-10-27T11:48:12Z

report.updated is triggered when the report itself is modified, but not when moderation notes are left on it. There is currently no webhook event for that.

Gargron added moderation Administration and moderation tooling NLnet Funded by NLnet, see https://nlnet.nl/project/Mastodon/ labels May 25, 2022

Gargron force-pushed the feature-admin-webhooks branch 8 times, most recently from 10d190e to 2ffb811 Compare June 7, 2022 21:56

Gargron marked this pull request as ready for review June 7, 2022 21:58

Add administrative webhooks

a4298da

Gargron force-pushed the feature-admin-webhooks branch from 2ffb811 to a4298da Compare June 7, 2022 22:24

ClearlyClaire approved these changes Jun 8, 2022

View reviewed changes

Fix error when webhook is deleted before delivery worker runs

a8d22c3

ClearlyClaire approved these changes Jun 9, 2022

View reviewed changes

Gargron merged commit a2871cd into main Jun 9, 2022

Gargron deleted the feature-admin-webhooks branch June 9, 2022 20:01

shleeable mentioned this pull request Oct 31, 2022

webhook account.created needs a account.confirmed #19577

Closed

kadoshita pushed a commit to kadoshita/mastodon that referenced this pull request Nov 19, 2022

Add administrative webhooks (mastodon#18510)

b247db9

* Add administrative webhooks * Fix error when webhook is deleted before delivery worker runs

mattwynne mentioned this pull request Dec 23, 2022

Onboarding email nelson-social/community#13

Draft

This was referenced Jan 4, 2023

Add webhook account.approved #22938

Merged

Webhooks undocumented mastodon/documentation#1142

Open

trwnh mentioned this pull request Apr 30, 2023

currently undocumented mastodon/documentation#1211

Open

74 tasks

ClearlyClaire pushed a commit to ClearlyClaire/mastodon that referenced this pull request Jul 6, 2023

Add administrative webhooks (mastodon#18510)

b0aff56

* Add administrative webhooks * Fix error when webhook is deleted before delivery worker runs

carrotcypher mentioned this pull request Oct 27, 2023

Proposal for adding minimal documentation for undocumented moderation API (webhook) mastodon/documentation#1329

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add administrative webhooks #18510

Add administrative webhooks #18510

Gargron commented May 25, 2022 •

edited

Loading

ClearlyClaire left a comment

ClearlyClaire Jun 8, 2022

Gargron Jun 9, 2022

ClearlyClaire Jun 8, 2022

ashleyhull-versent commented Jun 9, 2022 •

edited

Loading

wioxjk commented Nov 20, 2022

Saiv46 commented Jan 4, 2023

kevquirk commented Oct 25, 2023

carrotcypher commented Oct 25, 2023 •

edited

Loading

carrotcypher commented Oct 26, 2023 •

edited

Loading

ClearlyClaire commented Oct 27, 2023

Add administrative webhooks #18510

Add administrative webhooks #18510

Conversation

Gargron commented May 25, 2022 • edited Loading

ClearlyClaire left a comment

Choose a reason for hiding this comment

ClearlyClaire Jun 8, 2022

Choose a reason for hiding this comment

Gargron Jun 9, 2022

Choose a reason for hiding this comment

ClearlyClaire Jun 8, 2022

Choose a reason for hiding this comment

ashleyhull-versent commented Jun 9, 2022 • edited Loading

wioxjk commented Nov 20, 2022

Saiv46 commented Jan 4, 2023

kevquirk commented Oct 25, 2023

carrotcypher commented Oct 25, 2023 • edited Loading

1) Create the new webhook endpoint on Mattermost

2) Create the new webhook event on Mastodon

3) Change the "Payload Template" of the Mastodon webhook event to:

carrotcypher commented Oct 26, 2023 • edited Loading

ClearlyClaire commented Oct 27, 2023

Gargron commented May 25, 2022 •

edited

Loading

ashleyhull-versent commented Jun 9, 2022 •

edited

Loading

carrotcypher commented Oct 25, 2023 •

edited

Loading

carrotcypher commented Oct 26, 2023 •

edited

Loading