Moderation provider docs #2619

owenpearson · 2025-05-19T17:25:15Z

don't merge until website/control API have been updated

Adds docs for the new tisane and bodyguard moderation providers, as implemented in https://github.com/ably/realtime/pull/7151 and https://github.com/ably/realtime/pull/7414 respectively.

https://ably.atlassian.net/browse/CHA-994

coderabbitai · 2025-05-19T17:25:22Z

Important

Review skipped

Auto reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

Review comments: Directly reply to a review comment made by CodeRabbit. Example:
- I pushed a fix in commit <commit_id>, please review it.
- Explain this complex logic.
- Open a follow-up GitHub issue for this discussion.
Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
- @coderabbitai explain this code block.
- @coderabbitai modularize this function.
PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
- @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
- @coderabbitai read src/utils.ts and explain its main purpose.
- @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
- @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

@coderabbitai pause to pause the reviews on a PR.
@coderabbitai resume to resume the paused reviews.
@coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
@coderabbitai full review to do a full review from scratch and review all the files again.
@coderabbitai summary to regenerate the summary of the PR.
@coderabbitai generate docstrings to generate docstrings for this PR.
@coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
@coderabbitai resolve resolve all the CodeRabbit review comments.
@coderabbitai configuration to show the current CodeRabbit configuration for the repository.
@coderabbitai help to get help.

Other keywords and placeholders

Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (`.coderabbit.yaml`)

You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
Please see the configuration documentation for more information.
If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

Visit our Documentation for detailed information on how to use CodeRabbit.
Join our Discord Community to get help, request features, and share feedback.
Follow us on X/Twitter for updates and announcements.

AndyTWF · 2025-06-09T08:29:25Z

src/pages/docs/chat/moderation/direct/bodyguard.mdx

+
+The following are the fields specific to Bodyguard configuration:
+
+| Field | Description |


We've now got 4 rules that will have a lot of field crossover (the before publish rule config). Worth extracting the common fields to an overview page (like we have for "custom") so that we don't have to repeat ourselves on every page (and just have a callout to that page on the individual provider pages)?

good point! have extracted them all out now, see f0dbdf0

src/pages/docs/chat/moderation/direct/bodyguard.mdx

src/pages/docs/chat/moderation/direct/tisane.mdx

AndyTWF · 2025-07-10T11:13:12Z

src/pages/docs/chat/moderation/index.mdx

@@ -37,12 +49,26 @@ Alternatively, you might have a custom solution you wish to integrate with, or a

 [Hive](https://hivemoderation.com) provide automated content moderation solutions. The first of these is the [model only](/docs/chat/moderation/direct/hive-model-only) solution, which provides access to a powerful ML model that takes content and categorises it against various criteria, for example, violence or hate speech. For each classification, it also provides an indication of the severity of the infraction. Using this information, you can determine what level of classification is appropriate for your chat room and filter / reject content accordingly. Hive offer free credits to allow you to experiment with this solution.

-The second solution is the [dashboard](/docs/chat/moderation/direct/hive-dashboard). This is an all-in-one moderation tool, that allows you to combine automated workflows using ML models as well as human review and decisions to control the content in your chat room. 
+The second solution is the [dashboard](/docs/chat/moderation/direct/hive-dashboard). This is an all-in-one moderation tool, that allows you to combine automated workflows using ML models as well as human review and decisions to control the content in your chat room.


Not this PR but a quick fix - there's a few links in the direct vs custom section that are broken

will fix in a separate PR!

AndyTWF · 2025-07-10T11:13:53Z

src/pages/docs/chat/moderation/index.mdx

@@ -19,6 +19,18 @@ When using before publish moderation, a message is reviewed by an automated mode

 This approach provides additional safety guarantees, but may come at the cost of a small amount of latency, as messages must be vetted prior to being published.

+#### Common Configuration Fields <a id="common-config"/>


We also have this table in the index.mdx for before publish lambda - so we should probably lift this here too, or have the fields once each in custom and provider

ah yes good idea, have de-duped now 👍

Sentence case for headings please 🙂

m-hulbert

Mostly a few suggestions to improve readability, and one question. Looks good other than that.

src/pages/docs/chat/moderation/direct/bodyguard.mdx

m-hulbert · 2025-07-11T10:43:23Z

src/pages/docs/chat/moderation/direct/bodyguard.mdx

+Messages will be rejected if Bodyguard's analysis returns a `REMOVE` recommended action based on the moderation rules configured in your Bodyguard channel.
+
+## Handling rejections <a id="rejections"/>
+
+If a message fails moderation and the rule policy is to reject, then it will be rejected by the server.


These seem to be saying sort of the same thing in different words?

src/pages/docs/chat/moderation/direct/tisane.mdx

m-hulbert · 2025-07-11T10:47:53Z

src/pages/docs/chat/moderation/direct/tisane.mdx

+
+## Handling rejections <a id="rejections"/>
+
+If a message fails moderation and the rule policy is to reject, then it will be rejected by the server.


Same question for Bodyguard, but this suggests that you set up a policy as to what happens when a message fails moderation. But to me it looks like we reject if it's at, or above the threshold, and this isn't any additional config to do something other than reject.

yeah good point, this was misleading. have updated to be more correct + explicit

m-hulbert · 2025-07-11T10:48:19Z

src/pages/docs/chat/moderation/index.mdx

@@ -19,6 +19,18 @@ When using before publish moderation, a message is reviewed by an automated mode

 This approach provides additional safety guarantees, but may come at the cost of a small amount of latency, as messages must be vetted prior to being published.

+#### Common Configuration Fields <a id="common-config"/>


Sentence case for headings please 🙂

src/pages/docs/chat/moderation/index.mdx

Co-authored-by: Mark Hulbert <39801222+m-hulbert@users.noreply.github.com>

m-hulbert · 2025-07-11T15:23:24Z

src/pages/docs/chat/moderation/direct/bodyguard.mdx

+Messages will be rejected if Bodyguard's analysis returns a `REMOVE` recommended action based on the moderation rules configured in your Bodyguard channel.
+
+## Handling rejections <a id="rejections"/>
+
+If a message fails moderation the message will not be published and the publish request will be rejected.


I think these two sentences have a similar level of confusion as Tisane's did. Worth re-wording?

Could you explain what's confusing about them?

owenpearson requested a review from AndyTWF May 19, 2025 17:25

ably-ci temporarily deployed to ably-docs-moderation-pr-vem3fw May 19, 2025 17:25 Inactive

AndyTWF reviewed Jun 9, 2025

View reviewed changes

matt423 temporarily deployed to ably-docs-moderation-pr-kcyvj3 June 12, 2025 14:12 Inactive

owenpearson closed this Jun 26, 2025

owenpearson reopened this Jun 26, 2025

ably-ci temporarily deployed to ably-docs-moderation-pr-ydqvfj June 26, 2025 16:38 Inactive

owenpearson force-pushed the moderation-provider-docs branch from 27f0fbc to f0dbdf0 Compare July 9, 2025 16:29

owenpearson marked this pull request as ready for review July 9, 2025 16:29

owenpearson requested review from m-hulbert and AndyTWF July 9, 2025 16:29

AndyTWF added the review-app Create a Heroku review app label Jul 10, 2025

ably-ci temporarily deployed to ably-docs-moderation-pr-fganz2 July 10, 2025 10:57 Inactive

AndyTWF reviewed Jul 10, 2025

View reviewed changes

owenpearson force-pushed the moderation-provider-docs branch from f0dbdf0 to b915cd8 Compare July 10, 2025 14:09

owenpearson requested a review from AndyTWF July 10, 2025 14:09

ably-ci temporarily deployed to ably-docs-moderation-pr-fganz2 July 10, 2025 14:10 Inactive

AndyTWF approved these changes Jul 10, 2025

View reviewed changes

m-hulbert reviewed Jul 11, 2025

View reviewed changes

ably-ci temporarily deployed to ably-docs-moderation-pr-fganz2 July 11, 2025 14:48 Inactive

owenpearson and others added 4 commits July 11, 2025 15:50

chat/moderation: add tisane rule docs

7f21afa

chat/moderation: add bodyguard rule docs

fed2af2

chat/moderation: extract common fields for before-publish rules

cda6da1

Apply suggestions from code review

edd76f6

Co-authored-by: Mark Hulbert <39801222+m-hulbert@users.noreply.github.com>

owenpearson force-pushed the moderation-provider-docs branch from 230772e to edd76f6 Compare July 11, 2025 14:50

owenpearson requested a review from m-hulbert July 11, 2025 14:51

ably-ci deployed to ably-docs-moderation-pr-fganz2 July 11, 2025 14:51 View deployment

m-hulbert reviewed Jul 11, 2025

View reviewed changes


		The following are the fields specific to Bodyguard configuration:

		\| Field \| Description \|

		@@ -19,6 +19,18 @@ When using before publish moderation, a message is reviewed by an automated mode

		This approach provides additional safety guarantees, but may come at the cost of a small amount of latency, as messages must be vetted prior to being published.

		#### Common Configuration Fields <a id="common-config"/>


		## Handling rejections <a id="rejections"/>

		If a message fails moderation and the rule policy is to reject, then it will be rejected by the server.

Moderation provider docs #2619

Are you sure you want to change the base?

Moderation provider docs #2619

Uh oh!

Conversation

owenpearson commented May 19, 2025

Uh oh!

coderabbitai bot commented May 19, 2025

Review skipped

Chat

Support

CodeRabbit Commands (Invoked using PR comments)

Other keywords and placeholders

CodeRabbit Configuration File (.coderabbit.yaml)

Documentation and Community

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

m-hulbert left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

CodeRabbit Configuration File (`.coderabbit.yaml`)