fix(moderation): narrow the off-scope criterion to curb false positives#29
Merged
Conversation
The off-scope rule ("a heavy task clearly unrelated to a data platform,
e.g. generate a large program") over-fired on legitimate detailed/technical
work, notably sub-agent task delegations, blocking them as off-platform.
Reframe the moderation prompt around the actual abuse target: using the
assistant as a free general-purpose tool unrelated to the platform's data.
Data exploration/analysis/visualization, working with dataset/file content,
small scripts that consume the platform's data or API, and delegated
sub-agent tasks now stay in scope even when technical; writing substantial
general-purpose software stays out; the ambiguous middle resolves to allow.
Update the unit test and architecture doc to match.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rework the moderation gate's system prompt so its off-scope criterion
targets genuine misuse rather than any "heavy task."
What changed:
(e.g. generate a large program)", which over-fired on legitimate
detailed/technical work — notably sub-agent task delegations.
general-purpose tool unrelated to the platform's data (general chatbot
use, essays, substantial general-purpose software).
content, small scripts that consume the platform's data or API, and
delegated sub-agent tasks stay in scope even when technical. Writing
general-purpose code stays out; the ambiguous middle resolves to allow.
architecture doc.
Why: false-positive blocks were hitting sub-agents running targeted
technical/data tasks; the gate was too strict on legitimate detailed work.
Regression risks:
this widens the allow-set platform-wide, not just for sub-agent traffic.
provenance (modelRole is attacker-controlled, so it intentionally doesn't
relax the policy). The small-script-vs-general-software line is a
text-only judgment — expect some leakage in both directions.
not covered by CI; only the prompt wording is asserted.