Skip to content

fix(auto-reply): default room events to silence#99144

Merged
obviyus merged 2 commits into
mainfrom
codex/fix-ambient-prompt
Jul 2, 2026
Merged

fix(auto-reply): default room events to silence#99144
obviyus merged 2 commits into
mainfrom
codex/fix-ambient-prompt

Conversation

@obviyus

@obviyus obviyus commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Part of #99142

What Problem This Solves

On ambient room_event turns the model chimes into group chats far too often (~11% of ambient turns in a production deployment) because the prompt actively leans toward replying: the envelope's only directive is "Decide whether to act", the embedded delivery hint instructs "Use the message tool to send the final user-visible answer" (presupposing an answer should exist) and falsely claims status updates are shown (they are fully suppressed on room events), the strongest selectivity guidance ("Be extremely selective...") ships only in the legacy NO_REPLY mode and never in the recommended message-tool ambient config, the always-on group intro says "Address the specific sender" for every message, and the "post only when you have concrete value" etiquette bullet is Discord-only.

Why This Change Was Made

Default-silent needs to be stated where the model actually looks — the per-turn envelope — and every ambient surface should agree. The room-event envelope now carries an explicit default-silent contract; room-event turns get a dedicated delivery hint that expects no reply and makes no false delivery claims (non-room-event message-tool turns keep the existing hint); the selectivity line ships in message-tool ambient mode; the always-on intro is lurk-aware; the group etiquette bullet applies to any group/channel message-tool-only session. buildGroupIntro's dead params were removed along the way.

User Impact

Ambient-mode agents stop being nudged toward posting on every observed message. Operator instructions like "only reply when spoken to" stop fighting contradictory harness prompt text.

Evidence

  • New/updated assertions across prompt-prelude, inbound-meta, groups, strip-inbound-meta, and system-prompt tests: envelope contains the default-silent contract; room-event turns get the room-event hint while non-room-event message-tool turns keep the old one; ambient group prompts contain the selectivity line.
  • node scripts/run-vitest.mjs over the touched surfaces: 7 files, 379 tests + 91 prompt/media tests; prompt snapshot generator clean locally and verified on Linux Node 24 Testbox (CI truth); pnpm tsgo; oxlint clean; repo autoreview clean (its first pass caught an automatic-delivery wording mismatch, fixed before commit).

@openclaw-barnacle openclaw-barnacle Bot added agents Agent runtime and tooling size: S maintainer Maintainer-authored PR labels Jul 2, 2026
@clawsweeper

clawsweeper Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs real behavior proof before merge. Reviewed July 2, 2026, 11:53 AM ET / 15:53 UTC.

Summary
The branch changes agent and auto-reply prompt text plus a plugin SDK delivery-hint constant so message-tool-only room-event turns are framed as default-silent, with updated prompt tests and a prompt snapshot.

PR surface: Source 0, Tests +44. Total +44 across 18 files.

Reproducibility: yes. from source. Current main contradicts the ambient room-event docs by prompting the model to decide whether to act, using a reply-presupposing message-tool hint, and retaining a generic status-update visibility claim for message-tool-only prompts.

Review metrics: 1 noteworthy metric.

  • Public SDK constants: 1 added. The branch adds a named export on an existing public plugin SDK subpath, so maintainers should verify the additive contract is intentional.

Stored data model
Persistent data-model change detected: vector/embedding metadata: src/auto-reply/reply/inbound-meta.test.ts. Confirm migration or upgrade compatibility proof before merge.

Root-cause cluster
Relationship: fixed_by_candidate
Canonical: #99142
Summary: This PR is one candidate fix for the open ambient group-mode issue; two sibling PRs address adjacent root-cause slices in Telegram history and notice-leak handling.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge readiness
Overall: 🦐 gold shrimp
Proof: 🌊 off-meta tidepool
Patch quality: 🦐 gold shrimp
Result: ready for maintainer review.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • Align the generic message-tool-only system prompt with room-event silence semantics.
  • Inspect or rerun the failing checks-node-compact-small-whole-2 CI shard.

Mantis proof suggestion
Native Telegram proof would materially help reviewers see the user-visible silence contract this prompt change is meant to produce. A maintainer can ask Mantis to capture proof by posting this exact PR comment:

@openclaw-mantis telegram desktop proof: verify that an unmentioned ambient Telegram group room event stays silent unless the agent explicitly sends with message(action=send).

Risk before merge

  • [P1] The branch leaves a model-visible contradiction unless the generic message-tool-only system prompt stops telling room-event turns that brief status updates are visible.
  • [P1] The PR changes prompt guidance that can alter when group/channel agents post visible messages; CI and unit tests cannot fully prove the LLM-level message-delivery effect.
  • [P1] The live PR has one failing CI shard, checks-node-compact-small-whole-2, whose log was not inspected in this read-only pass.

Maintainer options:

  1. Fix the remaining prompt contradiction (recommended)
    Make the generic message-tool-only system prompt truthful for room-event runs, either by passing the inbound event kind into the prompt builder or by removing the status-visibility claim from the generic line.
  2. Accept prompt-only behavioral risk
    Maintainers can intentionally land this as a partial prompt-posture fix if they accept that the generic status sentence remains for now and is tracked separately.
  3. Hold for the coordinated ambient batch
    Pause this PR until the related history and notice-leak PRs are reviewed together so the final ambient behavior is judged as one user-facing change.

Next step before merge

  • [P1] The PR has a protected maintainer label and a prompt-direction finding that needs maintainer choice on whether to make the system prompt event-aware or adjust the generic message-tool-only wording.

Security
Cleared: The diff changes prompt strings, tests, and one additive SDK constant; it does not touch dependencies, workflows, secrets, package resolution, or code-execution paths.

Review findings

  • [P2] Make room-event status visibility truthful — src/agents/system-prompt.ts:538-543
Review details

Best possible solution:

Align all model-bound room-event and message-tool-only prompt surfaces with the documented quiet-context contract, then review the coordinated sibling PRs for the linked ambient group-mode issue together.

Do we have a high-confidence way to reproduce the issue?

Yes from source. Current main contradicts the ambient room-event docs by prompting the model to decide whether to act, using a reply-presupposing message-tool hint, and retaining a generic status-update visibility claim for message-tool-only prompts.

Is this the best way to solve the issue?

No. The patch is in the right layer and fixes the per-turn envelope and inbound delivery hint, but it should also remove or specialize the generic message-tool-only status-update sentence before merge.

Full review comments:

  • [P2] Make room-event status visibility truthful — src/agents/system-prompt.ts:538-543
    This branch adds a room-event delivery hint saying no visible reply is expected and anything else stays private, but the same message-tool-only run still gets the generic Messaging section that says brief status updates between tool calls are visible. Since buildCliAgentSystemPrompt is called with sourceReplyDeliveryMode but no inbound event kind, room-event turns can still see contradictory status guidance; please either pass the event kind through and render a room-event-specific line, or remove the visibility claim from the generic message-tool-only text.
    Confidence: 0.78

Overall correctness: patch is incorrect
Overall confidence: 0.78

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against f8e1e0c5bfbc.

Label changes

Label changes:

  • add P1: The linked production report describes ambient group agents visibly over-replying in real rooms, which is a broken channel workflow affecting users now.
  • add merge-risk: 🚨 message-delivery: Prompt changes can suppress or encourage visible group/channel messages in message-tool-only turns, and green CI cannot fully prove the runtime LLM delivery behavior.
  • add rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🌊 off-meta tidepool and patch quality is 🦐 gold shrimp.
  • add status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Not applicable: The live PR carries the protected maintainer label, so the external contributor real-behavior proof gate is not applied; the PR body still reports targeted tests and Testbox validation.
  • add mantis: telegram-visible-proof: Mantis should capture Telegram visible proof. The linked bug is Telegram-visible ambient group behavior, and a short Telegram Desktop proof could show an unmentioned room event staying silent unless the agent explicitly sends.

Label justifications:

  • P1: The linked production report describes ambient group agents visibly over-replying in real rooms, which is a broken channel workflow affecting users now.
  • merge-risk: 🚨 message-delivery: Prompt changes can suppress or encourage visible group/channel messages in message-tool-only turns, and green CI cannot fully prove the runtime LLM delivery behavior.
  • rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🌊 off-meta tidepool and patch quality is 🦐 gold shrimp.
  • status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Not applicable: The live PR carries the protected maintainer label, so the external contributor real-behavior proof gate is not applied; the PR body still reports targeted tests and Testbox validation.
  • mantis: telegram-visible-proof: Mantis should capture Telegram visible proof. The linked bug is Telegram-visible ambient group behavior, and a short Telegram Desktop proof could show an unmentioned room event staying silent unless the agent explicitly sends.
Evidence reviewed

PR surface:

Source 0, Tests +44. Total +44 across 18 files.

View PR surface stats
Area Files Added Removed Net
Source 8 26 26 0
Tests 10 72 28 +44
Docs 0 0 0 0
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 18 98 54 +44

What I checked:

  • Repository policy read: Root and relevant scoped AGENTS.md files were read; root policy requires whole-surface PR review, SDK-boundary scrutiny, and protected-label handling, while scoped guides cover agents, embedded runner tests, agent tools, plugin SDK, and test helpers. (AGENTS.md:1, f8e1e0c5bfbc)
  • Live PR metadata: Live GitHub metadata shows the PR is open, mergeable, labeled maintainer, and linked to the broader ambient group-mode issue, so this cleanup workflow must keep it open rather than auto-close it. (b32e0a20b3b2)
  • Current main room-event envelope: Current main still tells room-event turns Treat this as observed room activity. Decide whether to act., which matches the bug described in the PR body and the linked issue. (src/auto-reply/reply/prompt-prelude.ts:156, f8e1e0c5bfbc)
  • Current main inbound hint: Current main always prepends MESSAGE_TOOL_ONLY_DELIVERY_HINT for message-tool-only inbound context, with no room-event-specific hint. (src/auto-reply/reply/inbound-meta.ts:554, f8e1e0c5bfbc)
  • Ambient docs contract: The public docs state ambient room events are quiet context and visible room output requires message(action=send), supporting the default-silent direction of the PR. Public docs: docs/channels/ambient-room-events.md. (docs/channels/ambient-room-events.md:11, f8e1e0c5bfbc)
  • Generic system-prompt contradiction: The generic message-tool-only Messaging section still says brief high-level status updates between tool calls are visible, which can still reach room-event message-tool-only runs. (src/agents/system-prompt.ts:526, f8e1e0c5bfbc)

Likely related people:

  • obviyus: Recent history includes the delivery-hints SDK exposure and message-tool progress-gating work, and this author also opened the linked coordinated ambient room-event issue and PRs. (role: recent area contributor; confidence: high; commits: d498b1cce49c, cf88b4c0241a; files: src/plugin-sdk/message-tool-delivery-hints.ts, src/auto-reply/reply/inbound-meta.ts, src/agents/system-prompt.ts)
  • ragesaq: Recent GitHub history attributes the message-tool-only final reply behavior change to this author, directly adjacent to the prompt contract this PR edits. (role: adjacent delivery behavior contributor; confidence: medium; commits: a15427d605fe; files: src/agents/system-prompt.ts, src/auto-reply/reply/inbound-meta.ts)
  • arkyu2077: Recent GitHub history shows this contributor changed group/channel intro and tool-only channel guidance, which overlaps the group prompt surface edited here. (role: adjacent group prompt contributor; confidence: medium; commits: 9e3a917d9ee0; files: src/auto-reply/reply/groups.ts)
  • steipete: Recent path history shows documentation and auto-reply routing/helper work around the same prompt and auto-reply internals. (role: recent area contributor; confidence: medium; commits: 82e5dd4da74f, afeab32780ef; files: src/auto-reply/reply/prompt-prelude.ts, src/auto-reply/reply/inbound-meta.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@clawsweeper clawsweeper Bot added rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. mantis: telegram-visible-proof Mantis should capture Telegram visible proof. P1 High-priority user-facing bug, regression, or broken workflow. merge-risk: 🚨 message-delivery 🚨 May drop, duplicate, misroute, suppress, or wrongly target messages. labels Jul 2, 2026
@clawsweeper clawsweeper Bot temporarily deployed to qa-live-shared July 2, 2026 15:56 Inactive
@obviyus obviyus force-pushed the codex/fix-ambient-prompt branch from b32e0a2 to ba51c1d Compare July 2, 2026 15:58
@obviyus obviyus merged commit e081028 into main Jul 2, 2026
97 checks passed
@obviyus obviyus deleted the codex/fix-ambient-prompt branch July 2, 2026 16:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling maintainer Maintainer-authored PR mantis: telegram-visible-proof Mantis should capture Telegram visible proof. merge-risk: 🚨 message-delivery 🚨 May drop, duplicate, misroute, suppress, or wrongly target messages. P1 High-priority user-facing bug, regression, or broken workflow. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. size: S status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant