Skip to content

ACP harness: agents silently fail to publish responses (no safety net for missed buzz messages send) #4709

Description

@azure5100

Problem

Agents running under the ACP harness sometimes produce complete, correct responses but fail to publish them to the Nostr relay — the response exists in local stdout but never reaches the channel. Other users see the agent as unresponsive or "thinking forever."

Root cause

The current architecture relies solely on the system prompt (base_prompt.md) to instruct agents to call buzz messages send. There is no harness-level safety net. When an LLM agent, deep in a complex multi-step reasoning chain, overlooks the final publish step, the entire turn's output is silently lost.

User message → relay → ACP harness → session/prompt → Agent runs
                                                         ↓
                                              Generates response
                                              (reasoning + tools + text)
                                                         ↓
                                              ❌ Forgets buzz messages send
                                                         ↓
                                              Response lost — nothing on relay

This is a structural gap, not a per-agent bug. Every agent type (Claude Code, Codex, Grok, OpenCode) is susceptible.

Proposed fix (three layers)

Layer Scope Location Effort
L1 Single agent Agent core memory (buzz mem set core) Immediate, one command
L2 All agents base_prompt.md — "first and last" reinforcement Already implemented (see below)
L3 Platform (root fix) buzz-acp harness — post-turn publish check ~50-100 lines Rust

L2: base_prompt.md changes (already implemented locally)

Two additions to crates/buzz-acp/src/base_prompt.md:

1. Top of file — A CRITICAL banner right after the intro, before any other content:

**CRITICAL — Read this first:** Your reasoning and tool calls are invisible to
other users. The ONLY way anyone sees your response is if you call
`buzz messages send`. A result, answer, or deliverable that you generate but
don't publish does not exist. Every turn that produces something worth
communicating MUST end with `buzz messages send`. No exceptions.

2. End of file — A "Before Ending Your Turn" sentinel with checklist:

## Before Ending Your Turn

**STOP. Ask yourself:** Did I produce a result, answer, deliverable, decision,
or blocker that someone needs to see?

If yes — **you MUST call `buzz messages send` before ending this turn.**
...
Checklist:
1. Was the user waiting on an answer? → buzz messages send
2. Did you finish delegated work? → buzz messages send (with callback @mention)
3. Did you find something worth reporting? → buzz messages send
4. Are you blocked and need input? → buzz messages send
5. Nothing to communicate? → Silence is correct. End without sending.

The design intent is "first and last" — the agent sees the CRITICAL banner immediately on session start, and is intercepted by the checklist before ending a turn.

L3: Harness-level safety net (proposed design, not yet implemented)

The ACP harness already knows the target channel UUID (it's in the session context). After session/prompt returns with EndTurn, the harness should:

  1. Query the relay: did the agent publish any events to the target channel since the turn started?
  2. If no events were published AND the agent produced stdout output → log a warning + optionally auto-publish a fallback message
  3. This catches the "agent produced output but forgot to publish" case regardless of agent type

Pseudo-code location: around crates/buzz-acp/src/acp.rs session_prompt_with_idle_timeout return path, or in the pool dispatch layer in pool.rs.

Impact assessment

  • Without this fix: Silent failures erode trust in agent reliability across the platform
  • L2 alone: Reduces but doesn't eliminate the failure mode (prompt-based, not enforced)
  • L2 + L3: Prompt reinforcement + infrastructure guarantee = defense in depth

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions